2025-11-01 12:45:07 [scrapy.utils.log] (PID: 119) INFO: Scrapy 2.12.0 started (bot: catalog_extraction)
2025-11-01 12:45:07 [scrapy.utils.log] (PID: 119) INFO: Versions: lxml 5.3.1.0, libxml2 2.12.9, cssselect 1.3.0, parsel 1.10.0, w3lib 2.3.1, Twisted 24.11.0, Python 3.11.13 (main, Jun 10 2025, 23:54:42) [GCC 12.2.0], pyOpenSSL 25.0.0 (OpenSSL 3.4.1 11 Feb 2025), cryptography 44.0.2, Platform Linux-6.9.12-x86_64-with-glibc2.36
2025-11-01 12:45:07 [ecoenclose] (PID: 119) INFO: Starting extraction spider ecoenclose...
2025-11-01 12:45:07 [scrapy.addons] (PID: 119) INFO: Enabled addons:
[]
2025-11-01 12:45:07 [py.warnings] (PID: 119) WARNING: /usr/local/lib/python3.11/site-packages/scrapy/utils/request.py:120: ScrapyDeprecationWarning: 'REQUEST_FINGERPRINTER_IMPLEMENTATION' is a deprecated setting.
It will be removed in a future version of Scrapy.
  return cls(crawler)

2025-11-01 12:45:07 [scrapy.extensions.telnet] (PID: 119) INFO: Telnet Password: 01e3f82e3b84dd30
2025-11-01 12:45:07 [py.warnings] (PID: 119) WARNING: /var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/extensions/bq_feedstorage.py:33: ScrapyDeprecationWarning: scrapy.extensions.feedexport.build_storage() is deprecated, call the builder directly.

2025-11-01 12:45:07 [scrapy.middleware] (PID: 119) INFO: Enabled extensions:
['scrapy.extensions.corestats.CoreStats',
 'scrapy.extensions.telnet.TelnetConsole',
 'scrapy.extensions.memusage.MemoryUsage',
 'scrapy.extensions.closespider.CloseSpider',
 'scrapy.extensions.feedexport.FeedExporter',
 'scrapy.extensions.logstats.LogStats',
 'spidermon.contrib.scrapy.extensions.Spidermon']
2025-11-01 12:45:07 [scrapy.crawler] (PID: 119) INFO: Overridden settings:
{'BOT_NAME': 'catalog_extraction',
 'CONCURRENT_ITEMS': 250,
 'CONCURRENT_REQUESTS': 24,
 'FEED_EXPORT_ENCODING': 'utf-8',
 'LOG_FILE': '/var/lib/scrapyd/logs/catalog_extraction/ecoenclose/950a4ca4b72011f0aeb54200a9fe0102.log',
 'LOG_FORMAT': '%(asctime)s [%(name)s] (PID: %(process)d) %(levelname)s: '
               '%(message)s',
 'LOG_LEVEL': 'INFO',
 'NEWSPIDER_MODULE': 'catalog_extraction.spiders',
 'REQUEST_FINGERPRINTER_CLASS': 'scrapy_poet.ScrapyPoetRequestFingerprinter',
 'REQUEST_FINGERPRINTER_IMPLEMENTATION': '2.7',
 'SPIDER_MODULES': ['catalog_extraction.spiders'],
 'TWISTED_REACTOR': 'twisted.internet.asyncioreactor.AsyncioSelectorReactor',
 'USER_AGENT': None}
2025-11-01 12:45:08 [scrapy_poet.injection] (PID: 119) INFO: Loading providers:
 [<class 'scrapy_poet.page_input_providers.HttpRequestProvider'>,
 <class 'scrapy_poet.page_input_providers.HttpResponseProvider'>,
 <class 'scrapy_poet.page_input_providers.HttpClientProvider'>,
 <class 'scrapy_poet.page_input_providers.PageParamsProvider'>,
 <class 'scrapy_poet.page_input_providers.RequestUrlProvider'>,
 <class 'scrapy_poet.page_input_providers.ResponseUrlProvider'>,
 <class 'scrapy_poet.page_input_providers.StatsProvider'>]
2025-11-01 12:45:08 [scrapy.middleware] (PID: 119) INFO: Enabled downloader middlewares:
['scrapy.downloadermiddlewares.offsite.OffsiteMiddleware',
 'scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
 'scraping_utils.middlewares.downloaders.ProxyManagerDownloaderMiddleware',
 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
 'scraping_utils.middlewares.downloaders.HeadersSpooferDownloaderMiddleware',
 'scrapy_poet.InjectionMiddleware',
 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
 'scrapy_poet.DownloaderStatsMiddleware']
2025-11-01 12:45:08 [NotFoundHandlerSpiderMiddleware] (PID: 119) INFO: NotFoundHandlerSpiderMiddleware running on PRODUCTION environment.
2025-11-01 12:45:08 [scrapy.middleware] (PID: 119) INFO: Enabled spider middlewares:
['catalog_extraction.middlewares.NotFoundHandlerSpiderMiddleware',
 'catalog_extraction.middlewares.FixtureSavingMiddleware',
 'scrapy_poet.RetryMiddleware',
 'scrapy.spidermiddlewares.referer.RefererMiddleware',
 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
 'scrapy.spidermiddlewares.depth.DepthMiddleware']
2025-11-01 12:45:08 [scrapy.middleware] (PID: 119) INFO: Enabled item pipelines:
['catalog_extraction.pipelines.DuplicatedSKUsFilterPipeline',
 'catalog_extraction.pipelines.DiscontinuedProductsAdjustmentPipeline',
 'catalog_extraction.pipelines.PriceRoundingPipeline',
 'scraping_utils.pipelines.AttachSupplierPipeline',
 'spidermon.contrib.scrapy.pipelines.ItemValidationPipeline']
2025-11-01 12:45:08 [scrapy.core.engine] (PID: 119) INFO: Spider opened
2025-11-01 12:45:08 [scrapy.extensions.closespider] (PID: 119) INFO: Spider will stop when no items are produced after 1800 seconds.
2025-11-01 12:45:08 [scrapy.extensions.logstats] (PID: 119) INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2025-11-01 12:45:08 [scrapy.extensions.telnet] (PID: 119) INFO: Telnet console listening on 127.0.0.1:6026
2025-11-01 12:45:09 [ProxyManagerDownloaderMiddleware] (PID: 119) INFO: Using brd-customer-hl_13cda1e4-zone-main_datacenter_proxy as the default proxy for ProxyManagerDownloaderMiddleware.
2025-11-01 12:45:09 [scrapy.core.scraper] (PID: 119) ERROR: Error downloading <GET https://www.ecoenclose.com/cart.php?action=add&product_id=2322>
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/twisted/internet/defer.py", line 2017, in _inlineCallbacks
    result = context.run(gen.send, result)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/scrapy/core/downloader/middleware.py", line 57, in process_request
    method(request=request, spider=spider)
  File "/usr/local/lib/python3.11/site-packages/scraping_utils/middlewares/downloaders/spoofer.py", line 101, in process_request
    generated_headers = self.generator.generate()
                        ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/browserforge/headers/generator.py", line 159, in generate
    generated: Dict[str, str] = self._get_headers(
                                ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/browserforge/headers/generator.py", line 267, in _get_headers
    return self.order_headers({**generated_sample, **request_dependent_headers})
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/browserforge/headers/generator.py", line 357, in order_headers
    raise ValueError("Failed to find User-Agent in generated response")
ValueError: Failed to find User-Agent in generated response
2025-11-01 12:45:09 [scrapy.spidermiddlewares.httperror] (PID: 119) INFO: Ignoring response <403 https://www.ecoenclose.com/decorative-100-recycled-tissue-paper-thoughtfully-packaged-print-20-x-30/>: HTTP status code is not handled or not allowed
2025-11-01 12:45:10 [scrapy.spidermiddlewares.httperror] (PID: 119) INFO: Ignoring response <403 https://www.ecoenclose.com/non-reinforced-water-activated-tape-3-x-600-ft/>: HTTP status code is not handled or not allowed
2025-11-01 12:45:10 [scrapy.spidermiddlewares.httperror] (PID: 119) INFO: Ignoring response <403 https://www.ecoenclose.com/kraft-flatback-carton-sealing-tape-2-x-60-yds/>: HTTP status code is not handled or not allowed
2025-11-01 12:45:10 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/ecox-mailer-this-is-algae-ink-algae-ink-10-x-13-case-of-250-mailers/ already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:10 [scrapy.spidermiddlewares.httperror] (PID: 119) INFO: Ignoring response <403 https://www.ecoenclose.com/enterprise-quote>: HTTP status code is not handled or not allowed
2025-11-01 12:45:10 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/shop/flexi-hex/ already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:10 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:10 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:10 [scrapy.core.scraper] (PID: 119) ERROR: Spider error processing <GET https://www.ecoenclose.com/decorative-100-recycled-tissue-paper-blue-stripes-print-20-x-30/> (referer: None)
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
    result = method(response=response, spider=spider)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input
    raise ProductNotFound(f"Page {response.url} returned 404 status code.")
scraping_utils.common.exceptions.ProductNotFound: Page https://www.ecoenclose.com/decorative-100-recycled-tissue-paper-blue-stripes-print-20-x-30/ returned 404 status code.
2025-11-01 12:45:10 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:11 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/ecox-mailer-this-is-algae-ink-algae-ink-12-x-15-case-of-250-mailers/ already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:11 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/cello-tape-eco-friendly-packing-tape-industrially-compostable-2-x-72-yds/ already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:11 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:11 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:11 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:11 [scrapy.core.scraper] (PID: 119) ERROR: Spider error processing <GET https://www.ecoenclose.com/we-care-cards-with-qr-code-algae-info/> (referer: None)
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
    result = method(response=response, spider=spider)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input
    raise ProductNotFound(f"Page {response.url} returned 404 status code.")
scraping_utils.common.exceptions.ProductNotFound: Page https://www.ecoenclose.com/we-care-cards-with-qr-code-algae-info/ returned 404 status code.
2025-11-01 12:45:11 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:11 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/shop/ecox-mailers/ already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:11 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/cart.php?suggest=7e130123-7d5a-4e33-8fe0-93c4572258a4 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:11 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:12 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:12 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:12 [scrapy.core.scraper] (PID: 119) ERROR: Error downloading <POST https://www.ecoenclose.com/graphql>
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/twisted/internet/defer.py", line 2017, in _inlineCallbacks
    result = context.run(gen.send, result)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/scrapy/core/downloader/middleware.py", line 57, in process_request
    method(request=request, spider=spider)
  File "/usr/local/lib/python3.11/site-packages/scraping_utils/middlewares/downloaders/spoofer.py", line 101, in process_request
    generated_headers = self.generator.generate()
                        ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/browserforge/headers/generator.py", line 159, in generate
    generated: Dict[str, str] = self._get_headers(
                                ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/browserforge/headers/generator.py", line 267, in _get_headers
    return self.order_headers({**generated_sample, **request_dependent_headers})
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/browserforge/headers/generator.py", line 357, in order_headers
    raise ValueError("Failed to find User-Agent in generated response")
ValueError: Failed to find User-Agent in generated response
2025-11-01 12:45:12 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:12 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/cart.php?suggest=8e551f1f-ecb4-4ca2-9c41-fc999241a2ce already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:12 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:12 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/shop/ecox-mailers/ already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:12 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:12 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/2-5-zero-waste-sticker-we-care-12-stickers-per-sheet/ already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:12 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/ecox-mailer-packaging-for-the-planet-algae-ink-10-x-13-case-of-250-mailers/ already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:13 [scrapy.core.scraper] (PID: 119) ERROR: Error downloading <POST https://www.ecoenclose.com/graphql>
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/twisted/internet/defer.py", line 2017, in _inlineCallbacks
    result = context.run(gen.send, result)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/scrapy/core/downloader/middleware.py", line 57, in process_request
    method(request=request, spider=spider)
  File "/usr/local/lib/python3.11/site-packages/scraping_utils/middlewares/downloaders/spoofer.py", line 101, in process_request
    generated_headers = self.generator.generate()
                        ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/browserforge/headers/generator.py", line 159, in generate
    generated: Dict[str, str] = self._get_headers(
                                ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/browserforge/headers/generator.py", line 267, in _get_headers
    return self.order_headers({**generated_sample, **request_dependent_headers})
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/browserforge/headers/generator.py", line 357, in order_headers
    raise ValueError("Failed to find User-Agent in generated response")
ValueError: Failed to find User-Agent in generated response
2025-11-01 12:45:13 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/shop/protective-packaging-void-fill/ already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:13 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/shop/spiropack-nest-eco/ already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:13 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:13 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:13 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:14 [scrapy.core.scraper] (PID: 119) ERROR: Error downloading <GET https://www.ecoenclose.com/resources/take-back-programs>
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/twisted/internet/defer.py", line 2017, in _inlineCallbacks
    result = context.run(gen.send, result)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/scrapy/core/downloader/middleware.py", line 57, in process_request
    method(request=request, spider=spider)
  File "/usr/local/lib/python3.11/site-packages/scraping_utils/middlewares/downloaders/spoofer.py", line 101, in process_request
    generated_headers = self.generator.generate()
                        ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/browserforge/headers/generator.py", line 159, in generate
    generated: Dict[str, str] = self._get_headers(
                                ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/browserforge/headers/generator.py", line 267, in _get_headers
    return self.order_headers({**generated_sample, **request_dependent_headers})
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/browserforge/headers/generator.py", line 357, in order_headers
    raise ValueError("Failed to find User-Agent in generated response")
ValueError: Failed to find User-Agent in generated response
2025-11-01 12:45:14 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/shop/ecox-mailers/ already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:14 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:14 [scrapy.core.scraper] (PID: 119) ERROR: Spider error processing <GET https://www.ecoenclose.com/2-5-zero-waste-sticker-we-care-12-stickers-per-sheet/> (referer: None)
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
    result = method(response=response, spider=spider)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input
    raise ProductNotFound(f"Page {response.url} returned 404 status code.")
scraping_utils.common.exceptions.ProductNotFound: Page https://www.ecoenclose.com/2-5-zero-waste-sticker-we-care-12-stickers-per-sheet/ returned 404 status code.
2025-11-01 12:45:14 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/cart.php?suggest=d3d0c95d-aca2-459b-83d8-81033ae33e78 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:14 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/cart.php already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:14 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/clearance-100-recycled-poly-mailers-trial-run-rcs100-10x13/ already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:14 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:14 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:14 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:15 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/cart.php already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:15 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/sway-packaging already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:15 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:15 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/cart.php?suggest=34e16daa-57df-4428-b7d6-3e3eea769421 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:15 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:15 [scrapy.core.scraper] (PID: 119) ERROR: Spider error processing <GET https://www.ecoenclose.com/100-recycled-retail-box-with-seaweed-window-square-4-x-4-x-4/> (referer: None)
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
    result = method(response=response, spider=spider)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input
    raise ProductNotFound(f"Page {response.url} returned 404 status code.")
scraping_utils.common.exceptions.ProductNotFound: Page https://www.ecoenclose.com/100-recycled-retail-box-with-seaweed-window-square-4-x-4-x-4/ returned 404 status code.
2025-11-01 12:45:15 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/cart.php?suggest=a2f424f4-8ded-4b69-a246-ca3be02208f8 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:15 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/cart.php?suggest=d394d877-364a-4808-8a49-c643a1906bbc already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:16 [scrapy.core.scraper] (PID: 119) ERROR: Error downloading <GET https://www.ecoenclose.com/2-5-zero-waste-sticker-green-is-the-new-black/>
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/twisted/internet/defer.py", line 2017, in _inlineCallbacks
    result = context.run(gen.send, result)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/scrapy/core/downloader/middleware.py", line 57, in process_request
    method(request=request, spider=spider)
  File "/usr/local/lib/python3.11/site-packages/scraping_utils/middlewares/downloaders/spoofer.py", line 101, in process_request
    generated_headers = self.generator.generate()
                        ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/browserforge/headers/generator.py", line 159, in generate
    generated: Dict[str, str] = self._get_headers(
                                ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/browserforge/headers/generator.py", line 267, in _get_headers
    return self.order_headers({**generated_sample, **request_dependent_headers})
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/browserforge/headers/generator.py", line 357, in order_headers
    raise ValueError("Failed to find User-Agent in generated response")
ValueError: Failed to find User-Agent in generated response
2025-11-01 12:45:16 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:16 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:16 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:16 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:16 [scrapy.core.scraper] (PID: 119) ERROR: Spider error processing <GET https://www.ecoenclose.com/clearance-100-recycled-poly-mailers-trial-run-rcs100-10x13/> (referer: None)
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
    result = method(response=response, spider=spider)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input
    raise ProductNotFound(f"Page {response.url} returned 404 status code.")
scraping_utils.common.exceptions.ProductNotFound: Page https://www.ecoenclose.com/clearance-100-recycled-poly-mailers-trial-run-rcs100-10x13/ returned 404 status code.
2025-11-01 12:45:16 [scrapy.core.scraper] (PID: 119) ERROR: Error downloading <GET https://www.linkedin.com/company/ecoenclose?ref=01H9V3QCWSYAE34TZE5107VNM1>
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/twisted/internet/defer.py", line 2017, in _inlineCallbacks
    result = context.run(gen.send, result)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/scrapy/core/downloader/middleware.py", line 57, in process_request
    method(request=request, spider=spider)
  File "/usr/local/lib/python3.11/site-packages/scraping_utils/middlewares/downloaders/spoofer.py", line 101, in process_request
    generated_headers = self.generator.generate()
                        ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/browserforge/headers/generator.py", line 159, in generate
    generated: Dict[str, str] = self._get_headers(
                                ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/browserforge/headers/generator.py", line 267, in _get_headers
    return self.order_headers({**generated_sample, **request_dependent_headers})
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/browserforge/headers/generator.py", line 357, in order_headers
    raise ValueError("Failed to find User-Agent in generated response")
ValueError: Failed to find User-Agent in generated response
2025-11-01 12:45:16 [scrapy.core.scraper] (PID: 119) ERROR: Error downloading <GET https://www.ecoenclose.com/cart.php?suggest=c9d446d6-6d39-49be-a88c-022f0e33ffef>
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/twisted/internet/defer.py", line 2017, in _inlineCallbacks
    result = context.run(gen.send, result)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/scrapy/core/downloader/middleware.py", line 57, in process_request
    method(request=request, spider=spider)
  File "/usr/local/lib/python3.11/site-packages/scraping_utils/middlewares/downloaders/spoofer.py", line 101, in process_request
    generated_headers = self.generator.generate()
                        ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/browserforge/headers/generator.py", line 159, in generate
    generated: Dict[str, str] = self._get_headers(
                                ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/browserforge/headers/generator.py", line 267, in _get_headers
    return self.order_headers({**generated_sample, **request_dependent_headers})
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/browserforge/headers/generator.py", line 357, in order_headers
    raise ValueError("Failed to find User-Agent in generated response")
ValueError: Failed to find User-Agent in generated response
2025-11-01 12:45:16 [scrapy.core.scraper] (PID: 119) ERROR: Error downloading <POST https://www.ecoenclose.com/graphql>
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/twisted/internet/defer.py", line 2017, in _inlineCallbacks
    result = context.run(gen.send, result)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/scrapy/core/downloader/middleware.py", line 57, in process_request
    method(request=request, spider=spider)
  File "/usr/local/lib/python3.11/site-packages/scraping_utils/middlewares/downloaders/spoofer.py", line 101, in process_request
    generated_headers = self.generator.generate()
                        ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/browserforge/headers/generator.py", line 159, in generate
    generated: Dict[str, str] = self._get_headers(
                                ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/browserforge/headers/generator.py", line 267, in _get_headers
    return self.order_headers({**generated_sample, **request_dependent_headers})
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/browserforge/headers/generator.py", line 357, in order_headers
    raise ValueError("Failed to find User-Agent in generated response")
ValueError: Failed to find User-Agent in generated response
2025-11-01 12:45:17 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:17 [scrapy.core.scraper] (PID: 119) ERROR: Spider error processing <GET https://www.ecoenclose.com/100-recycled-retail-box-with-seaweed-window-gable-top-5-x-2-5-x-7/> (referer: None)
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
    result = method(response=response, spider=spider)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input
    raise ProductNotFound(f"Page {response.url} returned 404 status code.")
scraping_utils.common.exceptions.ProductNotFound: Page https://www.ecoenclose.com/100-recycled-retail-box-with-seaweed-window-gable-top-5-x-2-5-x-7/ returned 404 status code.
2025-11-01 12:45:17 [scrapy.core.scraper] (PID: 119) ERROR: Error downloading <GET https://www.ecoenclose.com/100-recycled-ecox-mailer-packaging-for-the-planet-algae-ink-12-x-15/>
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/twisted/internet/defer.py", line 2017, in _inlineCallbacks
    result = context.run(gen.send, result)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/scrapy/core/downloader/middleware.py", line 57, in process_request
    method(request=request, spider=spider)
  File "/usr/local/lib/python3.11/site-packages/scraping_utils/middlewares/downloaders/spoofer.py", line 101, in process_request
    generated_headers = self.generator.generate()
                        ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/browserforge/headers/generator.py", line 159, in generate
    generated: Dict[str, str] = self._get_headers(
                                ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/browserforge/headers/generator.py", line 267, in _get_headers
    return self.order_headers({**generated_sample, **request_dependent_headers})
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/browserforge/headers/generator.py", line 357, in order_headers
    raise ValueError("Failed to find User-Agent in generated response")
ValueError: Failed to find User-Agent in generated response
2025-11-01 12:45:18 [scrapy.core.scraper] (PID: 119) ERROR: Spider error processing <GET https://www.ecoenclose.com/ecogeek-t-shirt-printed-with-algae-ink/> (referer: None)
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
    result = method(response=response, spider=spider)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input
    raise ProductNotFound(f"Page {response.url} returned 404 status code.")
scraping_utils.common.exceptions.ProductNotFound: Page https://www.ecoenclose.com/ecogeek-t-shirt-printed-with-algae-ink/ returned 404 status code.
2025-11-01 12:45:18 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/shop/ already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:18 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:18 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:18 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/reenclose-reusable-mailers-100-recycled-polyester-bundle-of-10/ already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:18 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/shop/ already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:18 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:18 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:18 [scrapy.core.scraper] (PID: 119) ERROR: Spider error processing <GET https://www.ecoenclose.com/we-care-cards-with-qr-code-algae-ink/> (referer: None)
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
    result = method(response=response, spider=spider)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input
    raise ProductNotFound(f"Page {response.url} returned 404 status code.")
scraping_utils.common.exceptions.ProductNotFound: Page https://www.ecoenclose.com/we-care-cards-with-qr-code-algae-ink/ returned 404 status code.
2025-11-01 12:45:18 [scrapy.core.scraper] (PID: 119) ERROR: Error downloading <GET https://www.ecoenclose.com/shop/custom-shipping-boxes/>
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/twisted/internet/defer.py", line 2017, in _inlineCallbacks
    result = context.run(gen.send, result)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/scrapy/core/downloader/middleware.py", line 57, in process_request
    method(request=request, spider=spider)
  File "/usr/local/lib/python3.11/site-packages/scraping_utils/middlewares/downloaders/spoofer.py", line 101, in process_request
    generated_headers = self.generator.generate()
                        ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/browserforge/headers/generator.py", line 159, in generate
    generated: Dict[str, str] = self._get_headers(
                                ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/browserforge/headers/generator.py", line 267, in _get_headers
    return self.order_headers({**generated_sample, **request_dependent_headers})
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/browserforge/headers/generator.py", line 357, in order_headers
    raise ValueError("Failed to find User-Agent in generated response")
ValueError: Failed to find User-Agent in generated response
2025-11-01 12:45:18 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:18 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/cart.php?suggest=d81d05dd-ac13-4739-ae51-b2c6d0dc6dfd already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:19 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:19 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/enterprise already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:19 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:19 [scrapy.core.scraper] (PID: 119) ERROR: Error downloading <POST https://www.ecoenclose.com/graphql>
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/twisted/internet/defer.py", line 2017, in _inlineCallbacks
    result = context.run(gen.send, result)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/scrapy/core/downloader/middleware.py", line 57, in process_request
    method(request=request, spider=spider)
  File "/usr/local/lib/python3.11/site-packages/scraping_utils/middlewares/downloaders/spoofer.py", line 101, in process_request
    generated_headers = self.generator.generate()
                        ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/browserforge/headers/generator.py", line 159, in generate
    generated: Dict[str, str] = self._get_headers(
                                ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/browserforge/headers/generator.py", line 267, in _get_headers
    return self.order_headers({**generated_sample, **request_dependent_headers})
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/browserforge/headers/generator.py", line 357, in order_headers
    raise ValueError("Failed to find User-Agent in generated response")
ValueError: Failed to find User-Agent in generated response
2025-11-01 12:45:19 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:19 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:19 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:19 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/shop/reusable-mailers/ already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:19 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:20 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:20 [scrapy.core.scraper] (PID: 119) ERROR: Spider error processing <GET https://www.ecoenclose.com/reenclose-reusable-mailers-sample> (referer: None)
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
    result = method(response=response, spider=spider)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input
    raise ProductNotFound(f"Page {response.url} returned 404 status code.")
scraping_utils.common.exceptions.ProductNotFound: Page https://www.ecoenclose.com/reenclose-reusable-mailers-sample returned 404 status code.
2025-11-01 12:45:20 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/shop/pouches already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:20 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:20 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:21 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:21 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/samples/ already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:21 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:22 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/cart.php?suggest=3d1dfdfc-5e22-429d-aa80-6fa8a1d6cec7 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:22 [scrapy.core.scraper] (PID: 119) ERROR: Error downloading <POST https://www.ecoenclose.com/graphql>
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/twisted/internet/defer.py", line 2017, in _inlineCallbacks
    result = context.run(gen.send, result)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/scrapy/core/downloader/middleware.py", line 57, in process_request
    method(request=request, spider=spider)
  File "/usr/local/lib/python3.11/site-packages/scraping_utils/middlewares/downloaders/spoofer.py", line 101, in process_request
    generated_headers = self.generator.generate()
                        ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/browserforge/headers/generator.py", line 159, in generate
    generated: Dict[str, str] = self._get_headers(
                                ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/browserforge/headers/generator.py", line 267, in _get_headers
    return self.order_headers({**generated_sample, **request_dependent_headers})
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/browserforge/headers/generator.py", line 357, in order_headers
    raise ValueError("Failed to find User-Agent in generated response")
ValueError: Failed to find User-Agent in generated response
2025-11-01 12:45:22 [HeadersSpooferDownloaderMiddleware] (PID: 119) WARNING: Request https://www.ecoenclose.com/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS.
2025-11-01 12:45:22 [scrapy.core.scraper] (PID: 119) ERROR: Spider error processing <GET https://www.ecoenclose.com/we-care-cards-with-qr-code/> (referer: None)
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
    result = method(response=response, spider=spider)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input
    raise ProductNotFound(f"Page {response.url} returned 404 status code.")
scraping_utils.common.exceptions.ProductNotFound: Page https://www.ecoenclose.com/we-care-cards-with-qr-code/ returned 404 status code.
2025-11-01 12:45:22 [scrapy.core.scraper] (PID: 119) ERROR: Spider error processing <GET https://www.ecoenclose.com/samples/> (referer: None)
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback
    yield await it.__anext__()
          ^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__
    return await self.data.__anext__()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain
    async for o in as_async_generator(it):
  File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator
    async for r in it:
  File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__
    return await self.data.__anext__()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain
    async for o in as_async_generator(it):
  File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator
    async for r in it:
  File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async
    async for r in iterable:
  File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async
    async for r in result:
  File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async
    async for r in iterable:
  File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async
    async for r in result:
  File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async
    async for r in iterable:
  File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async
    async for r in result:
  File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async
    async for r in iterable:
  File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/ecoenclose.py", line 56, in parse
    product_ids = [int(item.get()) for item in response.css("div::attr(data-id)")]
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/ecoenclose.py", line 56, in <listcomp>
    product_ids = [int(item.get()) for item in response.css("div::attr(data-id)")]
                   ^^^^^^^^^^^^^^^
ValueError: invalid literal for int() with base 10: 'sample-kits-by-product'
2025-11-01 12:45:23 [scrapy.core.engine] (PID: 119) INFO: Closing spider (finished)
2025-11-01 12:45:23 [ecoenclose] (PID: 119) INFO: [Spidermon] ------------------------------ MONITORS ------------------------------
2025-11-01 12:45:23 [ecoenclose] (PID: 119) INFO: [Spidermon] Extracted Items Monitor/test_stat_monitor... OK
2025-11-01 12:45:23 [ecoenclose] (PID: 119) INFO: [Spidermon] Item Validation Monitor/test_stat_monitor... SKIPPED (Unable to find 'spidermon/validation/fields/errors' in job stats.)
2025-11-01 12:45:23 [ecoenclose] (PID: 119) INFO: [Spidermon] Error Count Monitor/test_stat_monitor... FAIL
2025-11-01 12:45:23 [ecoenclose] (PID: 119) INFO: [Spidermon] Warning Count Monitor/test_stat_monitor... OK
2025-11-01 12:45:23 [ecoenclose] (PID: 119) INFO: [Spidermon] Finish Reason Monitor/Should have the expected finished reason(s)... OK
2025-11-01 12:45:23 [ecoenclose] (PID: 119) INFO: [Spidermon] Unwanted HTTP codes monitor/Should not hit the limit of unwanted http status... OK
2025-11-01 12:45:23 [ecoenclose] (PID: 119) INFO: [Spidermon] Field Coverage Monitor/test_check_if_field_coverage_rules_are_met... OK
2025-11-01 12:45:23 [ecoenclose] (PID: 119) INFO: [Spidermon] Retry Count monitor/Should not hit the limit of requests that reached the maximum retry amount... OK
2025-11-01 12:45:23 [ecoenclose] (PID: 119) INFO: [Spidermon] Downloader Exceptions monitor/test_stat_monitor... OK
2025-11-01 12:45:23 [ecoenclose] (PID: 119) INFO: [Spidermon] Successful Requests monitor/Should have at least the minimum number of successful requests... OK
2025-11-01 12:45:23 [ecoenclose] (PID: 119) INFO: [Spidermon] Total Requests monitor/Should not hit the total limit of requests... OK
2025-11-01 12:45:23 [ecoenclose] (PID: 119) INFO: [Spidermon] ----------------------------------------------------------------------
2025-11-01 12:45:23 [ecoenclose] (PID: 119) ERROR: [Spidermon] 
======================================================================
FAIL: Error Count Monitor/test_stat_monitor
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/spidermon/contrib/scrapy/monitors/base.py", line 184, in test_stat_monitor
    assertion_method(
AssertionError: Expecting 'log_count/ERROR' to be '<=' to '3.0'. Current value: '23'

2025-11-01 12:45:23 [ecoenclose] (PID: 119) INFO: [Spidermon] 11 monitors in 0.004s
2025-11-01 12:45:23 [ecoenclose] (PID: 119) INFO: [Spidermon] FAILED (failures=1, skipped=1)
2025-11-01 12:45:23 [ecoenclose] (PID: 119) INFO: [Spidermon] -------------------------- FINISHED ACTIONS --------------------------
2025-11-01 12:45:23 [ecoenclose] (PID: 119) INFO: [Spidermon] ----------------------------------------------------------------------
2025-11-01 12:45:23 [ecoenclose] (PID: 119) INFO: [Spidermon] 0 actions in 0.000s
2025-11-01 12:45:23 [ecoenclose] (PID: 119) INFO: [Spidermon] OK
2025-11-01 12:45:23 [ecoenclose] (PID: 119) INFO: [Spidermon] --------------------------- PASSED ACTIONS ---------------------------
2025-11-01 12:45:23 [ecoenclose] (PID: 119) INFO: [Spidermon] ----------------------------------------------------------------------
2025-11-01 12:45:23 [ecoenclose] (PID: 119) INFO: [Spidermon] 0 actions in 0.000s
2025-11-01 12:45:23 [ecoenclose] (PID: 119) INFO: [Spidermon] OK
2025-11-01 12:45:23 [ecoenclose] (PID: 119) INFO: [Spidermon] --------------------------- FAILED ACTIONS ---------------------------
2025-11-01 12:45:23 [spidermon.contrib.actions.slack] (PID: 119) WARNING: bot cannot finder user in slack org member list - default icon url used
2025-11-01 12:45:23 [ecoenclose] (PID: 119) INFO: [Spidermon] CustomTemplateSendSlackMessageSpiderFinished... OK
2025-11-01 12:45:23 [ecoenclose] (PID: 119) INFO: [Spidermon] ----------------------------------------------------------------------
2025-11-01 12:45:23 [ecoenclose] (PID: 119) INFO: [Spidermon] 1 action in 0.450s
2025-11-01 12:45:23 [ecoenclose] (PID: 119) INFO: [Spidermon] OK
2025-11-01 12:45:23 [ecoenclose] (PID: 119) INFO: 10 URLs returned ProductNotFound.
2025-11-01 12:45:24 [ecoenclose] (PID: 119) INFO: URL https://www.ecoenclose.com/100-recycled-retail-box-with-seaweed-window-gable-top-5-x-2-5-x-7/ is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table.
2025-11-01 12:45:24 [ecoenclose] (PID: 119) INFO: URL https://www.ecoenclose.com/100-recycled-retail-box-with-seaweed-window-square-4-x-4-x-4/ is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table.
2025-11-01 12:45:24 [ecoenclose] (PID: 119) INFO: URL https://www.ecoenclose.com/decorative-100-recycled-tissue-paper-blue-stripes-print-20-x-30/ is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table.
2025-11-01 12:45:24 [ecoenclose] (PID: 119) INFO: URL https://www.ecoenclose.com/ecogeek-t-shirt-printed-with-algae-ink/ is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table.
2025-11-01 12:45:24 [ecoenclose] (PID: 119) INFO: URL https://www.ecoenclose.com/reenclose-reusable-mailers-sample is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table.
2025-11-01 12:45:24 [ecoenclose] (PID: 119) INFO: URL https://www.ecoenclose.com/we-care-cards-with-qr-code-algae-info/ is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table.
2025-11-01 12:45:24 [ecoenclose] (PID: 119) INFO: URL https://www.ecoenclose.com/we-care-cards-with-qr-code-algae-ink/ is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table.
2025-11-01 12:45:24 [ecoenclose] (PID: 119) INFO: URL https://www.ecoenclose.com/we-care-cards-with-qr-code/ is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table.
2025-11-01 12:45:24 [ecoenclose] (PID: 119) WARNING: 10 URLs were not found in the `catalog_urls` table.
2025-11-01 12:45:24 [ecoenclose] (PID: 119) INFO: URLs not found and not flagged: {'https://www.ecoenclose.com/100-recycled-retail-box-with-seaweed-window-square-4-x-4-x-4/', 'https://www.ecoenclose.com/decorative-100-recycled-tissue-paper-blue-stripes-print-20-x-30/', 'https://www.ecoenclose.com/we-care-cards-with-qr-code-algae-info/', 'https://www.ecoenclose.com/we-care-cards-with-qr-code-algae-ink/', 'https://www.ecoenclose.com/clearance-100-recycled-poly-mailers-trial-run-rcs100-10x13/', 'https://www.ecoenclose.com/2-5-zero-waste-sticker-we-care-12-stickers-per-sheet/', 'https://www.ecoenclose.com/ecogeek-t-shirt-printed-with-algae-ink/', 'https://www.ecoenclose.com/100-recycled-retail-box-with-seaweed-window-gable-top-5-x-2-5-x-7/', 'https://www.ecoenclose.com/we-care-cards-with-qr-code/', 'https://www.ecoenclose.com/reenclose-reusable-mailers-sample'}
2025-11-01 12:45:24 [ecoenclose] (PID: 119) INFO: Finished processing 'not found' URLs in the `catalog_urls` table.
2025-11-01 12:45:25 [scrapy.extensions.feedexport] (PID: 119) INFO: Stored bq feed (119 items) in: bq://response-elt.scraper_data.catalog_item_scrape/batch:1
2025-11-01 12:45:25 [scrapy.statscollectors] (PID: 119) INFO: Dumping Scrapy stats:
{'HeadersSpooferDownloaderMiddleware/spoofed': 176,
 'NotFoundHandlerSpiderMiddleware/HttpError': 4,
 'NotFoundHandlerSpiderMiddleware/ProductNotFound': 10,
 'NotFoundHandlerSpiderMiddleware/ValueError': 4,
 'NotFoundHandlerSpiderMiddleware/not_found/404_response': 10,
 'NotFoundHandlerSpiderMiddleware/not_found/ignored': 8,
 'NotFoundHandlerSpiderMiddleware/not_found/retrieved': 8,
 'big_query/url': 108,
 'downloader/exception_count': 12,
 'downloader/exception_type_count/builtins.ValueError': 12,
 'downloader/request_bytes': 390695,
 'downloader/request_count': 176,
 'downloader/request_method_count/GET': 132,
 'downloader/request_method_count/POST': 44,
 'downloader/response_bytes': 7478421,
 'downloader/response_count': 176,
 'downloader/response_status_count/200': 131,
 'downloader/response_status_count/301': 19,
 'downloader/response_status_count/302': 12,
 'downloader/response_status_count/403': 4,
 'downloader/response_status_count/404': 10,
 'dupefilter/filtered': 4,
 'elapsed_time_seconds': 15.082591,
 'feedexport/success_count/BigQueryFeedStorage': 1,
 'finish_reason': 'finished',
 'finish_time': datetime.datetime(2025, 11, 1, 12, 45, 23, 265553, tzinfo=datetime.timezone.utc),
 'httpcompression/response_bytes': 31248966,
 'httpcompression/response_count': 145,
 'httperror/response_ignored_count': 4,
 'httperror/response_ignored_status_count/403': 4,
 'item_scraped_count': 119,
 'items_per_minute': None,
 'log_count/ERROR': 24,
 'log_count/INFO': 58,
 'log_count/WARNING': 78,
 'memusage/max': 125939712,
 'memusage/startup': 125939712,
 'poet/injector/catalog_extraction.pages.ecoenclose.EcoencloseProductsPageObject': 44,
 'product_status/not_available_online': 2,
 'proxy_manager/ignored/proxy_defined': 31,
 'proxy_manager/processed': 157,
 'request_depth_max': 1,
 'response_received_count': 145,
 'responses_per_minute': None,
 'scheduler/dequeued': 188,
 'scheduler/dequeued/memory': 188,
 'scheduler/enqueued': 188,
 'scheduler/enqueued/memory': 188,
 'spider_exceptions/ProductNotFound': 10,
 'spider_exceptions/ValueError': 1,
 'spidermon/validation/fields': 2499,
 'spidermon/validation/items': 119,
 'spidermon/validation/validators': 1,
 'spidermon/validation/validators/item/jsonschema': True,
 'spidermon_field_coverage/dict/brand': 1.0,
 'spidermon_field_coverage/dict/categories': 1.0,
 'spidermon_field_coverage/dict/countryOfOrigin': 1.0,
 'spidermon_field_coverage/dict/description': 1.0,
 'spidermon_field_coverage/dict/imageUrl': 1.0,
 'spidermon_field_coverage/dict/inStock': 1.0,
 'spidermon_field_coverage/dict/isFreeShipping': 1.0,
 'spidermon_field_coverage/dict/leadTime': 1.0,
 'spidermon_field_coverage/dict/manufacturer': 1.0,
 'spidermon_field_coverage/dict/manufacturerSku': 1.0,
 'spidermon_field_coverage/dict/name': 1.0,
 'spidermon_field_coverage/dict/packagingIncrement': 1.0,
 'spidermon_field_coverage/dict/prices': 1.0,
 'spidermon_field_coverage/dict/productStatus': 1.0,
 'spidermon_field_coverage/dict/relatedSkus': 1.0,
 'spidermon_field_coverage/dict/specifications': 1.0,
 'spidermon_field_coverage/dict/supplier': 1.0,
 'spidermon_field_coverage/dict/supplierSku': 1.0,
 'spidermon_field_coverage/dict/uom': 1.0,
 'spidermon_field_coverage/dict/url': 1.0,
 'spidermon_field_coverage/dict/weight': 1.0,
 'spidermon_item_scraped_count': 119,
 'spidermon_item_scraped_count/dict': 119,
 'spidermon_item_scraped_count/dict/brand': 119,
 'spidermon_item_scraped_count/dict/categories': 119,
 'spidermon_item_scraped_count/dict/countryOfOrigin': 119,
 'spidermon_item_scraped_count/dict/description': 119,
 'spidermon_item_scraped_count/dict/imageUrl': 119,
 'spidermon_item_scraped_count/dict/inStock': 119,
 'spidermon_item_scraped_count/dict/isFreeShipping': 119,
 'spidermon_item_scraped_count/dict/leadTime': 119,
 'spidermon_item_scraped_count/dict/manufacturer': 119,
 'spidermon_item_scraped_count/dict/manufacturerSku': 119,
 'spidermon_item_scraped_count/dict/name': 119,
 'spidermon_item_scraped_count/dict/packagingIncrement': 119,
 'spidermon_item_scraped_count/dict/prices': 119,
 'spidermon_item_scraped_count/dict/productStatus': 119,
 'spidermon_item_scraped_count/dict/relatedSkus': 119,
 'spidermon_item_scraped_count/dict/specifications': 119,
 'spidermon_item_scraped_count/dict/supplier': 119,
 'spidermon_item_scraped_count/dict/supplierSku': 119,
 'spidermon_item_scraped_count/dict/uom': 119,
 'spidermon_item_scraped_count/dict/url': 119,
 'spidermon_item_scraped_count/dict/weight': 119,
 'start_requests/big_query': 108,
 'start_time': datetime.datetime(2025, 11, 1, 12, 45, 8, 182962, tzinfo=datetime.timezone.utc)}
2025-11-01 12:45:25 [scrapy.core.engine] (PID: 119) INFO: Spider closed (finished)