2025-11-15 14:00:04 [scrapy.utils.log] (PID: 79) INFO: Scrapy 2.12.0 started (bot: catalog_extraction) 2025-11-15 14:00:05 [scrapy.utils.log] (PID: 79) INFO: Versions: lxml 5.3.1.0, libxml2 2.12.9, cssselect 1.3.0, parsel 1.10.0, w3lib 2.3.1, Twisted 24.11.0, Python 3.11.13 (main, Jun 10 2025, 23:54:42) [GCC 12.2.0], pyOpenSSL 25.0.0 (OpenSSL 3.4.1 11 Feb 2025), cryptography 44.0.2, Platform Linux-6.9.12-x86_64-with-glibc2.36 2025-11-15 14:00:05 [avery] (PID: 79) INFO: Starting extraction spider avery... 2025-11-15 14:00:05 [scrapy.addons] (PID: 79) INFO: Enabled addons: [] 2025-11-15 14:00:05 [py.warnings] (PID: 79) WARNING: /usr/local/lib/python3.11/site-packages/scrapy/utils/request.py:120: ScrapyDeprecationWarning: 'REQUEST_FINGERPRINTER_IMPLEMENTATION' is a deprecated setting. It will be removed in a future version of Scrapy. return cls(crawler) 2025-11-15 14:00:05 [scrapy.extensions.telnet] (PID: 79) INFO: Telnet Password: fa8160b42881f100 2025-11-15 14:00:05 [py.warnings] (PID: 79) WARNING: /var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/extensions/bq_feedstorage.py:33: ScrapyDeprecationWarning: scrapy.extensions.feedexport.build_storage() is deprecated, call the builder directly. 2025-11-15 14:00:05 [scrapy.middleware] (PID: 79) INFO: Enabled extensions: ['scrapy.extensions.corestats.CoreStats', 'scrapy.extensions.telnet.TelnetConsole', 'scrapy.extensions.memusage.MemoryUsage', 'scrapy.extensions.closespider.CloseSpider', 'scrapy.extensions.feedexport.FeedExporter', 'scrapy.extensions.logstats.LogStats', 'spidermon.contrib.scrapy.extensions.Spidermon'] 2025-11-15 14:00:05 [scrapy.crawler] (PID: 79) INFO: Overridden settings: {'BOT_NAME': 'catalog_extraction', 'CONCURRENT_ITEMS': 250, 'CONCURRENT_REQUESTS': 24, 'FEED_EXPORT_ENCODING': 'utf-8', 'LOG_FILE': '/var/lib/scrapyd/logs/catalog_extraction/avery/60b83400c22b11f09c134200a9fe0102.log', 'LOG_FORMAT': '%(asctime)s [%(name)s] (PID: %(process)d) %(levelname)s: ' '%(message)s', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'catalog_extraction.spiders', 'REQUEST_FINGERPRINTER_CLASS': 'scrapy_poet.ScrapyPoetRequestFingerprinter', 'REQUEST_FINGERPRINTER_IMPLEMENTATION': '2.7', 'SPIDER_MODULES': ['catalog_extraction.spiders'], 'TWISTED_REACTOR': 'twisted.internet.asyncioreactor.AsyncioSelectorReactor', 'USER_AGENT': None} 2025-11-15 14:00:05 [scrapy_poet.injection] (PID: 79) INFO: Loading providers: [, , , , , , ] 2025-11-15 14:00:05 [scrapy.middleware] (PID: 79) INFO: Enabled downloader middlewares: ['scrapy.downloadermiddlewares.offsite.OffsiteMiddleware', 'scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware', 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware', 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware', 'scraping_utils.middlewares.downloaders.ProxyManagerDownloaderMiddleware', 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware', 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware', 'scraping_utils.middlewares.downloaders.HeadersSpooferDownloaderMiddleware', 'scrapy_poet.InjectionMiddleware', 'scrapy.downloadermiddlewares.retry.RetryMiddleware', 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware', 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware', 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware', 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware', 'scrapy_poet.DownloaderStatsMiddleware'] 2025-11-15 14:00:05 [NotFoundHandlerSpiderMiddleware] (PID: 79) INFO: NotFoundHandlerSpiderMiddleware running on PRODUCTION environment. 2025-11-15 14:00:06 [scrapy.middleware] (PID: 79) INFO: Enabled spider middlewares: ['catalog_extraction.middlewares.NotFoundHandlerSpiderMiddleware', 'catalog_extraction.middlewares.FixtureSavingMiddleware', 'scrapy_poet.RetryMiddleware', 'scrapy.spidermiddlewares.referer.RefererMiddleware', 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware', 'scrapy.spidermiddlewares.depth.DepthMiddleware'] 2025-11-15 14:00:06 [scrapy.middleware] (PID: 79) INFO: Enabled item pipelines: ['catalog_extraction.pipelines.DuplicatedSKUsFilterPipeline', 'catalog_extraction.pipelines.DiscontinuedProductsAdjustmentPipeline', 'catalog_extraction.pipelines.PriceRoundingPipeline', 'scraping_utils.pipelines.AttachSupplierPipeline', 'spidermon.contrib.scrapy.pipelines.ItemValidationPipeline'] 2025-11-15 14:00:06 [scrapy.core.engine] (PID: 79) INFO: Spider opened 2025-11-15 14:00:06 [scrapy.extensions.closespider] (PID: 79) INFO: Spider will stop when no items are produced after 1800 seconds. 2025-11-15 14:00:06 [scrapy.extensions.logstats] (PID: 79) INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min) 2025-11-15 14:00:06 [scrapy.extensions.telnet] (PID: 79) INFO: Telnet console listening on 127.0.0.1:6024 2025-11-15 14:00:07 [ProxyManagerDownloaderMiddleware] (PID: 79) INFO: Using brd-customer-hl_13cda1e4-zone-main_datacenter_proxy as the default proxy for ProxyManagerDownloaderMiddleware. 2025-11-15 14:00:08 [scrapy_poet._request_fingerprinter] (PID: 79) WARNING: Cannot serialize page params {'product_url': 'https://www.avery.com/products/binders/14803', 'original_response': <200 https://www.avery.com/products/binders/14803>, 'is_discontinued': False} of request as JSON. This can be an issue if you have requests that are identical except for their page params, because unserializable page params are treated the same as missing or empty page params for purposes of request fingerprinting (see https://docs.scrapy.org/en/latest/topics/request-response.html#request-fingerprints). This will be the only warning about this issue, other requests might be also affected. 2025-11-15 14:00:08 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/03002 returned 404 status code. 2025-11-15 14:00:08 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:08 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:08 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:08 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:08 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/27051 returned 404 status code. 2025-11-15 14:00:08 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:08 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:08 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:08 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/cards/type/postcards?page=2 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/cards/5629 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:09 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/dividers/11996 returned 404 status code. 2025-11-15 14:00:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/cards/5889 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/cards/15389 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:09 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/dividers/11415 returned 404 status code. 2025-11-15 14:00:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/cards/5919 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/cards/8383 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/cards/3329 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/cards/8386 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:09 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/79495 returned 404 status code. 2025-11-15 14:00:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/cards/3380 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/cards/5689 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/cards/5625 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/cards/5389 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/cards/8387 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:10 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/79496 returned 404 status code. 2025-11-15 14:00:10 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:10 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:10 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:10 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:10 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:10 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:10 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:10 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:10 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:10 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:10 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/79829 returned 404 status code. 2025-11-15 14:00:10 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/68026 returned 404 status code. 2025-11-15 14:00:10 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:10 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/72268 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:10 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:10 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:10 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:10 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:10 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/68028 returned 404 status code. 2025-11-15 14:00:11 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/79896 returned 404 status code. 2025-11-15 14:00:11 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/dividers/11276 returned 404 status code. 2025-11-15 14:00:11 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47671 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:11 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:11 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:11 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:11 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:11 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:11 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:11 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:11 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:11 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/72268 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:11 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47760 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:11 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:11 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47697 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:11 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:11 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:12 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:12 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:12 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:12 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:12 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:12 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:12 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/41456 returned 404 status code. 2025-11-15 14:00:12 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/73516 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:12 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:12 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/folder-pockets/72268 returned 404 status code. 2025-11-15 14:00:12 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:12 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:12 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47671 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:12 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/40218 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:12 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/73503 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:12 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:12 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:12 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47697 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:12 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/73518 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:12 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:12 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/6478 returned 404 status code. 2025-11-15 14:00:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47760 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/73516 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/13406 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:13 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/6573 returned 404 status code. 2025-11-15 14:00:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/40218 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/73897 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/73518 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/73503 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/48395 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:13 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/folder-pockets/73516 returned 404 status code. 2025-11-15 14:00:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:14 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/folder-pockets/47760 returned 404 status code. 2025-11-15 14:00:14 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:14 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:14 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/13406 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:14 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:14 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/folder-pockets/40218 returned 404 status code. 2025-11-15 14:00:14 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/48395 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:14 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/73897 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:14 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:14 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/custom-printing/labels/calculator/material-film-clear-matte already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:14 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:14 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:15 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/folder-pockets/73503 returned 404 status code. 2025-11-15 14:00:15 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/folder-pockets/73518 returned 404 status code. 2025-11-15 14:00:15 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:15 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/custom-printing/labels/calculator/material-film-clear-matte already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:15 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:15 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/83170 returned 404 status code. 2025-11-15 14:00:15 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/avery.py", line 23, in parse async for item in page.get_items(): File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/avery.py", line 66, in get_items raise NotProductPage(f"URL {product_url} landed on a page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.avery.com/category/products/eco-friendly/ landed on a page that is not a product page. 2025-11-15 14:00:15 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:16 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/21009 returned 404 status code. 2025-11-15 14:00:16 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:17 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/17439 returned 404 status code. 2025-11-15 14:00:17 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/tags/11014 returned 404 status code. 2025-11-15 14:00:17 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:17 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:17 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:17 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:17 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:17 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:17 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:17 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:17 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:17 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:17 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:17 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:17 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:17 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges?page=2 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/namebadges/8781 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/namebadges/74471 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:18 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/dividers/12171 returned 404 status code. 2025-11-15 14:00:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/namebadges/8522 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/namebadges/8520 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/namebadges/5390 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/namebadges/74459 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/namebadges/25395 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/namebadges/5143 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/namebadges/5147 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:18 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/17163 returned 404 status code. 2025-11-15 14:00:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/namebadges/5395 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/namebadges/5392 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/namebadges/8395 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/8781 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/8522 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/74471 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/8520 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:19 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/5392 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:19 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/5395 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:19 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/25395 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:19 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/5143 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:19 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/74459 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:19 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/5147 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:19 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/5390 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:19 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:19 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/8522 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:19 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/8395 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:19 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/8520 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:19 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/8781 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:19 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/5395 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:19 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/74471 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:19 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/74459 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:19 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/25395 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:19 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/5390 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:19 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/5392 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:19 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:19 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:19 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/5147 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:19 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/8395 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:19 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:20 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/5143 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:20 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:20 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:20 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:20 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:20 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:20 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:20 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:20 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:20 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47696 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:20 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:20 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:20 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:20 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:20 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:20 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:20 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:20 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:21 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:21 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:21 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:21 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:21 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:21 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:21 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/4383 returned 404 status code. 2025-11-15 14:00:21 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:21 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47696 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:21 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:21 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:21 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:21 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47781 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:21 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:21 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:22 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/22801 returned 404 status code. 2025-11-15 14:00:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47979 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47960 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47781 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/25395 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47979 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47960 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/74090 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/73907 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:23 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/25395 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:23 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:23 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:23 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/74100 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:23 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:23 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:23 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:23 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/74090 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:23 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/80509 returned 404 status code. 2025-11-15 14:00:23 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:23 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/73907 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:23 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/45395 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:24 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/74100 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:24 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:24 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:24 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:24 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:24 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:24 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:24 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:24 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:24 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/21007 returned 404 status code. 2025-11-15 14:00:24 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/27052 returned 404 status code. 2025-11-15 14:00:24 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:24 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:24 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/45395 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:24 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:24 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/sheet-protectors/73907 returned 404 status code. 2025-11-15 14:00:25 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:25 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:25 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:26 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:26 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:26 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:26 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:26 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:26 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:26 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:26 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:26 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:26 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:27 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:27 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:27 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:27 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:27 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:27 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:27 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/dividers/11408 returned 404 status code. 2025-11-15 14:00:27 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:28 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:28 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:28 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:28 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:28 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:28 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:28 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47811 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:28 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:28 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/4144 returned 404 status code. 2025-11-15 14:00:28 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:28 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47867 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:28 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:28 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:28 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:28 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:28 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:28 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/73517 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:28 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/76009 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:28 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/4161 returned 404 status code. 2025-11-15 14:00:28 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:28 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47811 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:28 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:28 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/8395 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47867 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/76009 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/73517 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/stamp-pad-inkers/21082 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:29 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/tags/12501 returned 404 status code. 2025-11-15 14:00:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/8395 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/73721 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:29 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/markers/27176 returned 404 status code. 2025-11-15 14:00:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/73471 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/stamp-pad-inkers/21082 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:29 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/folder-pockets/47867 returned 404 status code. 2025-11-15 14:00:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/75263 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:30 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/8202 returned 404 status code. 2025-11-15 14:00:30 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:30 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/75537 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:30 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:30 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/73471 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:30 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:30 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/custom-printing/labels/material already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:30 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/75263 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:30 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/73721 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:30 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:30 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:31 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/name-badges/73471 returned 404 status code. 2025-11-15 14:00:31 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/custom-printing/labels/material/all-materials already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:31 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/75537 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:31 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:31 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:31 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:31 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:31 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:31 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:31 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:31 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:31 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:31 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:31 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:31 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:31 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:31 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:31 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:31 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/tags/11027 returned 404 status code. 2025-11-15 14:00:31 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:31 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:31 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:32 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/binder-accessories/89101 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:32 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:32 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:33 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:33 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:33 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:33 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:33 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:33 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:33 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: https://www.avery.com/custom-printing/labels/material/all-materials) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/avery.py", line 80, in get_items raise NotProductPage(f"URL {product_url} landed on a page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.avery.com/custom-printing/labels/material/all-materials landed on a page that is not a product page. 2025-11-15 14:00:33 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/dividers/11525 returned 404 status code. 2025-11-15 14:00:33 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/pens/49986 returned 404 status code. 2025-11-15 14:00:33 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/binder-accessories/89101 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:33 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:33 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:33 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:33 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:33 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:33 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:33 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:34 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/73504 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:34 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/73505 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:34 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:34 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:34 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:34 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:34 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:34 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/dividers/81825 returned 404 status code. 2025-11-15 14:00:34 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:34 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:34 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:34 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:34 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:34 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:34 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:34 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/73505 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:34 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:34 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:34 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:34 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/73504 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:34 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/2162 returned 404 status code. 2025-11-15 14:00:34 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/41451 returned 404 status code. 2025-11-15 14:00:34 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:34 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:35 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:35 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:35 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/folder-pockets/73505 returned 404 status code. 2025-11-15 14:00:35 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:35 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:35 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/folder-pockets/73504 returned 404 status code. 2025-11-15 14:00:35 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/41448 returned 404 status code. 2025-11-15 14:00:35 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:35 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:35 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/40216 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:35 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/5068 returned 404 status code. 2025-11-15 14:00:35 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:35 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/74652 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:35 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:35 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:35 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:35 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:35 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:35 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:35 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/58164 returned 404 status code. 2025-11-15 14:00:35 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:35 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:35 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:36 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/custom-printing/labels/calculator/material-film-white-surface-safe already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:36 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/40216 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:36 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:36 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:36 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/74652 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:36 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/73555 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:36 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/73553 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:36 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/74204 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:36 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:36 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:36 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/custom-printing/labels/calculator/material-film-white-surface-safe already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:36 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:36 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:36 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/21010 returned 404 status code. 2025-11-15 14:00:37 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/74204 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:37 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:37 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/73555 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:37 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/73553 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:37 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:37 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/folder-pockets/40216 returned 404 status code. 2025-11-15 14:00:37 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/40192 returned 404 status code. 2025-11-15 14:00:37 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:37 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:37 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:37 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:37 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:38 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:38 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:38 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:38 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:38 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:38 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:38 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:38 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:38 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/dividers/11529 returned 404 status code. 2025-11-15 14:00:38 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:38 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:38 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:38 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:38 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:38 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:38 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:38 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:38 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:39 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:39 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:39 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:39 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:39 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:39 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/dividers/16184 returned 404 status code. 2025-11-15 14:00:39 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:39 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:39 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:40 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/5205 returned 404 status code. 2025-11-15 14:00:40 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:40 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:40 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:40 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:40 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:40 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/41412 returned 404 status code. 2025-11-15 14:00:40 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:40 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:40 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:40 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:40 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:40 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/48165 returned 404 status code. 2025-11-15 14:00:40 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:40 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:40 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47988 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:40 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:40 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:40 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/custom-printing/labels/calculator/material-film-white-glossy-vinyl-durable-ultra returned 404 status code. 2025-11-15 14:00:40 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:40 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:40 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:40 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:40 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:41 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:41 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:41 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:41 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:41 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47988 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:41 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/stamp-pad-inkers/21282 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:41 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:41 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/8699 returned 404 status code. 2025-11-15 14:00:41 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/tags/11004 returned 404 status code. 2025-11-15 14:00:41 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:41 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:41 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/73554 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:41 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:42 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/stamp-pad-inkers/21282 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:42 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:42 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:42 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:42 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:42 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:42 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:42 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:42 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:42 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:42 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:42 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:42 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:42 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:42 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/73554 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:42 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:42 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:42 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:42 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:42 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:42 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:42 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:42 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/79493 returned 404 status code. 2025-11-15 14:00:42 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:43 [py.warnings] (PID: 79) WARNING: /var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/extensions/bq_feedstorage.py:33: ScrapyDeprecationWarning: scrapy.extensions.feedexport.build_storage() is deprecated, call the builder directly. 2025-11-15 14:00:43 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:43 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:43 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/dividers/07712 returned 404 status code. 2025-11-15 14:00:43 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:44 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:44 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/dividers/7720 returned 404 status code. 2025-11-15 14:00:44 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:44 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:44 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:44 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:44 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:44 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:44 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:44 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:44 [scrapy.extensions.feedexport] (PID: 79) INFO: Stored bq feed (1000 items) in: bq://response-elt.scraper_data.catalog_item_scrape/batch:1 2025-11-15 14:00:44 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:44 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:44 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:44 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:44 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:44 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:44 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:44 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/dividers/11275 returned 404 status code. 2025-11-15 14:00:44 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:44 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:44 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:44 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:44 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:44 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47990 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47986 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47990 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/74471 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/45144 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/74553 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47986 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/75091 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/6175 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/5384 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:46 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/74471 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:46 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/74553 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:46 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/45144 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:46 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:46 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:46 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:46 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/6175 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:46 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/5384 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:46 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/75091 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:46 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:46 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:47 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:47 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:47 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:47 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:47 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:47 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:47 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:47 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:48 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:48 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:48 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/custom-printing/labels/calculator/asset-tag-tamper-evident returned 404 status code. 2025-11-15 14:00:48 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:48 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:48 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:48 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:51 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/binder-accessories/75307 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:51 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:51 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:51 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:51 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:51 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:51 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:51 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:51 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:51 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:51 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:51 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:51 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:51 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/binder-accessories/75307 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:51 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:51 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:51 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:52 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:52 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:52 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:52 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:52 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:52 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/dividers/11995 returned 404 status code. 2025-11-15 14:00:52 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:52 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:52 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:52 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:52 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:52 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/dividers/11700 returned 404 status code. 2025-11-15 14:00:52 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/68040 returned 404 status code. 2025-11-15 14:00:52 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:52 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:52 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:52 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:52 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/79409 returned 404 status code. 2025-11-15 14:00:52 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/79830 returned 404 status code. 2025-11-15 14:00:52 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/72611 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:52 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:52 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:52 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:52 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:53 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:53 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:53 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:53 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:53 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:53 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:53 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:53 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/72611 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:53 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:53 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:53 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:54 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:54 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:54 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:54 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/5199 returned 404 status code. 2025-11-15 14:00:54 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:54 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:54 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:54 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/72052 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:54 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:54 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:54 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:54 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/tags/11012 returned 404 status code. 2025-11-15 14:00:54 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:54 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/6723 returned 404 status code. 2025-11-15 14:00:54 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/6735 returned 404 status code. 2025-11-15 14:00:54 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/6740 returned 404 status code. 2025-11-15 14:00:54 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:54 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/tags/15370 returned 404 status code. 2025-11-15 14:00:54 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/tags/12445 returned 404 status code. 2025-11-15 14:00:54 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:54 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:54 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:54 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/72052 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:54 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:54 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/74404 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:54 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:55 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:55 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:55 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/custom-printing/labels/floor-decals/index returned 404 status code. 2025-11-15 14:00:56 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/tags/11038 returned 404 status code. 2025-11-15 14:00:56 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/17409 returned 404 status code. 2025-11-15 14:00:56 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:56 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:56 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:56 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:56 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:56 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/74404 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:56 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:56 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:57 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/pens/49988 returned 404 status code. 2025-11-15 14:00:57 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/folder-pockets/72052 returned 404 status code. 2025-11-15 14:00:57 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/dividers/23800 returned 404 status code. 2025-11-15 14:00:58 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:58 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:58 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:58 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:58 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:58 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:58 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:58 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:58 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:58 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:58 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:58 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:58 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:58 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:58 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:58 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/68034 returned 404 status code. 2025-11-15 14:00:58 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:58 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:58 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:58 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:00:59 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/68031 returned 404 status code. 2025-11-15 14:00:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:00 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:00 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:00 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:00 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:00 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:00 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:00 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:00 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:00 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:00 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47856 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:00 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:00 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:00 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:00 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:00 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:00 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:00 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:00 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:00 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:00 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47993 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:00 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:00 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:01 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:01 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47038 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:01 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47856 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:01 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:01 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:01 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/74472 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:01 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:01 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/74549 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:01 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47993 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:01 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:01 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:01 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:01 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/74130 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:01 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/42395 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:01 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/74472 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:02 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:02 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:02 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47038 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:02 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:02 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/tags/18801 returned 404 status code. 2025-11-15 14:01:02 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:02 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/74130 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:02 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/42395 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:02 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:02 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:02 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:02 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/21016 returned 404 status code. 2025-11-15 14:01:02 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/tags/11031 returned 404 status code. 2025-11-15 14:01:02 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:02 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/74549 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:02 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/17410 returned 404 status code. 2025-11-15 14:01:02 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:02 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:03 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/folder-pockets/47038 returned 404 status code. 2025-11-15 14:01:03 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:03 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/11778 returned 404 status code. 2025-11-15 14:01:03 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/17438 returned 404 status code. 2025-11-15 14:01:03 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:03 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:03 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:03 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:03 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:03 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:03 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:03 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:03 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:03 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:03 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:03 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:03 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:03 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:03 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/79344 returned 404 status code. 2025-11-15 14:01:03 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:03 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:04 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:04 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:04 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:04 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:04 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:04 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:04 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:04 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:04 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:04 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/dividers/11567 returned 404 status code. 2025-11-15 14:01:04 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:04 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/27275 returned 404 status code. 2025-11-15 14:01:04 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:04 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/79825 returned 404 status code. 2025-11-15 14:01:04 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:04 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:04 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:04 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:05 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:05 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:05 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:05 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:05 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47326 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:05 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:05 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:05 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:05 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:05 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:05 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:05 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/4033 returned 404 status code. 2025-11-15 14:01:05 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:05 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:05 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47326 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:05 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/41427 returned 404 status code. 2025-11-15 14:01:05 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:05 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:05 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:05 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/5589 returned 404 status code. 2025-11-15 14:01:05 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:05 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:05 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:06 [scrapy.extensions.logstats] (PID: 79) INFO: Crawled 1363 pages (at 1363 pages/min), scraped 1707 items (at 1707 items/min) 2025-11-15 14:01:06 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/41429 returned 404 status code. 2025-11-15 14:01:06 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:06 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:06 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47762 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:06 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/highlighters/23550 returned 404 status code. 2025-11-15 14:01:06 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:06 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:06 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/folder-pockets/47326 returned 404 status code. 2025-11-15 14:01:06 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/73900 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:06 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/74804 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:06 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:06 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:06 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/75530 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:06 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:06 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47762 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:06 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:06 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/avery.py", line 23, in parse async for item in page.get_items(): File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/avery.py", line 66, in get_items raise NotProductPage(f"URL {product_url} landed on a page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.avery.com/industrial/products/markers landed on a page that is not a product page. 2025-11-15 14:01:06 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:06 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:06 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/3218 returned 404 status code. 2025-11-15 14:01:06 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:06 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/74804 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:07 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/avery.py", line 23, in parse async for item in page.get_items(): File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/avery.py", line 66, in get_items raise NotProductPage(f"URL {product_url} landed on a page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.avery.com/industrial/products/tags landed on a page that is not a product page. 2025-11-15 14:01:07 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/10803 returned 404 status code. 2025-11-15 14:01:07 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/75530 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:07 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/73900 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:07 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:07 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:07 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:07 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:07 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:07 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:07 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:07 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:07 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/folder-pockets/47762 returned 404 status code. 2025-11-15 14:01:07 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:07 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:07 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:07 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/27050 returned 404 status code. 2025-11-15 14:01:07 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:08 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:08 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:08 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:08 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/27356 returned 404 status code. 2025-11-15 14:01:08 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:08 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:08 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:08 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:08 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:08 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:08 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:08 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/dividers/11701 returned 404 status code. 2025-11-15 14:01:08 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/dividers/11626 returned 404 status code. 2025-11-15 14:01:08 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:09 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/dividers/11409 returned 404 status code. 2025-11-15 14:01:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47691 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47759 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:10 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/48126 returned 404 status code. 2025-11-15 14:01:10 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/5226 returned 404 status code. 2025-11-15 14:01:10 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:10 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:10 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47691 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:10 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47799 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:10 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/55360 returned 404 status code. 2025-11-15 14:01:10 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:10 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:10 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:10 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:10 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:10 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:10 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:10 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:10 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:10 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47759 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:10 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:10 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:10 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:10 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:10 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47799 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:10 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:10 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:10 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:10 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/6583 returned 404 status code. 2025-11-15 14:01:10 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/74171 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:10 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:11 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/folder-pockets/47691 returned 404 status code. 2025-11-15 14:01:11 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:11 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:11 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:11 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/folder-pockets/47759 returned 404 status code. 2025-11-15 14:01:11 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/custom-printing/labels/calculator/durable already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:11 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/folder-pockets/47799 returned 404 status code. 2025-11-15 14:01:11 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:11 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/74171 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:12 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/custom-printing/labels/calculator/stickers-promotion/stickers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:12 [py.warnings] (PID: 79) WARNING: /var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/extensions/bq_feedstorage.py:33: ScrapyDeprecationWarning: scrapy.extensions.feedexport.build_storage() is deprecated, call the builder directly. 2025-11-15 14:01:12 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/21018 returned 404 status code. 2025-11-15 14:01:12 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/21005 returned 404 status code. 2025-11-15 14:01:12 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/17407 returned 404 status code. 2025-11-15 14:01:12 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/17570 returned 404 status code. 2025-11-15 14:01:12 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/21008 returned 404 status code. 2025-11-15 14:01:13 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/05004 returned 404 status code. 2025-11-15 14:01:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:13 [scrapy.extensions.feedexport] (PID: 79) INFO: Stored bq feed (1000 items) in: bq://response-elt.scraper_data.catalog_item_scrape/batch:2 2025-11-15 14:01:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:13 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/dividers/11192 returned 404 status code. 2025-11-15 14:01:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/custom-printing/labels/calculator/stickers-promotion/stickers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:13 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/dividers/11625 returned 404 status code. 2025-11-15 14:01:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:13 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/dividers/11185 returned 404 status code. 2025-11-15 14:01:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:14 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:14 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:14 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/dividers/16061 returned 404 status code. 2025-11-15 14:01:14 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:14 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:14 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:14 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:14 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:15 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47780 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:15 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:15 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:15 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:15 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/5693 returned 404 status code. 2025-11-15 14:01:15 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/41452 returned 404 status code. 2025-11-15 14:01:15 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47964 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:15 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47977 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:15 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:15 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:15 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:15 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:15 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:15 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/41454 returned 404 status code. 2025-11-15 14:01:15 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:15 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:15 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:15 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47780 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:15 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:15 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:15 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:15 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:15 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:15 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47964 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:15 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/78706 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:16 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47977 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:16 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:16 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:16 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:16 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/74082 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:16 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:16 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/45143 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:16 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:16 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/74160 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:16 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:16 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:16 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:16 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:16 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:16 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/74161 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:16 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/78706 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:16 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:16 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:17 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/74160 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:17 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/74082 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:17 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/45143 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:17 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:17 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:17 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:17 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:17 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:17 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:17 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:17 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/74161 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:17 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/24603 returned 404 status code. 2025-11-15 14:01:17 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/21015 returned 404 status code. 2025-11-15 14:01:17 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:17 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:17 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:17 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/name-badges/45143 returned 404 status code. 2025-11-15 14:01:17 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:17 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:18 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/17640 returned 404 status code. 2025-11-15 14:01:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:18 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/11718 returned 404 status code. 2025-11-15 14:01:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:18 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/79335 returned 404 status code. 2025-11-15 14:01:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:19 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:19 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:19 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:19 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:19 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:19 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:19 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:19 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:19 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:19 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/dividers/11150 returned 404 status code. 2025-11-15 14:01:19 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:19 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:19 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:19 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:19 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/79492 returned 404 status code. 2025-11-15 14:01:19 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:19 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:20 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:20 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:20 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:20 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:20 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/55463 returned 404 status code. 2025-11-15 14:01:20 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:20 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:20 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47978 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:20 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:20 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:20 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:20 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:20 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/41559 returned 404 status code. 2025-11-15 14:01:20 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:20 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:20 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:20 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/stamp-pad-inkers/21021 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:20 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:20 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:20 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/41432 returned 404 status code. 2025-11-15 14:01:20 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47978 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:20 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/6695 returned 404 status code. 2025-11-15 14:01:20 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:21 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/markers/23556 returned 404 status code. 2025-11-15 14:01:21 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/tags/11017 returned 404 status code. 2025-11-15 14:01:21 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:21 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/25410 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:21 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/stamp-pad-inkers/21021 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:21 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:21 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:21 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:21 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/74098 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:21 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:21 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/74102 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:21 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:21 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/75536 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:21 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/5362 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:21 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/88127 returned 404 status code. 2025-11-15 14:01:21 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:21 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/5361 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:21 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/25410 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:21 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/tags/15372 returned 404 status code. 2025-11-15 14:01:21 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:21 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/73552 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:21 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/74102 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/74098 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/5362 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:22 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/sheet-protectors/25410 returned 404 status code. 2025-11-15 14:01:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/5361 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/75536 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/binder-accessories/89103 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/binder-accessories/89105 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:23 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:23 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/73552 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:23 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:23 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/binder-accessories/89105 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:23 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:23 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:23 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/binder-accessories/89103 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:23 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:23 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:24 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:24 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:24 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:24 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:24 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:24 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:24 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:24 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:24 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:24 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:24 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:24 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:25 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/dividers/11144 returned 404 status code. 2025-11-15 14:01:25 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:25 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:25 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:25 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:25 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:25 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:25 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:25 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:25 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:25 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:25 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/79494 returned 404 status code. 2025-11-15 14:01:25 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:25 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:25 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:25 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:25 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:25 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:25 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/dividers/41416 returned 404 status code. 2025-11-15 14:01:26 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:26 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47364 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:26 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47359 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:26 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/45280 returned 404 status code. 2025-11-15 14:01:26 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:26 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:26 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:26 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:26 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:26 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47976 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:26 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:26 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47364 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:26 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:26 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:26 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:26 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47359 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:26 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:26 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/41439 returned 404 status code. 2025-11-15 14:01:26 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:26 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:26 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/41567 returned 404 status code. 2025-11-15 14:01:26 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:26 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:26 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:26 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47976 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:26 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/41560 returned 404 status code. 2025-11-15 14:01:27 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:27 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:27 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:27 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:27 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/74461 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:27 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/folder-pockets/47364 returned 404 status code. 2025-11-15 14:01:27 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:27 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/folder-pockets/47359 returned 404 status code. 2025-11-15 14:01:27 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:27 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/tags/22803 returned 404 status code. 2025-11-15 14:01:27 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:27 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:27 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:27 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:27 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/74461 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:28 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:28 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:28 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:28 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/markers/9232 returned 404 status code. 2025-11-15 14:01:28 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:28 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:28 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:28 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:28 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:28 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:28 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:28 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:28 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:28 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:28 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:28 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:28 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:28 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:28 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/18720 returned 404 status code. 2025-11-15 14:01:28 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:28 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:28 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:28 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:29 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/79369 returned 404 status code. 2025-11-15 14:01:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:29 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/79892 returned 404 status code. 2025-11-15 14:01:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:30 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:30 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:30 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:30 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:30 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:30 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:30 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:30 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:30 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:30 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:30 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47847 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:30 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:30 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:30 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:30 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:30 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:30 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:30 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:30 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/41570 returned 404 status code. 2025-11-15 14:01:30 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/13401 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:30 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/76004 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:30 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:30 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:30 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:30 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:30 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:31 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47847 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:31 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/13401 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:31 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:31 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/76004 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:31 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/73720 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:31 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:31 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:31 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:31 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:31 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:31 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:31 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:31 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:31 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/custom-printing/labels/calculator/material-film-white-ultraduty already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:32 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/73720 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:32 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:32 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:32 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:32 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:32 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:32 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:32 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:33 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/custom-printing/labels/calculator/material-film-white-ultraduty already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:33 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/binder-accessories/14825 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:33 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:33 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:33 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:33 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:33 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:33 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:33 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:33 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:33 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/17416 returned 404 status code. 2025-11-15 14:01:33 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/binder-accessories/14825 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:33 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:34 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/05850 returned 404 status code. 2025-11-15 14:01:35 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:35 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:35 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:35 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:35 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:35 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:35 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:35 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:35 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:35 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:36 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:36 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:36 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:36 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:36 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:36 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:36 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:36 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:36 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:36 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:36 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:36 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:36 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:36 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:36 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:36 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:36 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:36 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:36 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:36 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:36 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:36 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:37 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:37 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:37 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:37 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:37 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:37 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:37 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:37 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:37 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/75540 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:37 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/74459 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:37 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:37 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:37 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:37 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:37 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/05277 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:37 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/41459 returned 404 status code. 2025-11-15 14:01:37 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/73472 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:37 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/5151 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:37 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/2921 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:37 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:38 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/74203 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:38 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:38 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:38 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:38 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/75540 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:38 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/74172 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:38 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/05277 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:38 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/74203 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:38 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/5151 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:38 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/74459 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:38 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:38 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:38 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:38 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/2921 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:38 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/73472 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:38 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/custom-printing/labels/calculator/asset-tag-silver already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:38 [py.warnings] (PID: 79) WARNING: /var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/extensions/bq_feedstorage.py:33: ScrapyDeprecationWarning: scrapy.extensions.feedexport.build_storage() is deprecated, call the builder directly. 2025-11-15 14:01:38 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/74172 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:39 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/sheet-protectors/05277 returned 404 status code. 2025-11-15 14:01:39 [scrapy.extensions.feedexport] (PID: 79) INFO: Stored bq feed (1000 items) in: bq://response-elt.scraper_data.catalog_item_scrape/batch:3 2025-11-15 14:01:39 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/avery.py", line 23, in parse async for item in page.get_items(): File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/avery.py", line 66, in get_items raise NotProductPage(f"URL {product_url} landed on a page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.avery.com/custom-printing/labels/wall-signs/ landed on a page that is not a product page. 2025-11-15 14:01:40 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/23023 returned 404 status code. 2025-11-15 14:01:40 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/avery.py", line 23, in parse async for item in page.get_items(): File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/avery.py", line 66, in get_items raise NotProductPage(f"URL {product_url} landed on a page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.avery.com/category/products/stickers/ landed on a page that is not a product page. 2025-11-15 14:01:40 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/name-badges/73472 returned 404 status code. 2025-11-15 14:01:40 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/custom-printing/labels/calculator/asset-tag-silver already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:40 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:40 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:40 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:40 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:40 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:40 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:40 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:40 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:40 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:40 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:40 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:40 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:40 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/sheet-protectors/74172 returned 404 status code. 2025-11-15 14:01:42 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/17408 returned 404 status code. 2025-11-15 14:01:43 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/binder-accessories/75243 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:43 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:43 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:43 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:43 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:43 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:43 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:43 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:43 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:43 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/dividers/11451 returned 404 status code. 2025-11-15 14:01:43 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:43 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:43 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:43 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:43 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/binder-accessories/75243 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:43 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:43 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:44 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/dividers/11388 returned 404 status code. 2025-11-15 14:01:44 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:44 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:44 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:44 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:44 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:44 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/79499 returned 404 status code. 2025-11-15 14:01:44 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/79832 returned 404 status code. 2025-11-15 14:01:44 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/79405 returned 404 status code. 2025-11-15 14:01:44 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:44 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:44 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:44 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:44 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/79333 returned 404 status code. 2025-11-15 14:01:44 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:44 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:45 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/68029 returned 404 status code. 2025-11-15 14:01:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47356 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:45 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/gluesticks/00102 returned 404 status code. 2025-11-15 14:01:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47690 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:45 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/4020 returned 404 status code. 2025-11-15 14:01:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47879 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:45 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/highlighters/07745 returned 404 status code. 2025-11-15 14:01:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:45 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/highlighters/23521 returned 404 status code. 2025-11-15 14:01:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47356 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:46 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/41450 returned 404 status code. 2025-11-15 14:01:46 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:46 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47985 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:46 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:46 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:46 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47879 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:46 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:46 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:46 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47690 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:46 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:46 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:46 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:46 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/folder-pockets/47356 returned 404 status code. 2025-11-15 14:01:46 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/markers/07912 returned 404 status code. 2025-11-15 14:01:46 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/5326 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:46 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:46 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:46 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47985 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:46 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:46 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:47 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/77007 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:47 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:47 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:47 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/markers/29857 returned 404 status code. 2025-11-15 14:01:47 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/folder-pockets/47879 returned 404 status code. 2025-11-15 14:01:47 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:47 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/2920 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:47 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/5326 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:47 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/2922 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:47 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:47 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:47 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/tags/80503 returned 404 status code. 2025-11-15 14:01:47 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:47 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/98122 returned 404 status code. 2025-11-15 14:01:47 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:47 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/dividers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:47 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/74400 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:47 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/77007 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:47 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:47 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/2922 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:47 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/5153 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:47 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/2920 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:47 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/ already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:47 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/avery.py", line 23, in parse async for item in page.get_items(): File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/avery.py", line 66, in get_items raise NotProductPage(f"URL {product_url} landed on a page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.avery.com/custom-printing/labels/ landed on a page that is not a product page. 2025-11-15 14:01:47 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:48 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/74400 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:48 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:48 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/dividers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:48 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/21014 returned 404 status code. 2025-11-15 14:01:48 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/23000 returned 404 status code. 2025-11-15 14:01:48 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:48 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:48 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/5153 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:48 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:48 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/tags/11030 returned 404 status code. 2025-11-15 14:01:48 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:48 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/avery.py", line 23, in parse async for item in page.get_items(): File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/avery.py", line 66, in get_items raise NotProductPage(f"URL {product_url} landed on a page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.avery.com/ landed on a page that is not a product page. 2025-11-15 14:01:48 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:48 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:48 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/binder-accessories/89109 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:48 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/11780 returned 404 status code. 2025-11-15 14:01:48 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/06535 returned 404 status code. 2025-11-15 14:01:48 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:48 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:48 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:48 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:49 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:49 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:49 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:49 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:49 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:49 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/dividers/11109 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:49 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/dividers?page=2 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:49 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/dividers/01701 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:49 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:49 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:49 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:49 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:49 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/dividers/01330 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:49 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/dividers/11227 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:49 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/dividers/01702 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:49 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/dividers/11112 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:49 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/binder-accessories/89109 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:49 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:49 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/dividers/11111 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:49 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/dividers/11225 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:49 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:49 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:49 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/dividers/11226 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:49 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:49 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/50000 returned 404 status code. 2025-11-15 14:01:49 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:49 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:49 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/dividers/11321 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:49 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/dividers/11370 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:49 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:49 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:49 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:49 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/dividers/11228 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:49 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:50 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:50 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:50 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:50 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:50 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:50 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47344 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:50 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:50 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:50 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:50 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:50 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:50 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:50 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:50 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:50 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:50 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:50 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:50 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:51 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:51 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:51 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47344 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:51 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:51 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:51 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:51 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:51 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:51 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:51 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/41447 returned 404 status code. 2025-11-15 14:01:51 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:51 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:51 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:51 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/73506 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:51 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:51 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/5392 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:51 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/76025 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:51 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:51 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/73506 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:51 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/77004 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:51 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/folder-pockets/47344 returned 404 status code. 2025-11-15 14:01:52 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/tags/11011 returned 404 status code. 2025-11-15 14:01:52 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/76016 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:52 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/74558 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:52 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/76025 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:52 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/5398 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:52 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:52 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:52 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/5154 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:52 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:52 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/98110 returned 404 status code. 2025-11-15 14:01:52 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:52 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:52 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/77004 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:52 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:52 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/5392 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:52 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/folder-pockets/73506 returned 404 status code. 2025-11-15 14:01:52 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/74558 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:53 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/5154 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:53 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/5398 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:53 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:53 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:53 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:53 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/3219 returned 404 status code. 2025-11-15 14:01:53 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/15720 returned 404 status code. 2025-11-15 14:01:53 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/name-badges/74558 returned 404 status code. 2025-11-15 14:01:54 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/76016 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:54 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/binder-accessories/89107 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:54 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:54 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:54 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:54 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:54 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:54 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:54 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:54 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:54 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:54 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:54 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:55 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/binder-accessories/89107 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:55 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:55 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:55 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:55 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/17347 returned 404 status code. 2025-11-15 14:01:56 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:56 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/26728 returned 404 status code. 2025-11-15 14:01:56 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:56 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:56 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:56 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:56 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:56 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:56 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:56 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:56 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/dividers/12451 returned 404 status code. 2025-11-15 14:01:57 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:57 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:57 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:57 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:57 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/68030 returned 404 status code. 2025-11-15 14:01:57 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:57 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:57 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:57 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:57 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:57 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:57 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:57 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:57 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:57 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:57 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:57 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:57 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/55464 returned 404 status code. 2025-11-15 14:01:57 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:57 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:57 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:58 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:58 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:58 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:58 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:58 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/highlighters/23555 returned 404 status code. 2025-11-15 14:01:58 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/6574 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:58 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:58 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:58 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:58 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:58 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:58 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:58 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:58 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/40215 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:58 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:58 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/41565 returned 404 status code. 2025-11-15 14:01:58 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/6574 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/73515 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/73899 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/40215 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/2923 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/73515 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/74170 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/75201 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:01:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:00 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:00 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/folder-pockets/6574 returned 404 status code. 2025-11-15 14:02:00 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/73899 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:00 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:00 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/2923 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:00 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/74170 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:00 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:00 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/folder-pockets/40215 returned 404 status code. 2025-11-15 14:02:00 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/folder-pockets/73515 returned 404 status code. 2025-11-15 14:02:00 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/75201 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:00 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/73556 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:00 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:00 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:01 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/73551 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:01 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/binder-accessories/75308 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:01 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:01 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:01 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:01 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:01 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:01 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:01 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/73556 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:01 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:01 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/15761 returned 404 status code. 2025-11-15 14:02:01 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:01 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/sheet-protectors/73899 returned 404 status code. 2025-11-15 14:02:01 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/sheet-protectors/74170 returned 404 status code. 2025-11-15 14:02:01 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:01 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:01 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:01 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:01 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:02 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/73551 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:02 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/binder-accessories/75308 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:02 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:02 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/sheet-protectors/75201 returned 404 status code. 2025-11-15 14:02:02 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/dividers/11512 returned 404 status code. 2025-11-15 14:02:02 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:02 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:02 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:03 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:03 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:03 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:03 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:03 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:03 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:03 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:03 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:03 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:03 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:03 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:03 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:03 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/79406 returned 404 status code. 2025-11-15 14:02:03 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:03 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/72311 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:03 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:03 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:03 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:03 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:03 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:03 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:04 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:04 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:04 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:04 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/5568 returned 404 status code. 2025-11-15 14:02:04 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:04 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:04 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:04 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/41455 returned 404 status code. 2025-11-15 14:02:04 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/72311 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:04 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:04 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:04 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:04 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:04 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:04 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:04 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/40213 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:04 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:04 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/74457 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:05 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:05 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:05 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/22819 returned 404 status code. 2025-11-15 14:02:05 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:05 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:05 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/6471 returned 404 status code. 2025-11-15 14:02:05 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:05 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:05 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:05 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:05 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/5397 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:05 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/tags/41557 returned 404 status code. 2025-11-15 14:02:05 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/74457 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:05 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:05 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/avery.py", line 23, in parse async for item in page.get_items(): File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/avery.py", line 66, in get_items raise NotProductPage(f"URL {product_url} landed on a page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.avery.com/industrial/products/binders landed on a page that is not a product page. 2025-11-15 14:02:05 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/8206 returned 404 status code. 2025-11-15 14:02:05 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/40213 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:05 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:05 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/custom-printing/stickers/calculator/car-stickers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:05 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:05 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:06 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:06 [scrapy.extensions.logstats] (PID: 79) INFO: Crawled 2921 pages (at 1558 pages/min), scraped 3993 items (at 2286 items/min) 2025-11-15 14:02:06 [py.warnings] (PID: 79) WARNING: /var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/extensions/bq_feedstorage.py:33: ScrapyDeprecationWarning: scrapy.extensions.feedexport.build_storage() is deprecated, call the builder directly. 2025-11-15 14:02:06 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/5397 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:06 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/custom-printing/stickers/calculator/car-stickers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:06 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:06 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/28660 returned 404 status code. 2025-11-15 14:02:06 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:06 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:06 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:06 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/folder-pockets/40213 returned 404 status code. 2025-11-15 14:02:06 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:06 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:06 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/17250 returned 404 status code. 2025-11-15 14:02:06 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/26750 returned 404 status code. 2025-11-15 14:02:06 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:06 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:06 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:06 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:06 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:07 [scrapy.extensions.feedexport] (PID: 79) INFO: Stored bq feed (1000 items) in: bq://response-elt.scraper_data.catalog_item_scrape/batch:4 2025-11-15 14:02:07 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:07 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:07 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:07 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:07 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:07 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:07 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:07 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:07 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/name-badges/5397 returned 404 status code. 2025-11-15 14:02:07 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:07 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:07 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/dividers/24921 returned 404 status code. 2025-11-15 14:02:07 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/79831 returned 404 status code. 2025-11-15 14:02:07 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/68027 returned 404 status code. 2025-11-15 14:02:07 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:07 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:07 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:07 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:07 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:07 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:08 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:08 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:08 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:08 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:08 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/72279 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:08 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47665 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:08 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:08 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:08 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47710 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:08 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47358 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:08 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:08 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47796 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:08 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:08 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:08 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:08 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:08 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/41457 returned 404 status code. 2025-11-15 14:02:08 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:08 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:08 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:08 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:08 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47710 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:08 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:08 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47665 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:08 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/72279 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:08 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47358 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:08 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47796 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:09 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/6476 returned 404 status code. 2025-11-15 14:02:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/75538 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/75539 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/74651 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:09 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/tags/12302 returned 404 status code. 2025-11-15 14:02:09 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/folder-pockets/47665 returned 404 status code. 2025-11-15 14:02:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:09 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/folder-pockets/47358 returned 404 status code. 2025-11-15 14:02:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/5395 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/75538 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:09 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/folder-pockets/47796 returned 404 status code. 2025-11-15 14:02:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/custom-printing/labels/calculator/pet-product already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/75539 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:09 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/folder-pockets/72279 returned 404 status code. 2025-11-15 14:02:09 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:10 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/sheet-protectors/75538 returned 404 status code. 2025-11-15 14:02:10 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/5395 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:10 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/74651 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:10 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:10 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/6540 returned 404 status code. 2025-11-15 14:02:11 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/name-badges/74651 returned 404 status code. 2025-11-15 14:02:11 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/custom-printing/labels/calculator/stickers-promotion/stickers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:11 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:11 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:11 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:11 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:11 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/tags/11555 returned 404 status code. 2025-11-15 14:02:12 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/custom-printing/labels/calculator/stickers-promotion/stickers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:12 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:12 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/markers/9233 returned 404 status code. 2025-11-15 14:02:12 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:12 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:12 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:12 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:12 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:12 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:12 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:12 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:12 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:13 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/06635 returned 404 status code. 2025-11-15 14:02:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:13 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:14 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:14 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/68039 returned 404 status code. 2025-11-15 14:02:14 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:14 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:14 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:14 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:14 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:14 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:14 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47693 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:14 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:14 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:14 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:14 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:14 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:14 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:14 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:14 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47846 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:14 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:14 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:14 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:14 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47798 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:15 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/48267 returned 404 status code. 2025-11-15 14:02:15 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47992 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:15 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:15 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:15 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:15 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47693 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:15 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:15 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:15 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47846 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:15 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47798 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:15 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:15 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47992 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:15 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:15 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47342 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:15 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/73520 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:15 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:15 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:15 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:15 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/74540 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:15 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/74556 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:15 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/74084 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:15 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/folder-pockets/47693 returned 404 status code. 2025-11-15 14:02:16 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:16 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47342 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:16 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/folder-pockets/47798 returned 404 status code. 2025-11-15 14:02:16 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:16 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:16 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:16 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/73520 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:16 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/8117 returned 404 status code. 2025-11-15 14:02:16 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/5390 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:16 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:16 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:16 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/74556 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:16 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/74084 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:16 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/tags/41566 returned 404 status code. 2025-11-15 14:02:16 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:16 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/74540 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:16 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:16 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:16 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/folder-pockets/47342 returned 404 status code. 2025-11-15 14:02:16 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/folder-pockets/73520 returned 404 status code. 2025-11-15 14:02:17 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/5390 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:17 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47961 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:17 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:17 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/name-badges/74556 returned 404 status code. 2025-11-15 14:02:17 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:17 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:17 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/avery.py", line 23, in parse async for item in page.get_items(): File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/avery.py", line 66, in get_items raise NotProductPage(f"URL {product_url} landed on a page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.avery.com/campaign/labels/any-size/ landed on a page that is not a product page. 2025-11-15 14:02:17 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:17 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:17 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:17 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:17 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:17 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:17 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:17 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:17 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/avery.py", line 23, in parse async for item in page.get_items(): File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/avery.py", line 66, in get_items raise NotProductPage(f"URL {product_url} landed on a page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.avery.com/category/products/labels/ landed on a page that is not a product page. 2025-11-15 14:02:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47961 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:18 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/18721 returned 404 status code. 2025-11-15 14:02:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:18 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:19 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:19 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:19 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:19 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:19 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:19 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/06107 returned 404 status code. 2025-11-15 14:02:19 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:19 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:19 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47715 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:19 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:19 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:19 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:19 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47989 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:19 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:19 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:19 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:19 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:19 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:19 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:19 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:19 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47715 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:20 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:20 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:20 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47991 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:20 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:20 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/73514 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:20 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:20 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:20 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:20 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:20 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:20 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47989 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:20 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:20 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/74536 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:20 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/73514 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:20 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:20 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47991 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:20 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/74106 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:20 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:21 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:21 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/tags/11035 returned 404 status code. 2025-11-15 14:02:21 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/73610 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:21 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/5144 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:21 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:21 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/74536 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:21 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/73802 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:21 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:21 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:21 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:21 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:21 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/75182 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:21 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/74106 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:21 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/folder-pockets/73514 returned 404 status code. 2025-11-15 14:02:21 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:21 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/73610 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:21 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:21 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/5144 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:21 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/73609 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:21 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:21 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/custom-printing/stickers/calculator/water-bottle-stickers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:21 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/avery.py", line 23, in parse async for item in page.get_items(): File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/avery.py", line 66, in get_items raise NotProductPage(f"URL {product_url} landed on a page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.avery.com/industrial/products/stamppadinkers landed on a page that is not a product page. 2025-11-15 14:02:21 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:21 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/73802 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:21 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:21 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:22 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/03239 returned 404 status code. 2025-11-15 14:02:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/75182 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:22 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/sheet-protectors/73610 returned 404 status code. 2025-11-15 14:02:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/73609 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/custom-printing/stickers/calculator/water-bottle-stickers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:22 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/17346 returned 404 status code. 2025-11-15 14:02:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:22 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:23 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:23 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:23 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:23 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:23 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:23 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:23 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:23 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:23 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:23 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:23 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:23 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:23 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:23 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:23 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47346 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:23 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:23 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47699 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:23 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:23 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:23 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:24 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:24 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:24 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47692 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:24 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:24 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:24 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:24 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:24 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:24 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47699 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:24 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/41431 returned 404 status code. 2025-11-15 14:02:24 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:24 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47346 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:24 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:24 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:24 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:24 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:25 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47325 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:25 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:25 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:25 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:25 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47692 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:25 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:25 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:25 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:25 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/folder-pockets/47699 returned 404 status code. 2025-11-15 14:02:25 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/folder-pockets/47346 returned 404 status code. 2025-11-15 14:02:25 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/stamp-pad-inkers/21271 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:25 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:25 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:25 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:25 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47325 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:25 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/74541 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:25 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:25 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/28663 returned 404 status code. 2025-11-15 14:02:25 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/stamp-pad-inkers/21271 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:25 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:25 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/75009 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:25 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:25 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/tags/12506 returned 404 status code. 2025-11-15 14:02:25 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:25 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:25 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:25 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/75304 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:26 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/folder-pockets/47325 returned 404 status code. 2025-11-15 14:02:26 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/4015 returned 404 status code. 2025-11-15 14:02:26 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/17443 returned 404 status code. 2025-11-15 14:02:27 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/75009 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:27 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/74541 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:27 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:27 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:27 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:27 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:27 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:27 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:27 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:27 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/75304 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:27 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:27 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:27 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:28 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/binder-accessories/75254 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:28 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/17345 returned 404 status code. 2025-11-15 14:02:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/binder-accessories/75254 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:29 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:30 [py.warnings] (PID: 79) WARNING: /var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/extensions/bq_feedstorage.py:33: ScrapyDeprecationWarning: scrapy.extensions.feedexport.build_storage() is deprecated, call the builder directly. 2025-11-15 14:02:30 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:30 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:30 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:30 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:30 [scrapy.extensions.feedexport] (PID: 79) INFO: Stored bq feed (1000 items) in: bq://response-elt.scraper_data.catalog_item_scrape/batch:5 2025-11-15 14:02:30 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:30 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:30 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:30 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/73524 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:30 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:30 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47852 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:31 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:31 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:31 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:31 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:31 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:31 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:31 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:31 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:31 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:31 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/72262 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:31 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47852 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:31 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:31 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:31 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/5206 returned 404 status code. 2025-11-15 14:02:31 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/73524 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:31 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/76001 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:31 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:31 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:31 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:31 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/72262 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:32 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/tags/15371 returned 404 status code. 2025-11-15 14:02:32 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:32 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/73603 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:32 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:32 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:32 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/5140 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:32 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/76001 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:32 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/folder-pockets/73524 returned 404 status code. 2025-11-15 14:02:32 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/75183 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:32 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:32 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/5895 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:32 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:32 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/folder-pockets/47852 returned 404 status code. 2025-11-15 14:02:32 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:32 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/73603 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:32 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/21011 returned 404 status code. 2025-11-15 14:02:32 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/5140 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:32 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/75183 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:32 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:32 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:32 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:33 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/binder-accessories/75225 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:33 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:33 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:33 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:33 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/17411 returned 404 status code. 2025-11-15 14:02:33 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/5895 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:33 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:33 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:33 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:33 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:33 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: https://www.avery.com/custom-printing/labels/bumper-stickers/) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/avery.py", line 69, in get_items product, categories = self.extract_product_data(original_response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/avery.py", line 33, in extract_product_data product_dict = json.loads(product_text.get()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/json/__init__.py", line 339, in loads raise TypeError(f'the JSON object must be str, bytes or bytearray, ' TypeError: the JSON object must be str, bytes or bytearray, not NoneType 2025-11-15 14:02:33 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:33 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:33 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:33 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:33 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:33 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/dividers/11453 returned 404 status code. 2025-11-15 14:02:33 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:33 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:33 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:33 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:33 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:33 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/binder-accessories/75225 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:33 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:33 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:33 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:34 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:34 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:34 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:34 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:34 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:34 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:34 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:34 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:34 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:34 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:35 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/72278 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:35 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:35 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:35 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:35 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47701 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:35 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:35 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:35 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:35 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/6002 returned 404 status code. 2025-11-15 14:02:35 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/dividers/81824 returned 404 status code. 2025-11-15 14:02:35 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:35 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:35 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/72278 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:35 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:35 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:35 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:35 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:35 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/41453 returned 404 status code. 2025-11-15 14:02:35 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:35 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47975 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:35 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:36 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:36 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47701 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:36 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:36 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:36 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:36 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/74552 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:36 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/74554 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:36 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/pens/49987 returned 404 status code. 2025-11-15 14:02:36 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:36 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/5143 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:36 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:36 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:36 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/6970 returned 404 status code. 2025-11-15 14:02:36 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47975 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:36 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:36 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:36 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:36 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/folder-pockets/47701 returned 404 status code. 2025-11-15 14:02:36 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:38 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/74554 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:38 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/5143 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:38 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/74552 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:38 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:38 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:38 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:38 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:38 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:38 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:39 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/73550 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:39 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/3220 returned 404 status code. 2025-11-15 14:02:39 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/name-badges/74552 returned 404 status code. 2025-11-15 14:02:41 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/tags/11015 returned 404 status code. 2025-11-15 14:02:41 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/15725 returned 404 status code. 2025-11-15 14:02:41 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:41 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:41 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:41 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:41 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:41 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:41 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:41 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:42 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/06532 returned 404 status code. 2025-11-15 14:02:42 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/avery.py", line 23, in parse async for item in page.get_items(): File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/avery.py", line 66, in get_items raise NotProductPage(f"URL {product_url} landed on a page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.avery.com/category/products/ landed on a page that is not a product page. 2025-11-15 14:02:42 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/dividers/12453 returned 404 status code. 2025-11-15 14:02:42 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/73550 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:42 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:42 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:42 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:42 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:42 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:42 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:42 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:42 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:42 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:42 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:42 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:42 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:42 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:42 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:42 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:43 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:43 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:43 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:43 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:43 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:43 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:44 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:44 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:44 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:44 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:44 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:44 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:44 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:44 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:44 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:44 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/4022 returned 404 status code. 2025-11-15 14:02:44 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:44 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:44 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/5493 returned 404 status code. 2025-11-15 14:02:44 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:44 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:44 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:44 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:44 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:44 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:44 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:44 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/48260 returned 404 status code. 2025-11-15 14:02:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:45 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/5694 returned 404 status code. 2025-11-15 14:02:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/72053 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47826 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47314 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/74101 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:45 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/tags/12203 returned 404 status code. 2025-11-15 14:02:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47987 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:45 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/tags/6591 returned 404 status code. 2025-11-15 14:02:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:45 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:46 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/8891 returned 404 status code. 2025-11-15 14:02:46 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:46 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47314 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:46 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47987 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:46 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/72053 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:46 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47826 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:46 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/74101 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:46 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:46 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:46 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:46 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:47 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:47 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:47 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:47 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/folder-pockets/47314 returned 404 status code. 2025-11-15 14:02:47 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:47 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:47 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:47 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/folder-pockets/47826 returned 404 status code. 2025-11-15 14:02:47 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/binder-accessories/14230 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:48 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/tags/11013 returned 404 status code. 2025-11-15 14:02:48 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/binder-accessories/75312 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:48 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:48 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:48 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/03243 returned 404 status code. 2025-11-15 14:02:48 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:48 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:48 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:48 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/21002 returned 404 status code. 2025-11-15 14:02:49 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/binder-accessories/75312 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:49 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/binder-accessories/14230 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:49 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:49 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:49 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:49 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:49 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:49 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:49 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/21004 returned 404 status code. 2025-11-15 14:02:49 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:49 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:49 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:49 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:49 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:49 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/dividers/11998 returned 404 status code. 2025-11-15 14:02:49 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:49 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:50 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/05213 returned 404 status code. 2025-11-15 14:02:50 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:50 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/79332 returned 404 status code. 2025-11-15 14:02:50 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:50 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:50 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:50 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:50 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:50 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:50 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:50 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:50 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:50 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:50 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:50 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:50 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/dividers/16187 returned 404 status code. 2025-11-15 14:02:50 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:50 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:50 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:51 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:51 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:51 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:51 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:51 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:51 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:51 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47880 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:51 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:51 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:51 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:51 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/2180 returned 404 status code. 2025-11-15 14:02:51 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:51 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/5069 returned 404 status code. 2025-11-15 14:02:51 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:51 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/40217 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:51 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:51 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/74520 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:51 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:51 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:52 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:52 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:52 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47880 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:52 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:52 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/40217 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:52 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:52 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:52 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:52 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/74107 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:52 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/73803 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:52 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/5095 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:52 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:52 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/74520 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:52 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/5147 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:52 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/74301 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:52 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:52 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:52 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:52 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/folder-pockets/40217 returned 404 status code. 2025-11-15 14:02:53 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/74107 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:53 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/73803 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:53 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/5095 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:53 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:53 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:53 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/5147 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:53 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:53 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:53 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:53 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/74301 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:53 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:53 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:53 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:53 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:53 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:53 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/05303 returned 404 status code. 2025-11-15 14:02:53 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:53 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:53 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:54 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:54 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:54 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:54 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:54 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:54 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:54 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:54 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:54 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/sheet-protectors/74301 returned 404 status code. 2025-11-15 14:02:54 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:54 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:54 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:55 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:55 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:55 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:55 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/binders/79334 returned 404 status code. 2025-11-15 14:02:55 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:55 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:55 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:55 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/dividers/11413 returned 404 status code. 2025-11-15 14:02:55 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:55 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:55 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:55 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:55 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:55 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/40212 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:55 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:56 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47347 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:56 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:56 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:56 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:56 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/73523 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:56 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:56 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:56 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:56 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:56 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/47761 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:56 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:56 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:56 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:56 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/40212 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:56 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:56 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:56 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:56 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47347 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:56 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:56 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:56 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/47761 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:56 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/73523 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:56 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/5105 returned 404 status code. 2025-11-15 14:02:56 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/folder-pockets/73513 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:56 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:57 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:57 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:57 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:57 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/85783 returned 404 status code. 2025-11-15 14:02:57 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/73602 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:57 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/5141 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:57 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/folder-pockets/47347 returned 404 status code. 2025-11-15 14:02:57 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/labels/8590 returned 404 status code. 2025-11-15 14:02:57 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/name-badges/5146 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:57 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/folder-pockets/73513 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:57 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/folder-pockets/40212 returned 404 status code. 2025-11-15 14:02:57 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/tags/14314 returned 404 status code. 2025-11-15 14:02:57 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:57 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:57 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:57 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/folder-pockets/47761 returned 404 status code. 2025-11-15 14:02:57 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/avery.py", line 23, in parse async for item in page.get_items(): File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/avery.py", line 66, in get_items raise NotProductPage(f"URL {product_url} landed on a page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.avery.com/custom-printing/labels/shape/ landed on a page that is not a product page. 2025-11-15 14:02:57 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:57 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/5141 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:57 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/custom-printing/labels/calculator/asset-tag-white already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:57 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheetprotectors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:57 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/name-badges/5146 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:57 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/73602 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:58 [py.warnings] (PID: 79) WARNING: /var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/extensions/bq_feedstorage.py:33: ScrapyDeprecationWarning: scrapy.extensions.feedexport.build_storage() is deprecated, call the builder directly. 2025-11-15 14:02:58 [scrapy.core.scraper] (PID: 79) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.avery.com/products/folder-pockets/73513 returned 404 status code. 2025-11-15 14:02:58 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/custom-printing/labels/calculator/asset-tag-white already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:58 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheetprotectors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:58 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:58 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:58 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:58 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:58 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:58 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:58 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:58 [scrapy.extensions.feedexport] (PID: 79) INFO: Stored bq feed (1000 items) in: bq://response-elt.scraper_data.catalog_item_scrape/batch:6 2025-11-15 14:02:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheetprotectors/73720 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheetprotectors/74082 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheetprotectors/13401 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheetprotectors/74806 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheetprotectors/73601 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheetprotectors/75183 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheetprotectors/13406 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheetprotectors/76016 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheetprotectors/77004 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheetprotectors/76009 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheetprotectors/76025 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheetprotectors/76000 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors?page=2 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/74082 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/13401 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/74806 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/73720 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/73601 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/75183 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/77004 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/76009 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/76025 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/76000 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/74806 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/76016 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request http://www.avery.com/products/sheet-protectors/13406 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/77004 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/73601 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/75183 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/76025 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/76009 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:02:59 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/13401 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:03:00 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/74082 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:03:00 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:03:00 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/76000 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:03:00 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/76016 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:03:00 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:03:00 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/13406 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:03:00 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/api/v1/graphql already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:03:00 [HeadersSpooferDownloaderMiddleware] (PID: 79) WARNING: Request https://www.avery.com/products/sheet-protectors/73720 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-15 14:03:00 [scrapy.core.engine] (PID: 79) INFO: Closing spider (finished) 2025-11-15 14:03:00 [avery] (PID: 79) INFO: [Spidermon] ------------------------------ MONITORS ------------------------------ 2025-11-15 14:03:00 [avery] (PID: 79) INFO: [Spidermon] Extracted Items Monitor/test_stat_monitor... OK 2025-11-15 14:03:00 [avery] (PID: 79) INFO: [Spidermon] Item Validation Monitor/test_stat_monitor... SKIPPED (Unable to find 'spidermon/validation/fields/errors' in job stats.) 2025-11-15 14:03:00 [avery] (PID: 79) INFO: [Spidermon] Error Count Monitor/test_stat_monitor... FAIL 2025-11-15 14:03:00 [avery] (PID: 79) INFO: [Spidermon] Warning Count Monitor/test_stat_monitor... FAIL 2025-11-15 14:03:00 [avery] (PID: 79) INFO: [Spidermon] Finish Reason Monitor/Should have the expected finished reason(s)... OK 2025-11-15 14:03:00 [avery] (PID: 79) INFO: [Spidermon] Unwanted HTTP codes monitor/Should not hit the limit of unwanted http status... OK 2025-11-15 14:03:00 [avery] (PID: 79) INFO: [Spidermon] Field Coverage Monitor/test_check_if_field_coverage_rules_are_met... OK 2025-11-15 14:03:00 [avery] (PID: 79) INFO: [Spidermon] Retry Count monitor/Should not hit the limit of requests that reached the maximum retry amount... OK 2025-11-15 14:03:00 [avery] (PID: 79) INFO: [Spidermon] Downloader Exceptions monitor/test_stat_monitor... SKIPPED (Unable to find 'downloader/exception_count' in job stats.) 2025-11-15 14:03:00 [avery] (PID: 79) INFO: [Spidermon] Successful Requests monitor/Should have at least the minimum number of successful requests... OK 2025-11-15 14:03:00 [avery] (PID: 79) INFO: [Spidermon] Total Requests monitor/Should not hit the total limit of requests... OK 2025-11-15 14:03:00 [avery] (PID: 79) INFO: [Spidermon] ---------------------------------------------------------------------- 2025-11-15 14:03:00 [avery] (PID: 79) ERROR: [Spidermon] ====================================================================== FAIL: Error Count Monitor/test_stat_monitor ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/spidermon/contrib/scrapy/monitors/base.py", line 184, in test_stat_monitor assertion_method( AssertionError: Expecting 'log_count/ERROR' to be '<=' to '335.0'. Current value: '338' 2025-11-15 14:03:00 [avery] (PID: 79) ERROR: [Spidermon] ====================================================================== FAIL: Warning Count Monitor/test_stat_monitor ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/spidermon/contrib/scrapy/monitors/base.py", line 184, in test_stat_monitor assertion_method( AssertionError: Expecting 'log_count/WARNING' to be '<=' to '2.0'. Current value: '2447' 2025-11-15 14:03:00 [avery] (PID: 79) INFO: [Spidermon] 11 monitors in 0.004s 2025-11-15 14:03:00 [avery] (PID: 79) INFO: [Spidermon] FAILED (failures=2, skipped=2) 2025-11-15 14:03:00 [avery] (PID: 79) INFO: [Spidermon] -------------------------- FINISHED ACTIONS -------------------------- 2025-11-15 14:03:00 [avery] (PID: 79) INFO: [Spidermon] ---------------------------------------------------------------------- 2025-11-15 14:03:00 [avery] (PID: 79) INFO: [Spidermon] 0 actions in 0.000s 2025-11-15 14:03:00 [avery] (PID: 79) INFO: [Spidermon] OK 2025-11-15 14:03:00 [avery] (PID: 79) INFO: [Spidermon] --------------------------- PASSED ACTIONS --------------------------- 2025-11-15 14:03:00 [avery] (PID: 79) INFO: [Spidermon] ---------------------------------------------------------------------- 2025-11-15 14:03:00 [avery] (PID: 79) INFO: [Spidermon] 0 actions in 0.000s 2025-11-15 14:03:00 [avery] (PID: 79) INFO: [Spidermon] OK 2025-11-15 14:03:00 [avery] (PID: 79) INFO: [Spidermon] --------------------------- FAILED ACTIONS --------------------------- 2025-11-15 14:03:01 [spidermon.contrib.actions.slack] (PID: 79) WARNING: bot cannot finder user in slack org member list - default icon url used 2025-11-15 14:03:01 [avery] (PID: 79) INFO: [Spidermon] CustomTemplateSendSlackMessageSpiderFinished... OK 2025-11-15 14:03:01 [avery] (PID: 79) INFO: [Spidermon] ---------------------------------------------------------------------- 2025-11-15 14:03:01 [avery] (PID: 79) INFO: [Spidermon] 1 action in 0.449s 2025-11-15 14:03:01 [avery] (PID: 79) INFO: [Spidermon] OK 2025-11-15 14:03:01 [avery] (PID: 79) INFO: 323 URLs returned ProductNotFound. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/custom-printing/labels/calculator/asset-tag-tamper-evident is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/custom-printing/labels/calculator/material-film-white-glossy-vinyl-durable-ultra is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/custom-printing/labels/floor-decals/index is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/03002 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/03239 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/03243 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/05004 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/05213 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/05303 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/05850 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/06532 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/06535 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/06635 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/10803 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/11718 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/11778 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/11780 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/15720 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/15725 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/15761 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/17163 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/17250 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/17345 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/17346 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/17347 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/17407 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/17408 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/17409 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/17410 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/17411 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/17416 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/17438 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/17439 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/17443 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/17570 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/17640 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/18720 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/18721 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/21002 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/21004 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/21005 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/21007 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/21008 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/21009 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/21010 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/21011 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/21014 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/21015 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/21016 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/21018 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/23000 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/23023 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/26728 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/26750 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/27050 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/27051 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/27052 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/27275 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/27356 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/3218 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/3219 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/3220 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/50000 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/68026 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/68027 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/68028 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/68029 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/68030 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/68031 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/68034 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/68039 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/68040 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/79332 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/79333 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/79334 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/79335 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/79344 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/79369 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/79405 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/79406 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/79409 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/79492 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/79493 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/79494 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/79495 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/79496 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/79499 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/79825 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/79829 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/79830 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/79831 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/79832 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/79892 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/binders/79896 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/dividers/07712 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/dividers/11144 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/dividers/11150 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/dividers/11185 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/dividers/11192 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/dividers/11275 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/dividers/11276 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/dividers/11388 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/dividers/11408 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/dividers/11409 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/dividers/11413 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/dividers/11415 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/dividers/11451 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/dividers/11453 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/dividers/11512 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/dividers/11525 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/dividers/11529 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/dividers/11567 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/dividers/11625 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/dividers/11626 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/dividers/11700 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/dividers/11701 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/dividers/11995 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/dividers/11996 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/dividers/11998 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/dividers/12171 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/dividers/12451 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/dividers/12453 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/dividers/16061 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/dividers/16184 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/dividers/16187 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/dividers/23800 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/dividers/24921 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/dividers/41416 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/dividers/7720 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/dividers/81824 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/dividers/81825 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/gluesticks/00102 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/highlighters/07745 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/highlighters/23521 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/highlighters/23550 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/highlighters/23555 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/06107 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/2162 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/2180 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/22801 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/22819 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/24603 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/28660 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/28663 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/4015 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/40192 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/4020 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/4022 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/4033 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/41412 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/41427 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/41429 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/41431 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/41432 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/41439 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/4144 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/41447 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/41448 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/41450 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/41451 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/41452 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/41453 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/41454 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/41455 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/41456 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/41457 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/41459 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/41559 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/41560 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/41565 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/41567 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/41570 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/4161 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/4383 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/45280 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/48126 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/48165 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/48260 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/48267 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/5068 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/5069 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/5105 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/5199 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/5205 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/5206 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/5226 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/5493 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/55360 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/55463 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/55464 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/5568 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/5589 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/5693 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/5694 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/58164 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/6002 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/6471 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/6476 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/6478 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/6540 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/6573 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/6583 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/6695 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/6723 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/6735 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/6740 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/6970 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/80509 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/8117 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/8202 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/8206 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/83170 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/85783 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/8590 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/8699 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/88127 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/8891 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/98110 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/labels/98122 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/markers/07912 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/markers/23556 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/markers/27176 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/markers/29857 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/markers/9232 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/markers/9233 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/pens/49986 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/pens/49987 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/pens/49988 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/tags/11004 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/tags/11011 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/tags/11012 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/tags/11013 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/tags/11014 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/tags/11015 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/tags/11017 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/tags/11027 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/tags/11030 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/tags/11031 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/tags/11035 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/tags/11038 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/tags/11555 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/tags/12203 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/tags/12302 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/tags/12445 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/tags/12501 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/tags/12506 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/tags/14314 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/tags/15370 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/tags/15371 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/tags/15372 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/tags/18801 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/tags/22803 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/tags/41557 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/tags/41566 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/tags/6591 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URL https://www.avery.com/products/tags/80503 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-15 14:03:02 [avery] (PID: 79) WARNING: 323 URLs were not found in the `catalog_urls` table. 2025-11-15 14:03:02 [avery] (PID: 79) INFO: URLs not found and not flagged: {'https://www.avery.com/products/labels/8202', 'https://www.avery.com/products/binders/79496', 'https://www.avery.com/products/binders/3218', 'https://www.avery.com/products/folder-pockets/47691', 'https://www.avery.com/products/binders/17409', 'https://www.avery.com/products/dividers/11408', 'https://www.avery.com/products/dividers/11276', 'https://www.avery.com/products/labels/5069', 'https://www.avery.com/products/folder-pockets/47796', 'https://www.avery.com/products/folder-pockets/47359', 'https://www.avery.com/products/labels/85783', 'https://www.avery.com/products/tags/41566', 'https://www.avery.com/products/labels/4033', 'https://www.avery.com/products/tags/11015', 'https://www.avery.com/products/binders/21007', 'https://www.avery.com/products/labels/6970', 'https://www.avery.com/products/binders/79492', 'https://www.avery.com/products/markers/9232', 'https://www.avery.com/products/labels/41457', 'https://www.avery.com/products/labels/6740', 'https://www.avery.com/products/binders/03239', 'https://www.avery.com/products/binders/79832', 'https://www.avery.com/products/binders/21011', 'https://www.avery.com/products/tags/15372', 'https://www.avery.com/products/binders/23023', 'https://www.avery.com/products/name-badges/74651', 'https://www.avery.com/products/labels/80509', 'https://www.avery.com/products/labels/48126', 'https://www.avery.com/products/dividers/11700', 'https://www.avery.com/products/binders/79344', 'https://www.avery.com/products/markers/29857', 'https://www.avery.com/products/folder-pockets/47693', 'https://www.avery.com/products/labels/98122', 'https://www.avery.com/products/dividers/11144', 'https://www.avery.com/products/dividers/11998', 'https://www.avery.com/products/folder-pockets/40218', 'https://www.avery.com/products/labels/41560', 'https://www.avery.com/products/labels/41453', 'https://www.avery.com/products/binders/3219', 'https://www.avery.com/products/dividers/11275', 'https://www.avery.com/products/binders/27356', 'https://www.avery.com/products/dividers/81825', 'https://www.avery.com/products/dividers/11529', 'https://www.avery.com/products/labels/40192', 'https://www.avery.com/products/name-badges/74558', 'https://www.avery.com/products/dividers/41416', 'https://www.avery.com/products/folder-pockets/72052', 'https://www.avery.com/products/labels/6540', 'https://www.avery.com/products/dividers/11415', 'https://www.avery.com/products/labels/98110', 'https://www.avery.com/products/tags/12302', 'https://www.avery.com/products/binders/26728', 'https://www.avery.com/products/labels/8590', 'https://www.avery.com/products/labels/6476', 'https://www.avery.com/products/binders/68031', 'https://www.avery.com/products/binders/15725', 'https://www.avery.com/products/dividers/16061', 'https://www.avery.com/products/binders/79409', 'https://www.avery.com/products/tags/22803', 'https://www.avery.com/products/dividers/12171', 'https://www.avery.com/products/folder-pockets/40213', 'https://www.avery.com/products/labels/6002', 'https://www.avery.com/products/folder-pockets/47760', 'https://www.avery.com/products/name-badges/5397', 'https://www.avery.com/products/folder-pockets/47356', 'https://www.avery.com/products/tags/12506', 'https://www.avery.com/products/binders/21010', 'https://www.avery.com/products/labels/5226', 'https://www.avery.com/products/highlighters/23550', 'https://www.avery.com/products/labels/5205', 'https://www.avery.com/products/folder-pockets/47701', 'https://www.avery.com/products/dividers/11625', 'https://www.avery.com/products/sheet-protectors/25410', 'https://www.avery.com/products/labels/4022', 'https://www.avery.com/products/highlighters/23521', 'https://www.avery.com/products/binders/03243', 'https://www.avery.com/products/folder-pockets/47344', 'https://www.avery.com/products/labels/41567', 'https://www.avery.com/products/labels/8117', 'https://www.avery.com/products/binders/21005', 'https://www.avery.com/products/labels/41459', 'https://www.avery.com/products/dividers/23800', 'https://www.avery.com/products/binders/68028', 'https://www.avery.com/products/binders/68040', 'https://www.avery.com/products/binders/17640', 'https://www.avery.com/products/binders/17570', 'https://www.avery.com/custom-printing/labels/floor-decals/index', 'https://www.avery.com/products/labels/22819', 'https://www.avery.com/products/markers/23556', 'https://www.avery.com/products/dividers/12451', 'https://www.avery.com/products/labels/28660', 'https://www.avery.com/products/binders/05850', 'https://www.avery.com/products/dividers/11626', 'https://www.avery.com/products/folder-pockets/73524', 'https://www.avery.com/products/labels/6583', 'https://www.avery.com/products/labels/41439', 'https://www.avery.com/products/binders/68030', 'https://www.avery.com/products/binders/79896', 'https://www.avery.com/products/labels/41432', 'https://www.avery.com/products/dividers/16187', 'https://www.avery.com/custom-printing/labels/calculator/asset-tag-tamper-evident', 'https://www.avery.com/products/folder-pockets/47665', 'https://www.avery.com/products/folder-pockets/47358', 'https://www.avery.com/products/labels/28663', 'https://www.avery.com/products/folder-pockets/73513', 'https://www.avery.com/products/binders/79369', 'https://www.avery.com/products/folder-pockets/47699', 'https://www.avery.com/products/tags/11004', 'https://www.avery.com/products/dividers/11451', 'https://www.avery.com/products/name-badges/74556', 'https://www.avery.com/products/dividers/11388', 'https://www.avery.com/products/labels/5068', 'https://www.avery.com/products/labels/41452', 'https://www.avery.com/products/binders/79892', 'https://www.avery.com/products/binders/06535', 'https://www.avery.com/products/labels/5199', 'https://www.avery.com/products/binders/17411', 'https://www.avery.com/products/binders/06635', 'https://www.avery.com/products/tags/12501', 'https://www.avery.com/products/tags/15371', 'https://www.avery.com/products/sheet-protectors/74170', 'https://www.avery.com/products/binders/79406', 'https://www.avery.com/products/labels/24603', 'https://www.avery.com/products/sheet-protectors/73907', 'https://www.avery.com/products/binders/79335', 'https://www.avery.com/products/dividers/7720', 'https://www.avery.com/products/tags/12445', 'https://www.avery.com/products/dividers/24921', 'https://www.avery.com/products/binders/17345', 'https://www.avery.com/products/binders/79494', 'https://www.avery.com/products/tags/41557', 'https://www.avery.com/products/binders/79829', 'https://www.avery.com/products/folder-pockets/47314', 'https://www.avery.com/products/tags/11014', 'https://www.avery.com/products/folder-pockets/72279', 'https://www.avery.com/products/dividers/12453', 'https://www.avery.com/products/binders/79830', 'https://www.avery.com/products/binders/11778', 'https://www.avery.com/products/binders/79495', 'https://www.avery.com/products/labels/5693', 'https://www.avery.com/products/dividers/11413', 'https://www.avery.com/products/folder-pockets/40216', 'https://www.avery.com/products/sheet-protectors/73899', 'https://www.avery.com/products/folder-pockets/47762', 'https://www.avery.com/products/binders/05303', 'https://www.avery.com/products/binders/68034', 'https://www.avery.com/products/tags/11030', 'https://www.avery.com/products/highlighters/23555', 'https://www.avery.com/products/binders/79405', 'https://www.avery.com/products/binders/68027', 'https://www.avery.com/products/labels/5568', 'https://www.avery.com/products/folder-pockets/73504', 'https://www.avery.com/products/markers/9233', 'https://www.avery.com/products/binders/21018', 'https://www.avery.com/products/folder-pockets/47867', 'https://www.avery.com/products/labels/5206', 'https://www.avery.com/products/folder-pockets/47364', 'https://www.avery.com/products/binders/17439', 'https://www.avery.com/products/folder-pockets/47326', 'https://www.avery.com/products/binders/79493', 'https://www.avery.com/products/sheet-protectors/73610', 'https://www.avery.com/products/sheet-protectors/75201', 'https://www.avery.com/products/binders/10803', 'https://www.avery.com/products/folder-pockets/47799', 'https://www.avery.com/products/dividers/11567', 'https://www.avery.com/products/markers/07912', 'https://www.avery.com/products/dividers/11192', 'https://www.avery.com/products/binders/21014', 'https://www.avery.com/products/binders/03002', 'https://www.avery.com/products/dividers/11453', 'https://www.avery.com/products/labels/6471', 'https://www.avery.com/products/dividers/11185', 'https://www.avery.com/products/dividers/11995', 'https://www.avery.com/products/tags/11035', 'https://www.avery.com/products/tags/6591', 'https://www.avery.com/products/folder-pockets/47342', 'https://www.avery.com/products/binders/27275', 'https://www.avery.com/products/binders/18720', 'https://www.avery.com/products/labels/48165', 'https://www.avery.com/products/labels/5493', 'https://www.avery.com/products/binders/05004', 'https://www.avery.com/products/labels/83170', 'https://www.avery.com/products/pens/49988', 'https://www.avery.com/products/folder-pockets/73506', 'https://www.avery.com/products/markers/27176', 'https://www.avery.com/products/labels/41454', 'https://www.avery.com/products/folder-pockets/47346', 'https://www.avery.com/products/binders/17407', 'https://www.avery.com/products/binders/17416', 'https://www.avery.com/products/binders/21009', 'https://www.avery.com/products/tags/11012', 'https://www.avery.com/products/sheet-protectors/74172', 'https://www.avery.com/products/binders/27050', 'https://www.avery.com/products/binders/17408', 'https://www.avery.com/products/pens/49986', 'https://www.avery.com/products/highlighters/07745', 'https://www.avery.com/products/labels/41429', 'https://www.avery.com/products/dividers/07712', 'https://www.avery.com/products/labels/41455', 'https://www.avery.com/products/labels/5589', 'https://www.avery.com/products/binders/11780', 'https://www.avery.com/products/labels/88127', 'https://www.avery.com/products/binders/23000', 'https://www.avery.com/products/binders/15720', 'https://www.avery.com/products/labels/4383', 'https://www.avery.com/products/labels/41448', 'https://www.avery.com/products/binders/79831', 'https://www.avery.com/products/binders/79825', 'https://www.avery.com/products/tags/11555', 'https://www.avery.com/products/labels/6723', 'https://www.avery.com/products/tags/11011', 'https://www.avery.com/products/dividers/11996', 'https://www.avery.com/products/tags/80503', 'https://www.avery.com/products/labels/2162', 'https://www.avery.com/products/binders/17443', 'https://www.avery.com/products/folder-pockets/47826', 'https://www.avery.com/products/labels/58164', 'https://www.avery.com/custom-printing/labels/calculator/material-film-white-glossy-vinyl-durable-ultra', 'https://www.avery.com/products/binders/17347', 'https://www.avery.com/products/binders/06532', 'https://www.avery.com/products/labels/4144', 'https://www.avery.com/products/sheet-protectors/74301', 'https://www.avery.com/products/labels/41450', 'https://www.avery.com/products/labels/41565', 'https://www.avery.com/products/binders/26750', 'https://www.avery.com/products/binders/68039', 'https://www.avery.com/products/binders/05213', 'https://www.avery.com/products/dividers/81824', 'https://www.avery.com/products/name-badges/73471', 'https://www.avery.com/products/binders/79499', 'https://www.avery.com/products/labels/4161', 'https://www.avery.com/products/labels/55360', 'https://www.avery.com/products/folder-pockets/40215', 'https://www.avery.com/products/labels/48267', 'https://www.avery.com/products/name-badges/73472', 'https://www.avery.com/products/binders/17163', 'https://www.avery.com/products/labels/41570', 'https://www.avery.com/products/labels/5105', 'https://www.avery.com/products/labels/8206', 'https://www.avery.com/products/binders/17410', 'https://www.avery.com/products/labels/41456', 'https://www.avery.com/products/binders/79333', 'https://www.avery.com/products/sheet-protectors/75538', 'https://www.avery.com/products/labels/2180', 'https://www.avery.com/products/binders/27052', 'https://www.avery.com/products/gluesticks/00102', 'https://www.avery.com/products/folder-pockets/6574', 'https://www.avery.com/products/tags/11013', 'https://www.avery.com/products/labels/45280', 'https://www.avery.com/products/folder-pockets/73505', 'https://www.avery.com/products/binders/21015', 'https://www.avery.com/products/folder-pockets/47038', 'https://www.avery.com/products/binders/68026', 'https://www.avery.com/products/tags/11027', 'https://www.avery.com/products/labels/48260', 'https://www.avery.com/products/labels/41412', 'https://www.avery.com/products/folder-pockets/40217', 'https://www.avery.com/products/binders/79332', 'https://www.avery.com/products/labels/6573', 'https://www.avery.com/products/binders/17346', 'https://www.avery.com/products/labels/41431', 'https://www.avery.com/products/folder-pockets/73514', 'https://www.avery.com/products/dividers/11409', 'https://www.avery.com/products/binders/21004', 'https://www.avery.com/products/dividers/11701', 'https://www.avery.com/products/labels/8699', 'https://www.avery.com/products/tags/12203', 'https://www.avery.com/products/labels/22801', 'https://www.avery.com/products/dividers/16184', 'https://www.avery.com/products/tags/18801', 'https://www.avery.com/products/folder-pockets/47347', 'https://www.avery.com/products/labels/41447', 'https://www.avery.com/products/name-badges/45143', 'https://www.avery.com/products/tags/14314', 'https://www.avery.com/products/labels/6478', 'https://www.avery.com/products/binders/15761', 'https://www.avery.com/products/labels/5694', 'https://www.avery.com/products/binders/21002', 'https://www.avery.com/products/labels/55464', 'https://www.avery.com/products/binders/17250', 'https://www.avery.com/products/binders/17438', 'https://www.avery.com/products/folder-pockets/73516', 'https://www.avery.com/products/folder-pockets/73518', 'https://www.avery.com/products/tags/15370', 'https://www.avery.com/products/folder-pockets/73515', 'https://www.avery.com/products/folder-pockets/47852', 'https://www.avery.com/products/labels/4015', 'https://www.avery.com/products/dividers/11512', 'https://www.avery.com/products/dividers/11150', 'https://www.avery.com/products/binders/11718', 'https://www.avery.com/products/labels/41559', 'https://www.avery.com/products/tags/11017', 'https://www.avery.com/products/binders/21008', 'https://www.avery.com/products/labels/6695', 'https://www.avery.com/products/folder-pockets/47798', 'https://www.avery.com/products/dividers/11525', 'https://www.avery.com/products/labels/55463', 'https://www.avery.com/products/binders/68029', 'https://www.avery.com/products/folder-pockets/73503', 'https://www.avery.com/products/binders/3220', 'https://www.avery.com/products/folder-pockets/40212', 'https://www.avery.com/products/pens/49987', 'https://www.avery.com/products/binders/79334', 'https://www.avery.com/products/folder-pockets/72268', 'https://www.avery.com/products/binders/27051', 'https://www.avery.com/products/binders/18721', 'https://www.avery.com/products/folder-pockets/73520', 'https://www.avery.com/products/tags/11031', 'https://www.avery.com/products/labels/4020', 'https://www.avery.com/products/sheet-protectors/05277', 'https://www.avery.com/products/labels/8891', 'https://www.avery.com/products/folder-pockets/47761', 'https://www.avery.com/products/labels/6735', 'https://www.avery.com/products/binders/50000', 'https://www.avery.com/products/labels/41427', 'https://www.avery.com/products/labels/06107', 'https://www.avery.com/products/labels/41451', 'https://www.avery.com/products/folder-pockets/47879', 'https://www.avery.com/products/name-badges/74552', 'https://www.avery.com/products/tags/11038', 'https://www.avery.com/products/binders/21016', 'https://www.avery.com/products/folder-pockets/47325', 'https://www.avery.com/products/folder-pockets/47759'} 2025-11-15 14:03:02 [avery] (PID: 79) INFO: Finished processing 'not found' URLs in the `catalog_urls` table. 2025-11-15 14:03:03 [scrapy.extensions.feedexport] (PID: 79) INFO: Stored bq feed (140 items) in: bq://response-elt.scraper_data.catalog_item_scrape/batch:7 2025-11-15 14:03:03 [scrapy.statscollectors] (PID: 79) INFO: Dumping Scrapy stats: {'HeadersSpooferDownloaderMiddleware/spoofed': 4750, 'NotFoundHandlerSpiderMiddleware/NotProductPage': 56, 'NotFoundHandlerSpiderMiddleware/ProductNotFound': 323, 'NotFoundHandlerSpiderMiddleware/TypeError': 4, 'NotFoundHandlerSpiderMiddleware/not_found/404_response': 323, 'NotFoundHandlerSpiderMiddleware/not_found/ignored': 256, 'NotFoundHandlerSpiderMiddleware/not_found/retrieved': 256, 'big_query/url': 2312, 'downloader/request_bytes': 7663263, 'downloader/request_count': 4750, 'downloader/request_method_count/GET': 2883, 'downloader/request_method_count/POST': 1867, 'downloader/response_bytes': 96767069, 'downloader/response_count': 4750, 'downloader/response_status_count/200': 3908, 'downloader/response_status_count/301': 515, 'downloader/response_status_count/308': 4, 'downloader/response_status_count/404': 323, 'dupefilter/filtered': 91, 'elapsed_time_seconds': 174.943872, 'feedexport/success_count/BigQueryFeedStorage': 7, 'finish_reason': 'finished', 'finish_time': datetime.datetime(2025, 11, 15, 14, 3, 0, 973237, tzinfo=datetime.timezone.utc), 'httpcompression/response_bytes': 880233833, 'httpcompression/response_count': 4231, 'item_scraped_count': 6140, 'items_per_minute': None, 'log_count/ERROR': 340, 'log_count/INFO': 310, 'log_count/WARNING': 2449, 'memusage/max': 667672576, 'memusage/startup': 125759488, 'poet/injector/catalog_extraction.pages.avery.AveryMultiProductsPageObject': 1867, 'product_status/discontinued': 642, 'proxy_manager/ignored/proxy_defined': 519, 'proxy_manager/processed': 4231, 'request_depth_max': 2, 'response_received_count': 4231, 'responses_per_minute': None, 'scheduler/dequeued': 4750, 'scheduler/dequeued/memory': 4750, 'scheduler/enqueued': 4750, 'scheduler/enqueued/memory': 4750, 'spider_exceptions/NotProductPage': 14, 'spider_exceptions/ProductNotFound': 323, 'spider_exceptions/TypeError': 1, 'spidermon/validation/fields': 128940, 'spidermon/validation/items': 6140, 'spidermon/validation/validators': 1, 'spidermon/validation/validators/item/jsonschema': True, 'spidermon_field_coverage/dict/brand': 1.0, 'spidermon_field_coverage/dict/categories': 1.0, 'spidermon_field_coverage/dict/countryOfOrigin': 1.0, 'spidermon_field_coverage/dict/description': 1.0, 'spidermon_field_coverage/dict/imageUrl': 1.0, 'spidermon_field_coverage/dict/inStock': 1.0, 'spidermon_field_coverage/dict/isFreeShipping': 1.0, 'spidermon_field_coverage/dict/leadTime': 1.0, 'spidermon_field_coverage/dict/manufacturer': 1.0, 'spidermon_field_coverage/dict/manufacturerSku': 1.0, 'spidermon_field_coverage/dict/name': 1.0, 'spidermon_field_coverage/dict/packagingIncrement': 1.0, 'spidermon_field_coverage/dict/prices': 1.0, 'spidermon_field_coverage/dict/productStatus': 1.0, 'spidermon_field_coverage/dict/relatedSkus': 1.0, 'spidermon_field_coverage/dict/specifications': 1.0, 'spidermon_field_coverage/dict/supplier': 1.0, 'spidermon_field_coverage/dict/supplierSku': 1.0, 'spidermon_field_coverage/dict/uom': 1.0, 'spidermon_field_coverage/dict/url': 1.0, 'spidermon_field_coverage/dict/weight': 1.0, 'spidermon_item_scraped_count': 6140, 'spidermon_item_scraped_count/dict': 6140, 'spidermon_item_scraped_count/dict/brand': 6140, 'spidermon_item_scraped_count/dict/categories': 6140, 'spidermon_item_scraped_count/dict/countryOfOrigin': 6140, 'spidermon_item_scraped_count/dict/description': 6140, 'spidermon_item_scraped_count/dict/imageUrl': 6140, 'spidermon_item_scraped_count/dict/inStock': 6140, 'spidermon_item_scraped_count/dict/isFreeShipping': 6140, 'spidermon_item_scraped_count/dict/leadTime': 6140, 'spidermon_item_scraped_count/dict/manufacturer': 6140, 'spidermon_item_scraped_count/dict/manufacturerSku': 6140, 'spidermon_item_scraped_count/dict/name': 6140, 'spidermon_item_scraped_count/dict/packagingIncrement': 6140, 'spidermon_item_scraped_count/dict/prices': 6140, 'spidermon_item_scraped_count/dict/productStatus': 6140, 'spidermon_item_scraped_count/dict/relatedSkus': 6140, 'spidermon_item_scraped_count/dict/specifications': 6140, 'spidermon_item_scraped_count/dict/supplier': 6140, 'spidermon_item_scraped_count/dict/supplierSku': 6140, 'spidermon_item_scraped_count/dict/uom': 6140, 'spidermon_item_scraped_count/dict/url': 6140, 'spidermon_item_scraped_count/dict/weight': 6140, 'start_requests/big_query': 2312, 'start_time': datetime.datetime(2025, 11, 15, 14, 0, 6, 29365, tzinfo=datetime.timezone.utc)} 2025-11-15 14:03:03 [scrapy.core.engine] (PID: 79) INFO: Spider closed (finished)