1, problem

Due to reptile error/production environment replacement, etc., the timing SCRAPY mission queue is waiting too much

(passhttp://127.0.0.1:6800/listjobs.json?project=myprojectQuery of PENDING’s reptile queue, there are 600+….)

2. Solution

Enter the user directory-> DBS directory, delete the scrapy project db file of Pending

Because of the project data (including crawling mission queue) in the DBS directory

as shown below (the scrapy project is mytoken, delete mytoken.db here)

Note: View official documentation of scrapyd (https://scrapyd.readthedocs.io/en/stable/config.html), we can find:

dbs_dir: The directory where the project databases will be stored (this includes the spider queues).

A sample configuration file with all default values is as follows, and the default is in the user directory

Reference document:https://scrapyd.readthedocs.io/en/stable/config.html

 ————- END ————