web / lib/jobs/handlers/cleanup-stuck-scrapers-job
lib/jobs/handlers/cleanup-stuck-scrapers-job
Job handler for cleaning up stuck scrapers.
Identifies and resets scrapers that have been stuck in “running” status
for too long (default 4 hours). The threshold is intentionally generous
because lastRunAt records the trigger/queue time, not when processing
actually started — there can be significant delay due to queue backlog
or worker restarts.
Before resetting, also checks whether a Payload job is still actively processing the scraper to avoid killing in-progress work.
Mirrors the behavior of cleanup-stuck-scheduled-ingests-job.ts.
Interfaces
CleanupStuckScrapersJobInput
Properties
stuckThresholdHours?
optionalstuckThresholdHours?:number
Hours after which a running scraper is considered stuck (default: 4).
Uses 4h because lastRunAt is the trigger time, not when processing started.
dryRun?
optionaldryRun?:boolean
Whether to run in dry-run mode (default: false)
Variables
cleanupStuckScrapersJob
constcleanupStuckScrapersJob:object
Type Declaration
slug
slug:
string="cleanup-stuck-scrapers"
schedule
schedule:
object[]
concurrency
concurrency: () =>
string
Returns
string
handler
handler: (
context) =>Promise<{ output: \{ success: boolean; skipped: boolean; reason: string; }; } |{ output: \{ success: boolean; totalRunning: number; stuckCount: number; resetCount: number; dryRun: boolean; errors: object[] \| undefined; }; }>
Parameters
context
Returns
Promise<{ output: \{ success: boolean; skipped: boolean; reason: string; }; } | { output: \{ success: boolean; totalRunning: number; stuckCount: number; resetCount: number; dryRun: boolean; errors: object[] \| undefined; }; }>