web / lib/services/progress-tracking
lib/services/progress-tracking
Provides detailed progress tracking with per-stage metrics for import jobs.
This service provides comprehensive progress tracking including:
- Per-stage progress with batch information
- Processing rates (rows/second)
- Time estimates (ETA for completion)
- Weighted overall progress based on stage time estimates
Key responsibilities:
- Initialize detailed progress structures for all stages
- Track progress within each stage and batch
- Calculate weighted overall progress
- Estimate time remaining based on processing rates
Classes
ProgressTrackingService
Centralized progress tracking service for detailed per-stage tracking.
Constructors
Constructor
new ProgressTrackingService():
ProgressTrackingService
Returns
Methods
initializeStageProgress()
staticinitializeStageProgress(payload,jobId,totalRows):Promise<void>
Initialize progress tracking for all stages at job start.
This should be called once when the job begins (during ANALYZE_DUPLICATES). It sets up the progress structure for all 8 stages with “pending” status.
Parameters
payload
BasePayload
Payload instance
jobId
Import job ID
string | number
totalRows
number
Total number of rows in the import file
Returns
Promise<void>
startStage()
staticstartStage(payload,jobId,stage,rowsTotal):Promise<void>
Mark a stage as started and update its metadata.
This should be called when a job handler begins processing a stage. It marks the stage as “in_progress” and sets the start timestamp.
Parameters
payload
BasePayload
Payload instance
jobId
Import job ID
string | number
stage
Processing stage name
rowsTotal
number
Total rows for this stage (may differ from file total after deduplication)
Returns
Promise<void>
updateStageProgress()
staticupdateStageProgress(payload,job,stage,rowsProcessed,currentBatchRows):Promise<void>
Update progress within a stage.
This should be called after processing each batch to update:
- Rows processed
- Current batch progress
- Processing rate
- Time remaining estimate
Parameters
payload
BasePayload
Payload instance
job
ImportJob
stage
Processing stage name
rowsProcessed
number
Total rows processed so far in this stage
currentBatchRows
number
Rows processed in current batch
Returns
Promise<void>
completeBatch()
staticcompleteBatch(payload,job,stage,batchNumber):Promise<void>
Mark a batch as completed within a stage.
This should be called after each batch is fully processed to:
- Increment batch counter
- Reset current batch row count
Parameters
payload
BasePayload
Payload instance
job
ImportJob
stage
Processing stage name
batchNumber
number
Batch number just completed (1-indexed)
Returns
Promise<void>
completeStage()
staticcompleteStage(payload,jobId,stage):Promise<void>
Mark a stage as completed.
This should be called when a stage finishes successfully. It marks the stage as “completed” and sets the completion timestamp.
Parameters
payload
BasePayload
Payload instance
jobId
Import job ID
string | number
stage
Processing stage name
Returns
Promise<void>
skipStage()
staticskipStage(payload,jobId,stage):Promise<void>
Mark a stage as skipped.
This should be called when a stage is intentionally skipped (e.g., validation passed, no geocoding needed).
Parameters
payload
BasePayload
Payload instance
jobId
Import job ID
string | number
stage
Processing stage name
Returns
Promise<void>
calculateWeightedProgress()
staticcalculateWeightedProgress(stages):number
Calculate weighted overall progress across all stages.
Each stage contributes to overall progress based on its time weight. Stages with higher weights (slower stages) contribute more to overall %.
Parameters
stages
Record<string, StageProgress>
Stage progress data
Returns
number
Overall progress percentage (0-100)
estimateCompletionTime()
staticestimateCompletionTime(stages):Date|null
Estimate completion time based on current processing rates.
Uses processing rates from in-progress stages and estimates for remaining stages based on average rate and stage weights.
Parameters
stages
Record<string, StageProgress>
Stage progress data
Returns
Date | null
Estimated completion timestamp or null if cannot estimate
getBatchSizeForStage()
staticgetBatchSizeForStage(stage):number|null
Get the batch size for a specific stage.
Parameters
stage
Processing stage name
Returns
number | null
Batch size for the stage, or null if stage doesn’t use batching
updatePostDeduplicationTotals()
staticupdatePostDeduplicationTotals(payload,jobId,uniqueRows):Promise<void>
Update row totals for post-deduplication stages.
After deduplication, the total rows for remaining stages should be updated to reflect only unique rows.
Parameters
payload
BasePayload
Payload instance
jobId
Import job ID
string | number
uniqueRows
number
Number of unique rows after deduplication
Returns
Promise<void>