web / lib/services/progress-tracking
lib/services/progress-tracking
Provides detailed progress tracking with per-stage metrics for import jobs.
This service provides comprehensive progress tracking including:
- Per-stage progress with batch information
- Processing rates (rows/second)
- Time estimates (ETA for completion)
- Weighted overall progress based on stage time estimates
Key responsibilities:
- Initialize detailed progress structures for all stages
- Track progress within each stage and batch
- Calculate weighted overall progress
- Estimate time remaining based on processing rates
Classes
ProgressTrackingService
Centralized progress tracking service for detailed per-stage tracking.
Constructors
Constructor
new ProgressTrackingService():
ProgressTrackingService
Returns
Methods
initializeStageProgress()
staticinitializeStageProgress(payload,jobId,totalRows):Promise<void>
Initialize progress tracking for all stages at job start.
This should be called once when the job begins (during ANALYZE_DUPLICATES). It sets up the progress structure for all 8 stages with “pending” status.
Parameters
payload
BasePayload
Payload instance
jobId
Import job ID
string | number
totalRows
number
Total number of rows in the import file
Returns
Promise<void>
startStage()
staticstartStage(payload,jobId,stage,rowsTotal):Promise<void>
Mark a stage as started and update its metadata.
This should be called when a job handler begins processing a stage. It marks the stage as “in_progress” and sets the start timestamp.
Parameters
payload
BasePayload
Payload instance
jobId
Import job ID
string | number
stage
Processing stage name
rowsTotal
number
Total rows for this stage (may differ from file total after deduplication)
Returns
Promise<void>
updateStageProgress()
staticupdateStageProgress(payload,jobId,stage,rowsProcessed,currentBatchRows):Promise<void>
Update progress within a stage.
This should be called after processing each batch to update:
- Rows processed
- Current batch progress
- Processing rate
- Time remaining estimate
Parameters
payload
BasePayload
Payload instance
jobId
Import job ID
string | number
stage
Processing stage name
rowsProcessed
number
Total rows processed so far in this stage
currentBatchRows
number
Rows processed in current batch
Returns
Promise<void>
completeBatch()
staticcompleteBatch(payload,jobId,stage,batchNumber):Promise<void>
Mark a batch as completed within a stage.
This should be called after each batch is fully processed to:
- Increment batch counter
- Reset current batch row count
Parameters
payload
BasePayload
Payload instance
jobId
Import job ID
string | number
stage
Processing stage name
batchNumber
number
Batch number just completed (1-indexed)
Returns
Promise<void>
completeStage()
staticcompleteStage(payload,jobId,stage):Promise<void>
Mark a stage as completed.
This should be called when a stage finishes successfully. It marks the stage as “completed” and sets the completion timestamp.
Parameters
payload
BasePayload
Payload instance
jobId
Import job ID
string | number
stage
Processing stage name
Returns
Promise<void>
skipStage()
staticskipStage(payload,jobId,stage):Promise<void>
Mark a stage as skipped.
This should be called when a stage is intentionally skipped (e.g., validation passed, no geocoding needed).
Parameters
payload
BasePayload
Payload instance
jobId
Import job ID
string | number
stage
Processing stage name
Returns
Promise<void>
calculateWeightedProgress()
staticcalculateWeightedProgress(stages):number
Calculate weighted overall progress across all stages.
Each stage contributes to overall progress based on its time weight. Stages with higher weights (slower stages) contribute more to overall %.
Parameters
stages
Record<string, StageProgress>
Stage progress data
Returns
number
Overall progress percentage (0-100)
estimateCompletionTime()
staticestimateCompletionTime(stages):null|Date
Estimate completion time based on current processing rates.
Uses processing rates from in-progress stages and estimates for remaining stages based on average rate and stage weights.
Parameters
stages
Record<string, StageProgress>
Stage progress data
Returns
null | Date
Estimated completion timestamp or null if cannot estimate
getBatchSizeForStage()
staticgetBatchSizeForStage(stage):null|number
Get the batch size for a specific stage.
Parameters
stage
Processing stage name
Returns
null | number
Batch size for the stage, or null if stage doesn’t use batching
updatePostDeduplicationTotals()
staticupdatePostDeduplicationTotals(payload,jobId,uniqueRows):Promise<void>
Update row totals for post-deduplication stages.
After deduplication, the total rows for remaining stages should be updated to reflect only unique rows.
Parameters
payload
BasePayload
Payload instance
jobId
Import job ID
string | number
uniqueRows
number
Number of unique rows after deduplication
Returns
Promise<void>