web / lib/types/import-transforms
lib/types/import-transforms
Types for import transform rules that handle field mapping during data import.
Import transforms enable schema evolution by allowing datasets to map incoming field names to their canonical schema field names, supporting scenarios like column renames, case variations, and JSON path changes.
Interfaces
ImportTransform
A transform rule that maps incoming data fields to schema fields.
Applied before schema validation and event creation to normalize incoming data structure to match the dataset’s canonical schema.
Properties
id
id:
string
Unique identifier for this transform rule
type
type:
"rename"
Type of transformation to apply
from
from:
string
Source field path in the incoming data.
Examples:
- “date” (flat CSV column)
- “user.email” (nested JSON path)
- “coordinates.0” (array element)
to
to:
string
Target field path in the dataset schema.
Examples:
- “start_date”
- “contact.email”
- “latitude”
active
active:
boolean
Whether this transform is currently active.
Inactive transforms are retained for history but not applied.
addedAt?
optionaladdedAt:Date
Timestamp when this transform was created
addedBy?
optionaladdedBy:string
User ID who created this transform
confidence?
optionalconfidence:number
Confidence score if auto-detected (0-100).
Higher scores indicate stronger evidence that this transform represents an intentional field rename rather than coincidence.
autoDetected
autoDetected:
boolean
Whether this transform was suggested by auto-detection.
Auto-detected transforms have confidence scores and explanations, while manually created transforms do not.
TransformSuggestion
A suggested transform detected by comparing schema versions.
When schema changes are detected (e.g., field removed + field added), the system analyzes whether this represents a rename and suggests a transform rule if confidence is high enough.
Properties
type
type:
"rename"
Type of transformation being suggested
from
from:
string
Source field path (what’s in the new import file).
This is the field name that appears in the incoming data that doesn’t match the existing schema.
to
to:
string
Target field path (what’s in the existing schema).
This is the canonical field name in the dataset schema that the incoming field should be mapped to.
confidence
confidence:
number
Confidence score (0-100) based on multiple factors:
- Name similarity (Levenshtein distance)
- Type compatibility
- Common rename patterns
- Position proximity in schema
Threshold: >= 70 for suggestion, >= 80 for high confidence
reason
reason:
string
Human-readable explanation of why this transform was suggested.
Examples:
- “Similar names (87%), Compatible types, Matches common rename pattern”
- “Similar names (95%), Compatible types”
Type Aliases
TransformType
TransformType =
"rename"
Types of transformations that can be applied to import data.
Currently supports:
- rename: Map source field path to target field path
Future extensions could include:
- split: Split one field into multiple fields
- merge: Combine multiple fields into one
- transform: Apply custom transformation function
- compute: Calculate field from other fields