Transforms

Transforms modify incoming data during import — before schema detection and event creation. Use them to rename fields, parse dates, split columns, or convert JSON arrays into tags.

Transforms are configured per dataset and applied to every row during import. They run in order: the output of one transform feeds into the next.

Transform Types

Rename

Change a field name. Useful when your source renames columns between updates.

transforms:
  - type: rename
    from: event_name
    to: title

Parse Date

Convert date strings from a specific format into ISO 8601.

transforms:
  - type: date-parse
    from: datum
    inputFormat: DD.MM.YYYY
    outputFormat: YYYY-MM-DD

String Operation

Apply text operations: uppercase, lowercase, find-and-replace, or safe expressions.

transforms:
  - type: string-op
    from: status
    operation: lowercase

Concatenate

Join multiple fields into one with a separator.

transforms:
  - type: concatenate
    fromFields: [street, city, zip]
    separator: ", "
    to: full_address

Split

Split one field into multiple fields by a delimiter.

transforms:
  - type: split
    from: coordinates
    delimiter: ","
    toFields: [latitude, longitude]

Parse JSON Array

Convert a JSON-stringified array back into a native array. Used when a JSON API returns array fields (like categories or tags) that get serialized to strings during CSV conversion.

transforms:
  - type: parse-json-array
    from: category

Before: "[\"Sport\",\"Kultur\",\"Erwachsene\"]" (string)

After: ["Sport", "Kultur", "Erwachsene"] (native array, rendered as tag chips)

Split to Array

Split a delimited string into one array-valued field, rather than the multiple output columns created by Split. Whitespace around entries and empty entries are removed. The delimiter defaults to a comma; optional to writes to a different field.

transforms:
  - type: split-to-array
    from: categories
    delimiter: ";"

For example, "Music; Theatre; ; Sport" becomes ["Music", "Theatre", "Sport"].

Extract

Extract part of a string into another field using a regular expression. group selects the capture group (default: 1); the source field is unchanged. If the pattern does not match, no output is written. Unsafe regex patterns are rejected.

transforms:
  - type: extract
    from: source_url
    to: event_id
    pattern: "/events/([0-9]+)"
    group: 1

For example, /events/12345 produces event_id: "12345".

Pre-Processing

For JSON API sources, you can group records before they enter the import pipeline. This is useful when an API returns repeated entries for recurring events — one row per day instead of a single event with a date range.

Example: Bonn Event Calendar

The Bonn API returns 5,001 entries for 747 unique events. A daily exhibition appears as 291 separate rows (one per day). Pre-processing collapses them:

source:
  url: "https://www.bonn.de/citykey/events-json.php"
  format: json
  preProcessing:
    groupBy: uid
    mergeFields:
      startDate: min
      endDate: max

Result: 747 events with correct date ranges. An exhibition running March 2026 to February 2027 shows startDate: 2026-03-27 and endDate: 2027-02-28 instead of a single day.

Configuring Transforms

In Data Package YAMLs

Add a transforms array to your data package manifest:

# config/data-packages/bonn-veranstaltungen.yml
transforms:
  - type: parse-json-array
    from: category
 
source:
  url: "https://www.bonn.de/citykey/events-json.php"
  format: json
  preProcessing:
    groupBy: uid
    mergeFields:
      startDate: min
      endDate: max

In the Import Wizard

On the field-mapping step, add or edit transforms in the column table, or open the visual editor. Columns created by transforms can be assigned to event fields such as Title. When you remove or disable a transform, assignments to outputs that are no longer available are cleared; assign any missing required fields before continuing.

If the wizard offers a matching dataset configuration, Reuse import settings loads its transforms and available field assignments. Only settings editable in the wizard are reused; admin-only settings, such as whether duplicate detection is enabled, are not copied. New datasets use their defaults for those settings, and existing datasets retain their current values. Review the imported settings against the new file before starting the import.

Processing Order

Source data (JSON/CSV rows)
  → Pre-processing (group-by, date merge)     ← JSON APIs only
    → CSV conversion
      → Transforms (rename, parse, split...)   ← per row
        → Schema detection
          → Geocoding
            → Event creation

Transforms run after pre-processing but before schema detection. This means the schema sees the transformed field names and values.