Scheduled Imports

Fetch data from a URL automatically on a recurring schedule — hourly, daily, weekly, or custom cron expressions. The fetched data goes through the same pipeline as a manual file upload.

Setup

1. In the Import wizard, choose URL as your data source
2. Paste the URL and configure schema and field mappings as usual
3. In the Schedule step, choose Repeat on schedule
4. Set the frequency (e.g., every 6 hours, daily at midnight)
5. Optionally add authentication headers for protected APIs
6. Save

How It Works

A background job checks for due schedules every minute. When a schedule fires, it fetches the URL and runs the full import pipeline — schema detection, deduplication, geocoding, event creation.

If the source schema has changed between runs, you may be asked to approve the new mapping before processing continues.

If a schedule has invalid timing settings or keeps failing until it exhausts its retry budget, TimeTiles disables it, records the error, and emails the owner. Fix the schedule and re-enable it to resume automatic runs.

Cron Timezones and Daylight Saving

Custom five-field cron schedules use Croner, the same engine used by Payload. They use the configured timezone, defaulting to UTC. UTC schedules have no daylight-saving transitions.

For local-time schedules, Croner shifts a nonexistent spring-transition time forward by the clock change. For example, 30 2 * * * in Europe/Berlin runs at 03:30 on the spring-transition day. During the autumn transition, the repeated hour does not produce a second occurrence: after the first 02:30 has passed, the next scheduled 02:30 is on the following day.

This replaces TimeTiles’ earlier custom cron behavior, which skipped nonexistent spring times and included the second occurrence of repeated autumn times. Use UTC if the schedule should be independent of these local clock changes.

Authentication

Scheduled imports support several methods for protected URLs:

JSON API Pagination

For paginated JSON sources, the scheduled import’s JSON API configuration controls page size and stopping limits:

• limitValue: records requested per page (default 100).
• maxPages: maximum pages fetched per run (default 50, hard cap 500).
• maxRecords: maximum records collected across pages (default 100,000). You can explicitly raise this for larger sources; it is not a fixed ceiling.

These counts must be positive integers. Reaching a configured limit stops the fetch; if the final page exceeds the remaining record allowance, only the records that fit are retained. Choose limits large enough for the intended source, otherwise the run imports only a prefix of it. User quotas still apply separately.

Managing Schedules

From the Scheduled Imports section in your account:

• Pause/resume a schedule
• Trigger an immediate run
• View history of past runs
• Delete a schedule

Admins can view all schedules at /dashboard/collections/scheduled-ingests.

Webhook Triggers

Each scheduled import can optionally expose a webhook URL. POST to it to trigger an immediate run — useful for CI/CD pipelines or external automation:

curl -X POST https://your-instance.com/api/webhooks/trigger/{token}

No authentication header needed — the token in the URL is the credential.

The full webhook URL is shown only once when the webhook is first enabled or rotated. Copy it immediately. If you lose it, disable and re-enable the webhook to rotate the token and generate a new URL.

Caching

Scheduled imports use HTTP caching (RFC 7234) to avoid re-downloading unchanged data. If the source server sends appropriate Cache-Control headers, TimeTiles respects them. See Usage Limits for cache configuration.

Next Steps

• File Upload — One-time manual imports
• Scrapers — Custom scripts for non-tabular data sources
• Exploring Data — Map, timeline, filters, and sharing