Scheduled Imports

Fetch data from a URL automatically on a recurring schedule — hourly, daily, weekly, or custom cron expressions. The fetched data goes through the same pipeline as a manual file upload.

Setup

1. In the Upload step of the import wizard, choose URL as your data source and paste the URL
2. For a protected URL, set the authentication there (it can be adjusted again in the Schedule step)
3. Configure the dataset and field mappings as usual
4. In the Schedule step, choose Repeat on schedule and set the frequency (e.g., every 6 hours, daily at midnight)
5. Review and start the import

How It Works

A background job checks for due schedules every minute. When a schedule fires, it fetches the URL and runs the full import pipeline — schema detection, deduplication, geocoding, event creation.

If the source schema has changed between runs, you may be asked to approve the new mapping before processing continues. While a run waits for your review, the schedule does not fire again, so reviews cannot pile up. Approving or discarding the review releases it.

Each sheet of a file is imported on its own. If one sheet fails while others succeed, the run counts as failed, and the successful sheets keep their events.

If a schedule has invalid timing settings or keeps failing until it exhausts its retry budget, TimeTiles disables it, records the error, and emails the owner. Fix the schedule and re-enable it to resume automatic runs.

Cron Timezones and Daylight Saving

Custom five-field cron schedules use Croner, the same engine used by Payload. They use the configured timezone, defaulting to UTC. UTC schedules have no daylight-saving transitions.

For local-time schedules, Croner shifts a nonexistent spring-transition time forward by the clock change. For example, 30 2 * * * in Europe/Berlin runs at 03:30 on the spring-transition day. During the autumn transition, the repeated hour does not produce a second occurrence: after the first 02:30 has passed, the next scheduled 02:30 is on the following day.

Use UTC if the schedule should be independent of these local clock changes.

Authentication

Scheduled imports support several methods for protected URLs:

The wizard offers None, API Key, Bearer Token, and Basic Auth. OAuth 2.0 is configured on the scheduled import in the admin dashboard.

JSON API Pagination

For paginated JSON sources, the scheduled import’s JSON API configuration controls page size and stopping limits:

• limitValue: records requested per page (default 100).
• maxPages: maximum pages fetched per run (default 50, hard cap 500).
• maxRecords: maximum records collected across pages (default 100,000). You can explicitly raise this for larger sources; it is not a fixed ceiling.

These counts must be positive integers. Reaching a configured limit stops the fetch; if the final page exceeds the remaining record allowance, only the records that fit are retained. Choose limits large enough for the intended source, otherwise the run imports only a prefix of it. User quotas still apply separately.

Managing Schedules

From the Scheduled tab of your imports page (/account/imports), each schedule’s actions menu offers:

• Enable / Disable
• Run now
• Edit, which reopens the wizard with the current settings
• Delete

Each schedule also shows its last and next run and the history of past runs.

Admins can view all schedules at /dashboard/collections/scheduled-ingests.

Webhook Triggers

Each scheduled import can optionally expose a webhook URL. An admin or editor enables it on the scheduled import in the admin dashboard. POST to it to trigger an immediate run — useful for CI/CD pipelines or external automation:

curl -X POST https://your-instance.com/api/webhooks/trigger/{token}

No authentication header needed — the token in the URL is the credential.

The full webhook URL is shown only once when the webhook is first enabled or rotated. Copy it immediately. If you lose it, disable and re-enable the webhook to rotate the token and generate a new URL.

Caching

Scheduled imports use HTTP caching (RFC 7234) to avoid re-downloading unchanged data. If the source server sends appropriate Cache-Control headers, TimeTiles respects them. See Usage Limits for cache configuration.

Next Steps

• File Upload — One-time manual imports
• Scrapers — Custom scripts for non-tabular data sources
• Exploring Data — Map, timeline, filters, and sharing