web / lib/services/schema-builder/language-detection

lib/services/schema-builder/language-detection

Language detection service for import wizard.

Uses the franc library to detect the language of sample data, enabling language-aware field mapping detection.

Interfaces

LanguageDetectionResult

Result of language detection

Properties

code

code: string

ISO 639-3 language code (e.g., ‘eng’, ‘deu’, ‘fra’)

name

name: string

Human-readable language name

confidence

confidence: number

Confidence score from 0-1

isReliable

isReliable: boolean

Whether the detection is considered reliable (confidence > 0.5)

Type Aliases

SupportedLanguage

SupportedLanguage = typeof SUPPORTED_LANGUAGES[number]

Variables

SUPPORTED_LANGUAGES

const SUPPORTED_LANGUAGES: readonly ["eng", "deu", "fra", "spa", "ita", "nld", "por"]

Supported languages for field mapping detection. These must match the languages defined in FIELD_PATTERNS in field-mapping-detection.ts

LANGUAGE_NAMES

const LANGUAGE_NAMES: Record<string, string>

Human-readable names for supported languages

Functions

extractTextForLanguageDetection()

extractTextForLanguageDetection(sampleData, headers): string

Extracts text content suitable for language detection from sample data.

Filters out values that are not useful for language detection:

Non-string values (numbers, booleans, null)
Very short strings (likely IDs or codes)
Email addresses
URLs
ISO dates
Coordinate-like numbers

Parameters

sampleData

Record<string, unknown>[]

Array of sample data rows

headers

string[]

Column headers

Returns

string

Combined text content for language detection

detectLanguage()

detectLanguage(text): LanguageDetectionResult

Detects the language of text using the franc library.

Parameters

text

string

Text to analyze

Returns

LanguageDetectionResult

Language detection result

detectLanguageFromSamples()

detectLanguageFromSamples(sampleData, headers): LanguageDetectionResult

Detects language from sample data and headers.

This is the main entry point for language detection in the import wizard.

Parameters

sampleData

Record<string, unknown>[]

Array of sample data rows from uploaded file

headers

string[]

Column headers from uploaded file

Returns

LanguageDetectionResult

Language detection result

isSupportedLanguage()

isSupportedLanguage(code): code is “eng” | “deu” | “fra” | “spa” | “ita” | “nld” | “por”

Checks if a language code is supported.

Parameters

code

string

Returns

code is “eng” | “deu” | “fra” | “spa” | “ita” | “nld” | “por”