Ingestion
5 pages filed under Decisions › ingestion. A decision is a choice made for the project, with its context, its options and what it affects.
| Decision | Date | First line | Links |
|---|---|---|---|
| Locale per source | 2026-09-12 | Each source declares its locale as a BCP 47 tag, defaulting to the project locale. A language is described by data, never by code: a pack holding its name, its apostrophes, its collation options, its plural suffix rules and its stopwords, which the locale selects together with the type prefixes of the profile for that language; word segmentation and collation come from the platform's Unicode implementation. The engine ships en and fr; a regional variant such as fr-CA uses the pack of its language until a plugin registers a more specific one, other languages are packs shipped by plugins, and a tag with no pack is a build error. The adding a language pack process is the path. | 22 |
| Single extraction path | 2026-09-13 | The text of an office document comes from the PDF the converter produced, never from the document's own format. A reader still opens the original file, for its native properties, but the words that enter the search index, the recognition and the comparison of twin resources are read from the PDF, page by page, with one PDF library; a PDF given as a source goes through the same reading, and a transcript, which no converter touches, gives its cues. | 21 |
| Lock file for human decisions | 2026-09-13 | concordance.lock.yaml, in the configuration repository, records what a person decided and the tool must not reconsider: accepted and rejected links, merged and separated twin resources, rejected term candidates. The build reads the file the lock key names, validates it against its schema and stops on a missing or invalid one, as on a configuration error: a decision is never dropped in silence. The keyword discovery honours rejected_terms, compared on the normalised form of the language pack, and the reconciliation of twin resources the merged and separated pairs, whatever the score; the links are recorded for a later version, and the configuration validator says so. The build log counts the decisions applied. Nothing is ever written into a knowledge repository, and a decision lives next to the configuration, versioned, reviewed like any change. | 17 |
| Deterministic identifiers | 2026-09-12 | An entity's identifier is its source name and its relative path without extension or type suffix, slugified; a frontmatter identifier takes precedence. Never a random identifier. The identifier is the page address. | 13 |
| Standard markdown only | 2026-09-12 | Concordance reads CommonMark, GFM and optional YAML frontmatter, nothing else. No wikilink, no proprietary tag, no directive: every file stays readable without the tool, and structured information goes through frontmatter, headings and lists. | 9 |
5 ingestion of 5 — pagination by twenty. The Links column counts the related pages, which brings the most central ingestion of the journey to the top.