Working session (note: Meeting) on how the forms (52 passages, no note) of one document (note: Document), a deck (note: Deck), its notes (note: Note), the transcript of the session (3 passages, no note) where it was shown, are recognised (37 passages, no note) as one page. Three participants: the maintainer (3 passages, no note), an author of the specifications (3 passages, no note) and the quality owner (note: Quality owner). The transcript (note: Transcript) sits next to these minutes.
What was said
- The commit (29 passages, no note) and the folder never found a pair (47 passages, no note) on their own: an initial import (25 passages, no note) puts every file in one commit. A pair is founded by a declaration (17 passages, no note) in the frontmatter (note: Frontmatter), a shared base (19 passages, no note) name, a title equal to a heading, or similar extracted text (note: Extracted text); the commit and the folder proximity only reinforce it.
- Similar text is the expensive signal (9 passages, no note). Each text goes to its comparison form (note: Comparison form), then to shingles (5 passages, no note) of five words, then to a MinHash signature (3 passages, no note) under a fixed seed (4 passages, no note); LSH banding (3 passages, no note) enumerates the candidate pairs (3 passages, no note), so that the full matrix of a corpus (note: Corpus) is never built.
- A deck and its notes rarely share sentences (25 passages, no note): when the word counts of a pair differ by more than half, the pair is an inclusion (3 passages, no note) rather than a duplicate (10 passages, no note), its content signal is capped (3 passages, no note) and the finding (note: Finding) says so. The base name and the title do the rest.
- The page of a merged document names the signals of every scored pair, and the lock (note: Lock) wins over the score (note: Confidence) in both directions:
mergedpairs merge (27 passages, no note),separatedpairs neither merge nor report (47 passages, no note). - Two runs, or a shuffled input, give the same groups (23 passages, no note): resources (note: Resource) are sorted by identifier (note: Identifier) before anything else and every output is sorted.
Decision
MinHash for twin resources (note: MinHash for twin resources): the estimate (3 passages, no note) on 128 functions decides which pairs are close, the exact Jaccard index (3 passages, no note) is recomputed on the pairs estimated (3 passages, no note) at one half or more, and the finding gives the share of lines in common. The quality owner reviews the wording (4 passages, no note) of the duplicate candidate (note: Duplicate candidate) finding.