Word confidence

A value (35 passages, no note) between 0 and 1 (4 passages, no note) that says how much a candidate expression looks like a term of the corpus (note: Corpus) rather than a word of its own prose. Counts alone do not tell them apart, an adverb (3 passages, no note) and a business object (28 passages, no note) both reaching (3 passages, no note) eighty (3 passages, no note) occurrences; the shape of the distribution does. The confidence is the product of five signals read from the corpus alone, in the same pass as the score, without learning: the spread, the share of the files holding the expression, penalised beyond one half; the burst, its occurrences per file, penalised under one and a half from five files; the position, a bonus for the appearances in a heading, a written link or the frontmatter; the neighbourhood, a bonus when a defined term accompanies it in a frequent expression; the morphology, a soft penalty when a word ends like an inflected form of the language pack, suffixes.txt. A plain term of the subject scores 0.72, a word met once in every file 0.36, a word present in every file of the corpus 0. The publication threshold reads it as inference.keyword_pages.min_confidence, one half by default: under it the expression is suspected noise, gets no keyword page and no mark in the text, stays in the search index and is listed, with its reason, on the to-do page of the site, where a call invites to add it to the project's stopwords. The model carries the confidence (note: Confidence), its signals (9 passages, no note) and its penalties on every candidate, and the confidence among the attributes of every keyword page.