This revision is from 2026/10/25 01:44. You can Restore it.

• Root words only, remove inflections, plurals such as -ing -ly
• Primitives only, remove theasaurus. Two words with the same definition. Keep the simpler word.

The scope of the library likely always require some tricks, workarounds.

The search engine format

The master word list has all the words in the English language. The user enters a search request. By the terms in the search request, a selection of keywords are selected by the probability that those keywords are highly probable as being part of the answer. The highest probability words are selected and the portion of library is generated. After the generation the results are also ranked by probability using a range of filters.

1. You type a request, for example “a lonely lighthouse keeper writing to his daughter”, and a length, say 5 words.
2. The system picks a short list of words it thinks could appear in a good answer. This is the keyword selection.
3. It builds sentences only from that short list. Instead of 82,639 words per position, it uses a few hundred, which makes the library small enough to generate.
4. It ranks the results so the most sensible-looking sentences come first.

Rule of thumb

2–5 words: per-slot lists, generate everything, then rank.

6+ words: per-slot lists plus beam search, so you never generate the full product.

Short sentences (2–5 words): generate everything from the small per-slot lists, then rank.

Long sentences (6+ words): beam search, building one word at a time and keeping only the best partial sentences.

The cost of speed: you may miss unusual answers whose beginnings look weak.