dan e89feee5c3 Add the "remove_diacritics=2" option to the unicode61 tokenizer in both FTS5 ...

and FTS3/4.

FossilOrigin-Name: 06177f3f114b5d804b84c27ac843740282e2176fdf0f7a999feda0e1b624adec

2018-12-03 16:14:49 +00:00

..

CaseFolding.txt

Add an experimental tokenizer to fts4 - "unicode". This tokenizer works in the same way except that it understands unicode "simple case folding" and recognizes all characters not classified as "Letters" or "Numbers" by unicode as token separators.

2012-05-25 17:50:19 +00:00

mkunicode.tcl

Add the "remove_diacritics=2" option to the unicode61 tokenizer in both FTS5

2018-12-03 16:14:49 +00:00

parseunicode.tcl

Add the "remove_diacritics=2" option to the unicode61 tokenizer in both FTS5

2018-12-03 16:14:49 +00:00

UnicodeData.txt

Add an experimental tokenizer to fts4 - "unicode". This tokenizer works in the same way except that it understands unicode "simple case folding" and recognizes all characters not classified as "Letters" or "Numbers" by unicode as token separators.

2012-05-25 17:50:19 +00:00