80d3ea080a
Fix some fts5 problems with very large position lists.
...
FossilOrigin-Name: 2ea8f9cbe67dac60c1a0a661c95a03ecfa9a0b9a
2015-04-20 18:48:57 +00:00
687c5124e0
Logically store updates as (insert+delete) within the FTS tree. This allows keys to be annihilated more quickly under some circumstances.
...
FossilOrigin-Name: 50fae1f0006c0e946b5214e73eedf2687a0016f9
2015-04-15 18:49:20 +00:00
dbca0ab9b0
Fix a problem preventing doclist indexes from being loaded.
...
FossilOrigin-Name: b29109a083e5cd442cdd19f29d7be45b09c4f661
2015-04-15 16:01:42 +00:00
f1e58b7e8d
Begin changing fts5 to use a delete flag so that delete markers may be annihilated more quickly.
...
FossilOrigin-Name: 9341c070bb6140dbf559680952909674aa83fa55
2015-04-14 20:15:41 +00:00
929b2cfc2e
Have fts5 integrity check verify that prefix indexes contain the same values as returned by prefix queries on the main terms index.
...
FossilOrigin-Name: bdb8e82ab683f2438cde9f0b63e497dbf0141dcf
2015-04-11 18:25:04 +00:00
56e2497aaa
Improve fts5 integrity-check so that it checks that DESC queries return the same as ASC. Change the poslist format slightly to make room for a delete-flag.
...
FossilOrigin-Name: 49c1e74522a26e5dbe6f8305bc96487279b80dfb
2015-04-11 16:23:31 +00:00
71138b0f0f
Merge trunk changes with this branch.
...
FossilOrigin-Name: 142743918fd5a6d79fa10c44398360c3684a255a
2015-03-21 15:45:24 +00:00
8885708c3c
When scanning the full-text index as part of the fts5 integrity-check, also run a point query for every term and verify that these results are consistent with those found by the linear scan.
...
FossilOrigin-Name: ce972f6aab90f6929d018696f1ab3c2649eca802
2015-03-21 15:37:19 +00:00
998aaa03ea
Avoid an integer overflow in fts3 causing gcc 4.7.1 with -O2 to behave counter-intuitively (perhaps because the behaviour is undefined). Add an "ifcapable trace" to a test in shell4.test.
...
FossilOrigin-Name: e3e234649616f20610abce9ae9da1c572d3a4377
2015-03-21 12:22:51 +00:00
44723ce096
Improvements to the MSVC build. Fix harmless compiler warnings. Enable use of 'stdcall'.
...
FossilOrigin-Name: 737630b87314283b2c38790ace9d25ec05f81f4a
2015-03-21 02:22:37 +00:00
b7134d0863
Fix an FTS3/4 problem with handling empty tokenizer declarations (e.g. "CREATE VIRTUAL TABLE t(tokenize=);").
...
FossilOrigin-Name: 26d2def8a53094356008861636d66f9ae8f2448a
2015-03-19 16:25:42 +00:00
e0417626ea
Silently ignore any attempt to add a prefix index for prefixes zero bytes in size to an fts3/4 table. Or any prefix index size so large that it overflows a 32-bit signed integer.
...
FossilOrigin-Name: ad4b19d2ac0889a23fe3b0fd844286efc10cdd82
2015-03-19 15:52:07 +00:00
f5fab92d82
Add an optimization to the fts5 unicode tokenizer code.
...
FossilOrigin-Name: f5db489250029678fce845dfb2b1109fde46bea5
2015-03-11 14:51:39 +00:00
c9ca6fb85d
Avoid redundant string comparisons while merging fts5 segment b-trees.
...
FossilOrigin-Name: 5c46820d9b4aae791a8704b69145bd81f1e6780d
2015-03-10 19:24:30 +00:00
92faed663a
Fix some compiler warnings caused by signed/unsigned pointer conversions.
...
FossilOrigin-Name: cccee7b5b1e84523f1c549d3052fd170e32bde80
2015-03-07 15:46:41 +00:00
6f876c339d
Fix the bm25() function so that it multiplies scores by -1 before returning them. This means better matches have a lower numerical score, so "ORDER BY rank" (not "ORDER BY rank DESC") does what you want.
...
FossilOrigin-Name: 3ee7b5a9f987c269251620ae7cc0fc7876b58ee5
2015-03-07 11:50:31 +00:00
0b7ff5c11b
Fix harmless compiler warning.
...
FossilOrigin-Name: 580dae4615353d73c7d096107571bd60fec5b0f2
2015-03-04 20:18:55 +00:00
47c467c80e
Fix a couple of build problems.
...
FossilOrigin-Name: a5d5468c0509d129e198bf9432190ee07cedb7af
2015-03-04 08:29:24 +00:00
f400282bdf
Further minor optimizations to flushing fts5 data to disk.
...
FossilOrigin-Name: a07dcca9ef3821a6719ef9dbbc8ed861fa005035
2015-02-27 09:41:10 +00:00
5db7e2ca83
Fix suffix and prefix compression of terms in top-level fts5 segments. And a crash that could follow an OOM condition.
...
FossilOrigin-Name: bb104b3646c6f07ed002be7360b08433ee7980d4
2015-02-27 07:23:26 +00:00
38e40ee2e5
Fix potential memory leaks in the misc 'compress' extension.
...
FossilOrigin-Name: 3bc34fd427d9d7819cd9740237b1f5d4180341fa
2015-02-26 21:04:44 +00:00
dfdc4b4613
Optimize copying data from fts5 in-memory hash tables to top level segments.
...
FossilOrigin-Name: 8e3ca6323a2beab5f04250e24ae15b159d2aa0ac
2015-02-26 20:49:09 +00:00
4a7e11c629
Fix an fts5 bug in large incremental merges.
...
FossilOrigin-Name: 208e3cb6b6dc8c7d824b64dec2034004c9fcbba5
2015-02-26 14:54:03 +00:00
6d8f42ec81
Instead of the 4-byte fields, use regular varints for the poslist-size field in fts5_hash.c.
...
FossilOrigin-Name: 7eb022d7e5fdb180af823c82c47c938e4a7a355f
2015-02-25 19:24:37 +00:00
57fec54b53
Fix some problems with building fts5 and fts3 together using the amalgamation.
...
FossilOrigin-Name: fb10bbb9f9c4481e6043d323a3018a4ec68eb0ff
2015-02-02 11:32:20 +00:00
32348f7bd2
Merge latest trunk changes with this branch.
...
FossilOrigin-Name: 76212f2c9a3c3ff0e238d6dad776938c6af674e6
2015-02-02 09:40:36 +00:00
5082140311
Minor optimizations to fts5 writes.
...
FossilOrigin-Name: 1fffe51fa92f1784365140d5b163ab6c690981ae
2015-01-31 15:23:44 +00:00
719b0f66e5
Fix some problems with transactions that both read and write an fts5 table.
...
FossilOrigin-Name: 0e225b15357765f132c3364b222f9931a608a5b2
2015-01-29 20:59:34 +00:00
88392bf3ce
Optimize range constraints on the rowid column of fts3/4 tables even if there is no MATCH clause in the query.
...
FossilOrigin-Name: 85dc12625d300fe48f3c096f54ebcb8b6ef4e30a
2015-01-29 11:52:22 +00:00
e4a0d79b8a
Fix harmless compiler warnings.
...
FossilOrigin-Name: e7d2ec048c88237c124fbe598f8f7e950d43d90f
2015-01-27 21:24:33 +00:00
900f1922ac
Fix a problem with fts5 doclist-indexes that occured if the first rowid of the first non-term page of a doclist is zero.
...
FossilOrigin-Name: f704bc059e06b01f1d68fa7dad89e33eace6c389
2015-01-27 20:41:00 +00:00
6f0138e89e
Fix a bug in the fts3 snippet() function causing it to omit leading separator characters from snippets that begin with the first token in a column.
...
FossilOrigin-Name: adc9283dd9bc3a6463f8c4fe23dd58a3712c349d
2015-01-27 19:01:26 +00:00
18f6ff9eb7
Improve the performance of fts3/4 queries that use the OR operator and at least one auxiliary fts function.
...
FossilOrigin-Name: 245e8730451fbdc1c729beff7295c452df604009
2015-01-27 18:43:02 +00:00
8ac3025419
Have fts5 store rowids in ascending order. Query speed is virtually the same regardless of rowid order, and ascending order makes some insert optimizations easier.
...
FossilOrigin-Name: 5206ca6005bfa9dfc7346d4b89430c9748d32c10
2015-01-24 19:57:03 +00:00
641cb4360a
Fix compression of keys stored on internal segment b-tree nodes by fts5.
...
FossilOrigin-Name: 51444f67c0cc58a3023eb1cd78e7cf889da6c80f
2015-01-23 17:43:21 +00:00
626d9e3062
Remove some redundant code from fts5.
...
FossilOrigin-Name: 939b7a5de25e064bdf08e03864c35ab718da6f6f
2015-01-23 06:50:33 +00:00
0b520cc5f9
Add further tests and fixes for fts5.
...
FossilOrigin-Name: 5b295897153e9b26cd0d2e7ea112a4d461d0a665
2015-01-22 19:13:08 +00:00
4c2871bead
Further tests and fixes for fts5.
...
FossilOrigin-Name: c020a291ed293a66d21c5885e50a7fee04aa6366
2015-01-21 20:30:14 +00:00
dd8a501019
Fix an fts5 issue with loading doclist-indexes for a term that is the last thing on its leaf page.
...
FossilOrigin-Name: e0d614425f7f5ffe266fdc03642931b1ec19ad25
2015-01-21 18:23:25 +00:00
b461380333
Add further tests for fts5 backend.
...
FossilOrigin-Name: 09dabb3b9e140eec6cfda83bcb86b6b9f5cf54b6
2015-01-21 16:10:59 +00:00
997d798d92
Merge latest trunk changes with this branch.
...
FossilOrigin-Name: b3348b1e07e168b156636a29fc8c6d6afb3129c2
2015-01-21 06:36:07 +00:00
fd0b436e10
Add extra fault injection tests to fts5.
...
FossilOrigin-Name: f45a0dc0a8911c8aac5a1028ac4f543a709656e7
2015-01-20 20:34:17 +00:00
aa4d380a42
Handle the case where a tokenizer determines that there are zero tokens in an fts5 query term.
...
FossilOrigin-Name: 75f3d17f864072dfa2caee182b86cc4b9972d691
2015-01-19 11:15:36 +00:00
65545b5993
Eliminate all use of sprintf(), strcpy() and strcat() from test logic because
...
OpenBSD hates those functions.
FossilOrigin-Name: 10321910990195878c0af1e94b34ae0cdc0cb31b
2015-01-19 00:35:53 +00:00
d8736bc3b8
Ensure an up to date copy of the fts5 configuration has been loaded into memory before attempting to modify the same configuration.
...
FossilOrigin-Name: f30afd209aa4ce42766b1493750c4f5b5f1e9502
2015-01-17 20:01:52 +00:00
2656167f6e
Improve the performance of the fts5 porter tokenizer implementation.
...
FossilOrigin-Name: 96ea600440de05ee663e71c3f0d0de2c64108bf9
2015-01-17 17:48:10 +00:00
068a251d99
Simplify some code in rtree, to avoid confusing the optimizer in GCC on
...
some macs:
gcc version 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00).
Prior to these changes, compiling with -O3 would cause incorrect code to
be generated. The change to the nodeGetCell() routine is key. The other
changes are merely cosmetic details discovered while bug hunting.
FossilOrigin-Name: 882181ff9dd75f32db266db6e476671021fc567b
2015-01-13 21:26:17 +00:00
851ca6e715
Fix prefix indexes so that they work in characters, not bytes.
...
FossilOrigin-Name: af8d43a4a08528bbae25ee38fe25de8a86f8a21c
2015-01-13 17:25:08 +00:00
7bb22ac7ff
Silence some harmless uninitialized local variable warnings.
...
FossilOrigin-Name: 44375a34e59d1b0f0d765a99dcc7e6f30a4be7f7
2015-01-12 19:59:12 +00:00
7bb6e8e169
Get things compiling cleanly with MSVC and W4.
...
FossilOrigin-Name: c8725fa5fa361959b0f0a3fb36f204905d6f0ae9
2015-01-12 18:52:41 +00:00
dan