Convert fts2 to store data in a way which allows for much faster

updates.  Groups of documents form segments which are encoded in a
btree layered over a table of blocks, with various tricks to make
merges fast.  This performs 20x-25x faster than fts1 when loading the
Enron corpus, and is only slightly slower for queries. (CVS 3474)

FossilOrigin-Name: 85272b2f5394e37916afb1d509e7296810d976f5
This commit is contained in:
shess
2006-10-12 23:15:24 +00:00
parent 108bc320f9
commit 8a235d4d3b
3 changed files with 1802 additions and 408 deletions
+1794 -400
View File
File diff suppressed because it is too large Load Diff
+7 -7
View File
@@ -1,5 +1,5 @@
C Bug\sfix:\snamed\slocal\svariable\slockStyle\sas\slockingStyle\sin\sSQLITE_ENABLE_LOCKING_STYLE\sblock\sin\sallocateUnixFile\s(CVS\s3473)
D 2006-10-11T17:19:46
C Convert\sfts2\sto\sstore\sdata\sin\sa\sway\swhich\sallows\sfor\smuch\sfaster\nupdates.\s\sGroups\sof\sdocuments\sform\ssegments\swhich\sare\sencoded\sin\sa\nbtree\slayered\sover\sa\stable\sof\sblocks,\swith\svarious\stricks\sto\smake\nmerges\sfast.\s\sThis\sperforms\s20x-25x\sfaster\sthan\sfts1\swhen\sloading\sthe\nEnron\scorpus,\sand\sis\sonly\sslightly\sslower\sfor\squeries.\s(CVS\s3474)
D 2006-10-12T23:15:25
F Makefile.in 4379c909d46b38b8c5db3533084601621d4f14b2
F Makefile.linux-gcc 2d8574d1ba75f129aba2019f0b959db380a90935
F README 9c4e2d6706bdcc3efdd773ce752a8cdab4f90028
@@ -33,7 +33,7 @@ F ext/fts1/fulltext.h 08525a47852d1d62a0be81d3fc3fe2d23b094efd
F ext/fts1/simple_tokenizer.c 1844d72f7194c3fd3d7e4173053911bf0661b70d
F ext/fts1/tokenizer.h 0c53421b832366d20d720d21ea3e1f6e66a36ef9
F ext/fts2/README.txt 8c18f41574404623b76917b9da66fcb0ab38328d
F ext/fts2/fts2.c ba2b9e96fe561268d91ab9be3d8670df053ec347
F ext/fts2/fts2.c ddfca6aecbc5cbf13633f92aac29c2a8f24d6dec
F ext/fts2/fts2.h bbdab26d34f91974d5b9ade8b7836c140a7c4ce1
F ext/fts2/fts2_hash.c b3f22116d4ef0bc8f2da6e3fdc435c86d0951a9b
F ext/fts2/fts2_hash.h e283308156018329f042816eb09334df714e105e
@@ -410,7 +410,7 @@ F www/tclsqlite.tcl bb0d1357328a42b1993d78573e587c6dcbc964b9
F www/vdbe.tcl 87a31ace769f20d3627a64fa1fade7fed47b90d0
F www/version3.tcl 890248cf7b70e60c383b0e84d77d5132b3ead42b
F www/whentouse.tcl 97e2b5cd296f7d8057e11f44427dea8a4c2db513
P 5e8bbb85c1493e3ab2d807d24c68294f26838e49
R 876bb419d6818cdb37f5d2ebcd5b65dd
U aswift
Z 8ce919c2f327ed8920d3560f6006f3b4
P aa0b96c3dfa1d8008cd944281c227604b2ee16d1
R a2758723b7dd28d27444f107e79767e3
U shess
Z 668562e33633a331fce9e57ea3a5246b
+1 -1
View File
@@ -1 +1 @@
aa0b96c3dfa1d8008cd944281c227604b2ee16d1
85272b2f5394e37916afb1d509e7296810d976f5