Compare commits

...

1228 Commits

Author SHA1 Message Date
drh ec1f59f0cd All legacy tests are passing.
FossilOrigin-Name: 2c436806b8d5f57de99c00f6154b038454fb9ae427d00d7b4a46ab9c7c69bcb9
2023-11-28 13:35:53 +00:00
drh 5026ddb83d The json_patch() code for JSONB compiles and works sometimes, but there are
still issues.  Incremental check-in.

FossilOrigin-Name: e0099464a0045a04f4ccf29bc2b8325fc8c7f39ccf4847e74818f928c9153588
2023-11-28 12:28:28 +00:00
drh f46f89df97 More aggressive use of jsonBlobEdit(). Improvements to the MergePatch
implementation sketch.

FossilOrigin-Name: fbca9570fd2e1465739e4d3a8d9bb40fad594fd78ab49b2cb34efa27ebdd8361
2023-11-28 00:27:58 +00:00
drh eb04a0bb7b Add untested (#ifdefed-out) code for the MergePatch algorithm against JSONB.
Add (and test) the jsonBlobEdit() routine that is needed by the new MergePatch.

FossilOrigin-Name: 4d353387fc10e1038cfdd86e66007bf728c231a928e588897bbee0fbfe76f225
2023-11-27 23:46:12 +00:00
drh 8a3034add8 Enhance the (SQLITE_DEBUG-only) json_parse() routine so that it shows a
decoding of JSONB when given a BLOB argument.

FossilOrigin-Name: af267868562e0799ad691dccad05f17afbc34d609eede8c55f57d209290246ef
2023-11-27 17:13:18 +00:00
drh 821a4c9f8c Give the json_valid() function an optional second argument that determines
what is meant by "valid".

FossilOrigin-Name: a4e19ad43dac81e7655ec03ff69bb99d1d02b0c227034c90fb41415fd4793fe3
2023-11-27 15:57:11 +00:00
drh 85eafe6f3b Convert the json_tree() and json_each() virtual tables over to use JSONB has
their internal representation.

FossilOrigin-Name: ec18caa3f7790b780dde66c1ccbb6eb09d2f1507629cc45955fc1b08380b4017
2023-11-27 12:36:29 +00:00
drh b4e5bc6cdb All tests passing.
FossilOrigin-Name: b5a5660ca22437640c9bf32c44d92c76a7293dafcbaf4fa6a4c171128d64871d
2023-11-27 12:30:55 +00:00
drh 15c0b03c5d Fix corner-case error conditions.
FossilOrigin-Name: ec23d34ab75e1d7e9366e59c633e0d30def8759f6d4717583ebeb4c90aeccf0d
2023-11-26 00:56:40 +00:00
drh 50b37832b2 Same results as the legacy JsonNode implementation on a small set of test cases.
FossilOrigin-Name: c3da4b079a1a15a4c0b1a6e71f876648b1d9eb32eddc67b9946c2475c7b6d085
2023-11-26 00:48:37 +00:00
drh e09a38c2e8 Remove unused elements from the json_tree() cursor.
FossilOrigin-Name: 914a50117d477b2cd30d58388fb8d1b71ff7ff6842ba025f38efc6e9647d06d0
2023-11-25 23:00:50 +00:00
drh 796abda538 Remove the vestigal JsonNode logic from json_tree() and json_each().
FossilOrigin-Name: 66c2ab9ebbf90477742e6be0d30e061d827c409de038f2a5b73479ed9448c4a6
2023-11-25 20:59:03 +00:00
drh aea2d23121 Almost working. Path is still not exactly right when Root is defined on
json_tree().

FossilOrigin-Name: 92258246916a9c0d72785964513113848a850dec78bdade8b3f274e410df4e7e
2023-11-25 19:28:44 +00:00
drh c2474105ca Generate the fullkey and path columns of json_tree().
FossilOrigin-Name: ffaa468ab8871906121df9ee5ef3dc00129a0086ed9c18831ecda69bf7f71455
2023-11-25 18:11:11 +00:00
drh b7d5cb711a Handle the path argument to json_tree() and json_each().
FossilOrigin-Name: fded888469565b2a4687185a926bd23fccfbf167c8bebe6c10696fc7f972f41e
2023-11-25 13:40:19 +00:00
drh 5e6500c81c Continuing work on json_tree() against a JSONB.
FossilOrigin-Name: 3df891cb11feee65e239ee2506eda34a9688341f05210d7c2e25a05338cb71ad
2023-11-24 21:57:38 +00:00
drh abbdbdfc1f Incremental progress toward getting json_each() and json_tree() to work
directly off of a JSONB blob.

FossilOrigin-Name: f8cab41b3bc65af6ff34b481db693d640ea025d09463d50b8e56d855e2abc913
2023-11-24 18:44:00 +00:00
drh ab70266698 Fix jsonParseReset() to properly clear the JsonParse.aBlob element.
FossilOrigin-Name: ab2644aacf4757a51cf62e05cff6711a0a3605d60502a3dd310887df1b993545
2023-11-24 14:25:56 +00:00
drh c0f8d58b21 Omit precompiled binaries from the source tree.
FossilOrigin-Name: 7dbc2f496d7a362460bb4c262ecafe5f30e35a8744861163d12c996365c2142f
2023-11-24 14:03:20 +00:00
drh d47719ee02 Omit the precompiled binary from the source tree.
FossilOrigin-Name: 4ff103d294b79cf7734e87e94e9d88c0e9f0b087cbb352e6da2f0a3a6b268f46
2023-11-24 14:01:56 +00:00
drh ac8b6e70c9 Merge the latest trunk enhancements and fixes into the jsonb branch.
FossilOrigin-Name: a838ebcbbd9f554fd84a1d3176fb572faaef7d0ec0675f1a5bf9430865dafae0
2023-11-24 13:41:20 +00:00
larrybr 77044f30a2 CLI .output/.once to not redirect ".timer on" results.
FossilOrigin-Name: ce766ed51f772a960d0b7a52c113b55c7ae90ef35050496d8e2a77547eab1a4d
2023-11-24 13:26:42 +00:00
dan 226aebae07 Fix an fts5 problem that could occur when mixing regular and secure delete operations on a single table.
FossilOrigin-Name: 8eb3f40021994f0d25ef9d246873796c84886e5951bb8ab991e1e2df31998484
2023-11-23 11:18:06 +00:00
larrybr 5d175eb020 Fix output redirect bug reported in [forum:/forumposts/cbf4933cfeee74bb|the forum].
FossilOrigin-Name: ce542fee6f0150bbd13505dcd26b21a9d1d23eb2dc8e78e63f9da6b4298ec028
2023-11-23 07:08:49 +00:00
stephan a328c42724 Minor doc fix in src/betreeInt.h, prompted by an email report. No code changes.
FossilOrigin-Name: 30d49aaed20454fe006e2f89db58f0c0b2433c99b2676a0b9583f56b7adb7fdb
2023-11-22 22:59:28 +00:00
larrybr 92752a31d9 For CLI build with unused function warnings on, #ifdef out a console I/O function, fPutbUtf8().
FossilOrigin-Name: da36f90d94b3b55de71aa7965421ea850ac677e9e63d60744633c4dc59448de4
2023-11-22 21:24:50 +00:00
stephan c0314bc01d Correct the URL for loading sqlite3-worker1-bundler-friendly.mjs from sqlite3-worker1-promiser-bundler-friendly.js, so that the promiser can be used in bundler-using environments. Problem reported via email.
FossilOrigin-Name: 753a75218913c3b9c7ec5438387107369c34d1775d68b04d19ae18197e558605
2023-11-22 17:11:39 +00:00
drh cf72606a7d Inserts invalid JSONB should return "malformed JSON", not a json path error.
FossilOrigin-Name: 306ee66fbd0231a9f5b229e5630d5cc66c9cf08b466d2d9204e79e1f44a16374
2023-11-21 22:36:32 +00:00
drh 27fea97e44 Direct editing of JSONB using json_insert() and json_set().
FossilOrigin-Name: fffb7a9538838e26991e6f16ea3138346a30c33ea6c3d3834680ee6d1f6eece2
2023-11-21 20:13:08 +00:00
drh e1df37b947 Correct blob-to-text rendering in some corner cases.
FossilOrigin-Name: 7822e0e59f9b611fe6289cc762b0aff61f9b87c3a82c60de110f447589a2c125
2023-11-21 19:05:22 +00:00
larrybr d9f3d6d753 Merge console I/O changes for Windows CLI.
FossilOrigin-Name: 935a8a8ee76d0014df42c1480e044fd1c2dfc26e78abb587d99d861d2ae5eb27
2023-11-21 18:26:06 +00:00
drh 664fe310b5 Fix the translation of JSON5 numeric values from BLOB into text.
FossilOrigin-Name: 40c4fb441f220982e4d61fd42597cf18546791a302fbcc8eec2eed29ee15ef35
2023-11-21 18:23:43 +00:00
drh e97f295783 Merge all recent trunk fixes and enhancements into the jsonb branch.
FossilOrigin-Name: 6d78d50ed2357e6c943c1ef97b1d2ea0902cbadef90c2c35dccdbdc2bdf8702f
2023-11-21 17:54:55 +00:00
drh 8a4cecaea2 Make edits directly to the JSONB BLOB when the input to json_replace()
is a JSONB.

FossilOrigin-Name: d69c6acef54a81f46a97a05d443fe648635b4b70772069d6705ef829b718e985
2023-11-21 17:51:58 +00:00
larrybr e788255681 Sync w/trunk as pre-merge-to-trunk sanity check.
FossilOrigin-Name: 448d6a1182d29940d5d34be2ce67df5601b688cd902dbbe97e95073f982a49ce
2023-11-21 15:55:31 +00:00
drh a4050f1434 Fix the trace3-4.4 test to be more rebust against timing quirks.
FossilOrigin-Name: 8936daa08243729d8538bb7288bbefb43f3bd842a0d4b2e8019092f5701c2926
2023-11-21 12:02:04 +00:00
drh 4f77a27032 Back out an incorrect change to the sqlite3ExprCompareSkip() function that
was added way back on 2019-08-22 for [44578865fa7baf97|check-in 44578865fa7ba]
and which was only today discovered to be incorrect by
[forum:/forumpost/45ec3d9788|forum post 45ec3d9788].

FossilOrigin-Name: f5b3eb0fc8936ba274a7654ff6dfa7d4654bd8dbca7f3a5ec1134b0b5260d59d
2023-11-20 15:54:00 +00:00
drh 0fcaf16f20 Convert an assert in OP_VCheck into a branch that aborts the opcode, as this
can happen on some very obscure conditions, as discovered by dbsqlfuzz.
Test case in TH3.

FossilOrigin-Name: 7946c79567b0ccd3a00d12390e99896d7dc11407d0b52bc39338a16a493f56f6
2023-11-20 13:06:59 +00:00
dan 56bb07bad6 Fix an assert() in fts5 that could be true following an OOM or IO error in contentless-delete mode.
FossilOrigin-Name: 3fe89238c31ca163d021ef1fecde594328815aaba7053fef9e9c789562a62f45
2023-11-20 11:40:18 +00:00
larrybr 033127af75 Cherrypick shell1.test fix.
FossilOrigin-Name: 91c888eddbcb1b461f7d562c2f0eee57f1a2be7f9959b9d69e80fc7c1da20e9e
2023-11-20 00:21:24 +00:00
larrybr 4c9a02c9f0 Cherrypick shell1.test fix.
FossilOrigin-Name: 39e30c5f9cc6dcac003255734e3ce1ac5b05349ea1a25e1c108b5f6d1d97612b
2023-11-20 00:20:56 +00:00
mistachkin 73e3e0c21f Correct conditional compilation issue seen with MSVC in the Win32 mutex subsystem.
FossilOrigin-Name: 6f8f4bfe607f1405d313bb88a33490621002f63e8c02c980f4c083630ad3a6d2
2023-11-18 18:36:26 +00:00
dan 046a7f9493 When ENABLE_SETLK is defined, avoid ever blocking on the lock mutex in os_unix.c when requesting an exclusive lock.
FossilOrigin-Name: eb36d475e91bfdbf4a18b6fd9751fbcecf15d960dcd1c00d2d18b5bf1d7503fe
2023-11-18 17:20:04 +00:00
drh 52c41ed41b Adjust an assert() in fts5WritePoslistData() so that it only applies if there
have been no prior errors.
dbsqlfuzz 25dca9b2568f67dc78a0e32ff280133fe71994bd.

FossilOrigin-Name: 257cdbab90c6db8ccc9a8fd5df556b69c3a35a329d39cd4642c792d7359a54a5
2023-11-18 12:06:21 +00:00
drh b8688e131e Another assertion fault fix, similar to [a9443dbfbe25e588]. dbsqlfuzz
2d9af4e94aca188e0092900eec711401c5d51687.

FossilOrigin-Name: 3afaeac56dff58db596360bf6f8dca97cb31405f73be8e189e8c0e6a1e5b239d
2023-11-18 11:23:01 +00:00
dan 11c512d631 When SQLITE_ENABLE_SETLK_TIMEOUT is defined, use a separate mutex in os_unix.c for each shm locking slot.
FossilOrigin-Name: 64691df980cdf73ddc00aabe15baa6c018e1d76b0e87f55ab33a9348fd5680bd
2023-11-17 19:01:38 +00:00
drh f9ae48b9ed Add the new "run-fuzzcheck" target on the autoconf unix makefile. Requires
that FUZZDB be set to the name of the fuzz-data database file.

FossilOrigin-Name: 8fa3915dbfdd9a5b0fcf4d695590b3de7e7d19d2917924186ca889cd60bf0eb9
2023-11-17 17:55:03 +00:00
drh 003d419b33 Add an assert() to prove the sqlite3_prepare() always either returns
SQLITE_OK or else leaves *ppStmt set to NULL.  See
[forum:/forumpost/70bb8576c6c084c2|forum post 70bb8576c6c084c2].

FossilOrigin-Name: feadd4024228e578811447c4b2d2b60780ff3d3681f80ca903585aa7b289d758
2023-11-17 17:25:30 +00:00
dan 735e7ee62a When SQLITE_ENABLE_SETLK_TIMEOUT is defined, use a separate mutex in os_unix.c for each shm locking slot.
FossilOrigin-Name: 4098df9652d90f2d22d5591d915d672c5413471f7916223510ba6fd932bfdd36
2023-11-17 17:10:37 +00:00
drh e416251725 Merge the latest trunk enhancements and fixes into the jsonb branch.
FossilOrigin-Name: 162f0509ef27bcd3ec87629640281a71c773e7c3bbd2cd0df76faf481531e7f1
2023-11-17 17:03:45 +00:00
drh f2bb6ab3fb Fix harmless compiler warnings in debugging code.
FossilOrigin-Name: ce6a75622ea5bca517bc6613e738aa670c9e1dd863596220eded5c2379c616c7
2023-11-17 12:22:42 +00:00
drh c15133e82a Fix an incorrect assert() associated with ALTER TABLE where an unknown
aggregate function contains an ORDER BY clause.
dbsqlfuzz e0900262dadd5c78c2226ad6a435c7f0255be2cd.

FossilOrigin-Name: a9443dbfbe25e588b4adddde664ddf482f19f71c704fbf356d49cf3a6135e7fb
2023-11-17 11:58:26 +00:00
larrybr e1edecf5d3 Simplify and make more rational how console I/O package features are selected. (Motivated by Fiddle and other command-line utilities to soon use the package.) No substantive code-execution changes.
FossilOrigin-Name: 1cde05877b7e47cb2ab9de26edd5245eb5ff1d91e69b94f3b24944fd4f6ccd3e
2023-11-16 18:31:05 +00:00
drh 11e8242e2e Both json_remove() jsonb_remove() work on pure JSONB as long as the input
is JSONB.

FossilOrigin-Name: 68d551730be0a3ea9579646ed4836c73554c83ca7f2303b69a18843f1750f1a7
2023-11-15 20:32:06 +00:00
drh 5e3ae1ec76 Some simple test cases for JSONB direct remove.
FossilOrigin-Name: 8cb4d2cbfc1b5f3c612d85138e66da8735d6589c7538fdf494a761c9b8104f18
2023-11-15 19:21:09 +00:00
dan 91c8e65dd4 Changes so that if SQLITE_ENABLE_SETLK_TIMEOUT is defined as 2 instead of 1, all blocking locks are taken for a single millisecond and the default busy-handler invoked as normal.
FossilOrigin-Name: 79e24ec3dd40373bbb93792829b18d9ef40daf19d4606174e36c8e19e61a7529
2023-11-15 19:19:04 +00:00
drh 8c96d3c104 jsonb_remove() now works without having to use a JsonNode parse, assuming
that the input is JSONB.

FossilOrigin-Name: 5207679e929786e577a0553d0d84dda5125456dcde80c0f3156f14f4d8c804cb
2023-11-15 18:55:03 +00:00
drh af0c9ffb4a The jsonb_remove() routine now appears to be working.
FossilOrigin-Name: e76d48137ea823b7810dc8c3b70eb21adabdd6cfbac36050c85d1375e94be1de
2023-11-15 18:47:31 +00:00
drh 0399994759 Work toward getting jsonb_remove() to work directly on JSONB blobs.
FossilOrigin-Name: a79ff8e58fcaf718a6fb78e145117f1d6d40d133f31e9752bb9c6d484850a27b
2023-11-15 16:10:31 +00:00
drh 9eb7694d58 Improvements to the description of sqlite3_errmsg() and sqlite3_errstr().
FossilOrigin-Name: fe5cc4169c5348324e863d3e339de94ed01749a3b9c86aa9adc5e3244bd9d033
2023-11-15 14:09:37 +00:00
drh fb57c8a932 Merge all the latest enhancements and fixes from trunk into the jsonb branch.
FossilOrigin-Name: ba91408f4c044feda003ef93784ccefb619f99ab64379ced481ee8e9e890fd41
2023-11-15 13:23:40 +00:00
stephan 5cc4ab93f2 Fix an exception misuse in test-opfs-vfs.js.
FossilOrigin-Name: 9264955e6e47aa8fc3a6f8bed192a6c12f43de49f7fba2e0cc080e47abedde14
2023-11-15 11:31:49 +00:00
stephan 1b963102b7 JNI: add more wrapper1 Sqlite.Blob tests.
FossilOrigin-Name: d63f6e1ffda9a88a37acc7862cb7ee8b9cd669b0fdba64c1ec9dd221cf807714
2023-11-15 08:59:53 +00:00
stephan bd06d8672d JNI: clear out the sqlite3_context native pointer after calling UDF callbacks which do not have an argv (as was already done for those which have an argv). Add related tests and code commentary.
FossilOrigin-Name: 138f40543b26b2e02e27d830d92e30b12cfef5a8dc3f0b58b39c68e1b3c91cc6
2023-11-15 08:29:42 +00:00
stephan 4d6b10cdbe JNI doc updates.
FossilOrigin-Name: 1b1f36a206319e99ccaed969893ff95dcf3b8e97ed301544cf3cd3fee2780335
2023-11-15 06:28:51 +00:00
stephan 6b36312e8a JNI: add the @Experimental annotation and mark all java.nio.ByteBuffer-related methods with it.
FossilOrigin-Name: 0f4b223102e5dc9142c9d2cb8892b8d3cc476e579420028b93d4e12f4cf94d3e
2023-11-15 06:10:37 +00:00
stephan 973011432b JNI test code cleanups.
FossilOrigin-Name: 09142ac14347e6f41bbe50bc835920e271713452733a478ede547816cc291ace
2023-11-15 05:08:39 +00:00
stephan d7ef0e8f87 JNI: add CONFIG_LOG and CONFIG_SQLLOG support to wrapper1. Code-adjacent cleanups.
FossilOrigin-Name: 83c49b9e71e5ae8852bab60a6fa630e22164c8efbf074c85450136781d0fffd3
2023-11-15 04:55:38 +00:00
stephan 253727b33d JNI: apply an internal level of API indirection to avoid having to work with massively-mangled names for the various sqlite3_config() overloads.
FossilOrigin-Name: 24f20663b1c4e13ff7d9a89e1df6af2e6f5730e5042ae6479e0a8c1bb846f313
2023-11-15 03:02:11 +00:00
drh 7df044633d Add NEVER() to a branch made unreachable by [6f9eed826f5b3d1c].
FossilOrigin-Name: 26dcaa34e3521783bde687749d590eca54cecb4bb95ef332a4a6473c4141d052
2023-11-14 16:53:52 +00:00
stephan 5d59669711 JNI: reimplement Tester2.execSql() using the high-level API.
FossilOrigin-Name: 3a69679e41d23a760df349b2471a1ed306c087f9a7a7a2e2cba3723071dee6b2
2023-11-14 14:59:41 +00:00
dan d928e78088 Fix another obscure problem with nested aggregates. dbsqlfuzz 04408efc51.
FossilOrigin-Name: 6f9eed826f5b3d1cb08402925b406a7fe9b54259af1ba5eba92d4d37fbad628a
2023-11-14 14:50:34 +00:00
stephan cb3a0b1b67 JNI: add Sqlite.Stmt.step(boolean) overload which returns the result code of sqlite3_step() unless passed true, in which case it throws for any result other than ROW or DONE. The intent is to simplify handling of LOCKED and BUSY errors.
FossilOrigin-Name: f4f1cc201a7dc618a66617778ecf58ba60461ae700cb41f5c4abfa26dd0c0ed4
2023-11-14 14:38:11 +00:00
drh 34ccd18d65 Extra PRAGMA vdbe_addoptrace output showing when OP_Column gets translated
into OP_Copy due to using a coroutine instead of a cursor.

FossilOrigin-Name: eba180e77977acd6f3d4a0e3c99a2ebd4ce2839e9ce30c588929e36d69aa762a
2023-11-14 13:51:19 +00:00
stephan 4ce5bc2836 JNI: use ByteBuffer.limit() instead of ByteBuffer.capacity() when figuring out where the logical end of a ByteBuffer is, for reasons explained at length in new code comments. This is unfortunately slower but is the correct way to do it.
FossilOrigin-Name: 51539419edc08ee6c70d8719d0f4d5ad47dd545a7fd9bf01d03a434aabd41d68
2023-11-14 05:33:44 +00:00
stephan bdfc51dfef JNI: add sqlite3_blob_read_nio_buffer() and iron out the blob/ByteBuffer interface somewhat.
FossilOrigin-Name: 7df317b448a09ae77e2c68cc901fdb6d56a2246c1313f06bebd1f3e53f02c19b
2023-11-14 04:59:57 +00:00
larrybr f695620e46 Add a few asserts into *Put*() functions to catch calls that might cause an address fault.
FossilOrigin-Name: 2341f9b6a48634a94e90e849f579afd07d398c3e0304efccf07d18b41a0e40aa
2023-11-14 03:24:43 +00:00
stephan adcd13d243 JNI: change sqlite3_prepare_multi()'s exception-handling semantics to be more C-like and, to support that, add the package-private sqlite3_jni_db_error() method to set the db error state from package-level Java code.
FossilOrigin-Name: 46656b354311ec0a36832af1c4ccb3b6a244aa55cfb3681e25c3f42b13b387dd
2023-11-14 02:43:30 +00:00
stephan b481413d95 JNI: add sqlite3_blob_write() overload which accepts a java.nio.ByteBuffer. Cleanups in adjacent code.
FossilOrigin-Name: ca32af8542aa2725cc87f54541b19897556f610e4674edf9f22a84e3d4097a82
2023-11-14 01:33:15 +00:00
stephan ce2edab088 JNI: add sqlite3_column_nio_buffer() and sqlite3_value_nio_buffer() using an only-slightly roundabout approach to creating properly-sized ByteBuffer objects.
FossilOrigin-Name: efbc82b218d26b7ca9b881da69d5fd14d22b5211fbd85a835da50e5bfde3d160
2023-11-13 23:11:10 +00:00
stephan 87c407da18 JNI: add sqlite3_result_nio_buffer() and tests. Discover that we cannot create sensible sqlite3_column_nio_buffer() or sqlite3_value_nio_buffer() counterparts because of ByteBuffer interface limitations.
FossilOrigin-Name: 44b4df01ff86841fb85b6295cbada422c6ba8a32a420a2e840e2d607b6c90164
2023-11-13 18:35:37 +00:00
larrybr 5ed4ba0f7d Cure many warnings from gcc, clang and CL.exe.
FossilOrigin-Name: 29ea2a3aadd34facc45a2e9f9a567bac16fca76beb019b879e2611433c85bbb3
2023-11-13 15:59:27 +00:00
stephan 7f7d7bea01 JNI: add sqlite3_bind_nio_buffer() and initial tests for binding ByteBuffer objects as blobs on JVMs which have JNI support for nio buffers.
FossilOrigin-Name: b10ce1ef82d84726fbf6a8f624d6530f84fefb505f7868b4a0ea910fed7a877f
2023-11-13 14:58:37 +00:00
larrybr 41d4b7bdee Fix var-intro after executable code departure from old C.
FossilOrigin-Name: 08996f4fd52f5742e77225ce3da2ac43ea5cc44bb0d8d21126b6d624273842a5
2023-11-13 13:01:01 +00:00
larrybr c270dad95d Slight cleanup of length-limited strpbrk() replacement, and eliminate strpbrk() call.
FossilOrigin-Name: 97e2c9621e3e01048e33da63ca4146ededa33cf6adefa996fa49b8c6a8d555b5
2023-11-13 12:53:47 +00:00
larrybr 14a08730de Enhance console_io to permit emits limited in various ways, such as valid UTF-8, upto control chars, or with counted limits, all getting away from 0-termination as the sole limit. In CLI, use this capability to avoid certain emit-chars-singly procedures that were breaking up UTF-8 characters. This fixes broken json mode output (on Windows) and (maybe) C-literal-like emits.
FossilOrigin-Name: 906c5c4082e30b7a0d07df89a42566461e2113507c5a0b339827ca0822b8fe84
2023-11-13 05:24:00 +00:00
larrybr fdbd9119d4 Add sqlite3_x86.exe unversioned binary.
FossilOrigin-Name: a731cdddbb99dbd3f9d1875cad5094239b78969c84fe4c56ecd63e33a5874e3f
2023-11-12 19:57:23 +00:00
larrybr 782c9f2e9e Use setOutputStream() to designate implicit output for oput{z,f}() emit functions, and use them extensively.
FossilOrigin-Name: 7850fb98a19d0ae3535367de3bca9e50408a2c21504c5772947fc39f287aa830
2023-11-12 03:58:15 +00:00
larrybr cc2b3c2051 Write BOM without fwrite(), using sputz() so that if it goes to the console, it is translated "properly".
FossilOrigin-Name: 06ef96a8233c6b6ca2653a0b780b3a401cab2b77594125f4f1c54b269c22a9f9
2023-11-12 00:43:36 +00:00
larrybr ea80462c10 Remove SHELL_LEGACY_CONSOLE_IO PP symbol and code it made active. (It is in the repo if ever needed/wanted, but it is just inferior or dead code now.)
FossilOrigin-Name: 27c5bf6563bc24ba7b47865e8d2f3e2d439666e19038d86dd8445fcdb9abe97a
2023-11-11 22:53:55 +00:00
larrybr 1bcb7c4902 Complete shell transition to using {f,o,e}put{f,z}() emit functions. This fails test 13.1 in json501.test, but so does trunk in the same way.
FossilOrigin-Name: 923c6b8b3a508c715b816c6bcd2ae9ac519bc37a62afc4ef813085c00f1e7cb6
2023-11-11 20:46:12 +00:00
stephan 56e1610f7a JNI wrapper1: when checking for an out-of-bounds statement column index, perform the is-statement-finalized check before the range check so that the former exception trumps the latter.
FossilOrigin-Name: 0832f9a8e9f574b157c791c5cddc73aff7b2ff403509f5d78f310494d4a7f93d
2023-11-11 14:50:01 +00:00
stephan 307f95c1e4 Do not cache a statement's column count in the JNI wrapper1 API because an ALTER TABLE via another statement may invalidate it, as reported in [forum:6d80efd58d4591c7|forum post 6d80efd58d4591c7].
FossilOrigin-Name: a6ab88e9a67f23ab7885402776282b94033cb48dbe34d4d18356e4dc22aae7cd
2023-11-11 14:43:50 +00:00
larrybr d2c8658024 Fix malf with redirected input due to bad assumption that stdin is a console in the fgets() replacement.
FossilOrigin-Name: 79d1f2c1019964dd154fbdd3f349822cb946a2600883994523ed145047f0a9ea
2023-11-11 13:09:09 +00:00
larrybr 1229f8e6bd Pervasive changes to console_io.{c,h} in support of simplifying ubiquitous emit ops in shell, and to get better control of console streams that might be opened only via .read or .output commands. Changes to shell to use {s,o,e}put{f,z}(...) calls for initial testing, but this check-in has few such conversions so that most will be in a separate check-in. Many renames to better follow recent coding convention. This code seems to be working, but has not been tested on difficult platforms or with multiple console hosts yet. So it is a WIP.
FossilOrigin-Name: 14762a004cdf37d1e12f26aadff8ed3824893278f22ff141de86dd44d9b250f3
2023-11-11 06:20:38 +00:00
dan fb28a93f75 Fix another problem with mixed join types and the RIGHT JOIN strength-reduction optimization. [forum:/forumpost/befdab472d | Forum post befdab472d].
FossilOrigin-Name: f1eae192315335d7e385b0a801a17700a9718d245bda6628518c5df9a1e9d3d6
2023-11-10 20:55:20 +00:00
drh 88dcfe56da Additional debugging information on the tree-dump of the BETWEEN operator.
FossilOrigin-Name: aca31e49d1d25043769544ccf2a07980c5f162a8eb2486e393bf9d9d1a394a60
2023-11-10 20:46:58 +00:00
drh 10c815a55d Remove an incorrect ALWAYS() that was inserted yesterday [1e039b6eb59c0001].
FossilOrigin-Name: 12885e298b9d3f977f1de11a194692dfb5fbb7daeabd958674f884a5575ddd24
2023-11-10 20:35:59 +00:00
drh e8d4fd59e4 Merge recent trunk enhancements and fixes into the jsonb branch.
FossilOrigin-Name: 091a5f058dfe2115fb9213655b34f00bcec80aebb299b571975cfe4ecd5ec206
2023-11-10 18:59:23 +00:00
drh 5a81e6e5ce Ensure 8-byte alignment of data structues in sqlite3_database_file_object().
This should have appeared on trunk originally and then be cherry-picked onto
the branch.  Oh well....

FossilOrigin-Name: ac39800bb2685fa287c7d834faed75f0bc61320ef986de314392d6eadb574d30
2023-11-10 17:49:26 +00:00
drh 53381132be Remove a NEVER() from whereAddIndexedExpr() that is reachable if there
is an unknown indexed function in the schema.

FossilOrigin-Name: a976b7208ff8603d7353ce9a0bdfba8e681cbb2ed3de6cfb5f0e8b07312ab86f
2023-11-10 16:29:02 +00:00
dan 0eed27d38b Fix an obscure problem with the join-strength-reduction optimization that could occur when mixing LEFT and RIGHT joins in the same query. [forum:/forumpost/7f74ce0bee|Forum post 7f74ce0bee].
FossilOrigin-Name: 530d10e93a5f63b71aaa94a2b89102d012a2cda815997066beb0f585fe823536
2023-11-10 15:03:18 +00:00
stephan 9d60c3c256 Expose the missing SQLITE_SUBTYPE to wasm.
FossilOrigin-Name: ac9534b2ceb8185b1fc03282f881cd3e4aea64af75a02ebded1e07d4d8278739
2023-11-10 15:00:26 +00:00
stephan 828a00c287 Expose the missing SQLITE_SUBTYPE to JNI.
FossilOrigin-Name: 0f92f4c90eb9397325f1a86836e356862108e2e850c1801e0bec4a7030dea271
2023-11-10 15:00:11 +00:00
stephan af65e7d21f Expose SQLITE_RESULT_SUBTYPE to JNI.
FossilOrigin-Name: 3bf75875b8be8d2e878681506fb35f2062d59f07fa23f571c58cd92f270bb197
2023-11-10 14:15:07 +00:00
stephan 6db0b11e07 Expose SQLITE_RESULT_SUBTYPE to wasm.
FossilOrigin-Name: 6d2fe9848beb35a8206e49c2ffae29a3eb2fe6411d77f366e962ced3f83e4749
2023-11-09 17:53:44 +00:00
drh 3839dcf973 Add the SQLITE_RESULT_SUBTYPE flag for application-defined functions. Add
the -DSQLITE_STRICT_SUBTYPE=1 compile-time option that raises an error if
any function invokes sqlite3_result_subtype() without the SQLITE_RESULT_SUBTYPE
flag.  SQLITE_RESULT_SUBTYPE prevents an indexed value of that function from
being used to replace an equivalent expression, since the indexed expression
does not carry the subtype.  Fix for the problem described at
[forum:/forumpost/68d284c86b082c3e|forum post 68d284c86b082c3e].

FossilOrigin-Name: ba789a7804ab96d81b15d6ef6fed1f802fa69db47cf91d368933e55289fa1d6e
2023-11-09 17:36:37 +00:00
mistachkin 2009a5acba Fix compilation issue seen with MSVC.
FossilOrigin-Name: 0dfe790d8118ff1bacc9e7c97d4f8ff9e5789f6cda8ec6fd981ea38b4da3905d
2023-11-09 17:28:31 +00:00
drh beb06e6b0a Put an ALWAYS on an true branch.
FossilOrigin-Name: 1e039b6eb59c0001a9efdd2f9928a34d4e9e01972ee76aa04a1279369dc03840
2023-11-09 17:26:39 +00:00
drh cb1f190ed2 Futher documentation refinements.
FossilOrigin-Name: 311c2eba93097bbecfa286bbeaff9bd6fc75a238e20cd2b6f834e594032d8c59
2023-11-09 16:52:44 +00:00
drh b10c3d32e0 Fixes: (1) In the ->> function, instead of setting a subtype and clearing it,
do not set it in the first place, as doing the set would trigger an error
under SQLITE_STRICT_SUBTYPE.  (2) Allow the SQLITE_STRICT_SUBTYPE through
the property filter on sqlite3_create_function().

FossilOrigin-Name: 6195468b14f6f17ea072cf191c9ef1bd0713acd314bc6dc128be7322bfd612cc
2023-11-09 15:01:56 +00:00
stephan 752722e8d1 A .class file build dependencies fix in the JNI build.
FossilOrigin-Name: 8cc32915165efd7c261f008bb6fa4cc6581ee7bd73ea5da47513742b9e6d34e4
2023-11-09 13:00:33 +00:00
drh 6eb381ff4a Add the SQLITE_STRICT_SUBTYPE compile-time option. This change reveals that
the current SQLITE_RESULT_SUBTYPE design does not work unless we tag the ->>
operator with SQLITE_RESULT_SUBTYPE.  But that will disable an important
optimization.

FossilOrigin-Name: e98a9a65dd309f72c240e280c7bebabc58af664fae9ee0d30c3fa1c78db5bae9
2023-11-09 12:58:03 +00:00
stephan b9050dcec7 Add some notes about the JNI pointer-passing approach and convert a couple of potential NullPointerExceptions into appropriate C result codes. Clarify that invocation of undefined behaviour from the Java API does not (due to the addition of defensive code) mean the same thing as it does in C (e.g. no NULL pointer dereferences).
FossilOrigin-Name: 19c4778f45261006368b2d9460350fed1e55fed314c8b3e1af34cd8c3c73b7d8
2023-11-09 12:48:54 +00:00
drh b18bb822dc Do not replace expressions that return subtypes with values taken from an
index.

FossilOrigin-Name: a35d13db09e32ee339f3983fe36b073714753ee3d39f577ae8d20596d7adc3eb
2023-11-09 12:17:57 +00:00
drh 194b8d514b Omit the new SQLITE_VALUE_SUBTYPE name. Stay with legacy SQLTIE_SUBTYPE.
Add extra documentation to sqlite3_value_subtype() and sqlite3_result_subtype()
indicating that the SQLITE_SUBTYPE and SQLITE_RESULT_SUBTYPE properties are
required on functions that use those interfaces.

FossilOrigin-Name: 563ad3be60d22c45f1c5b9a3e67738593f8b38f137147c56514166fbabf95365
2023-11-09 12:08:16 +00:00
stephan ec427813ac Two more JNI build fixes for Windows/MinGW, reported in [forum:4f949edc312d2a75|forum post 4f949edc312d2a75].
FossilOrigin-Name: a3f9c39086e582e16ca15647961956b3c28d038655d3b43d4b94bd306fbec1a4
2023-11-09 12:01:02 +00:00
drh c060b5f3a8 JSON5 bug fix: Escape double-quotes that occur inside of single-quoted strings.
[forum:/forumpost/ddcad3e884|Forum post ddcad3e884].

FossilOrigin-Name: 1c98d46d60ef1494bd8b7561c7d0cd5aafc178201a6f1f0da25dea6140b91cd0
2023-11-09 01:54:26 +00:00
drh 243f2ec6a1 Make a distinction between functions that consume subtypes and functions
that generate subtypes.

FossilOrigin-Name: 48a92e3ad855227188a4c5afe4abbb7171761cf6fc930660084d9abeecfd91d9
2023-11-08 21:38:30 +00:00
drh 2cbe14098b Do not cover expressions using an indexed expression if the indexed expression
is a function that might set a subtype.

FossilOrigin-Name: e908b26a990929996b3c16f0429e8313cd8fcefe7c883c77f66ea69f4059d6e2
2023-11-08 18:08:07 +00:00
drh a4cf38ca9b Merge recent trunk enhancements into the jsonb branch, and especially the
finer-grain characterization of JSON function properties.

FossilOrigin-Name: 72393b003f9f8675e4a124dddd09607b5b34ddefe56716b283c68c0982fb3d96
2023-11-08 17:11:13 +00:00
drh b494366370 More precise characterization of JSON functions. Indicate when functions might
return JSON (subtype 'J') and when they make use of the function argument cache.

FossilOrigin-Name: b2b62546c4a5e9dccb8aa0cb8eda228d662c69159e320b01a377317bc909e89f
2023-11-08 16:37:12 +00:00
dan 6234b33478 Have the shell tool emit a warning if the user attempts to use ".scanstats vm" in a non-SQLITE_ENABLE_BYTECODE_VTAB build.
FossilOrigin-Name: 3978c084a509c3c739fbe87e20feec9ddf1325e35170329987af197ca9fd731a
2023-11-08 15:56:41 +00:00
dan e445586630 Remove old files related to wapptest.tcl from test/ directory.
FossilOrigin-Name: dd3e7b5bcad122ac1e7e19ec547f4486ce90a6a2aa89a64e36bea13a216492fe
2023-11-08 15:51:42 +00:00
dan 0ad5301378 Avoid blocking as part of passive checkpoint operations, even if SQLITE_ENABLE_SETLK_TIMEOUT is defined.
FossilOrigin-Name: e5ecc404cae1ce8b639d0263fa07571c066f11bfc62f5ba331ad7ae138e78572
2023-11-08 15:49:57 +00:00
stephan 8657eddcdb Remove an unused/invalid test from the wasm suite.
FossilOrigin-Name: 916ae898743a969295a48ae2a6e9e366586834b32d77d3fa281bbaf7f2818502
2023-11-08 15:34:03 +00:00
stephan 86b1898cb4 JNI build fixes for platforms where the jint type is not the same as int, as reported in [forum:9089d2049a|forum post 9089d2049a].
FossilOrigin-Name: b32b0873274bfe472114da8a308a04bee76ba26a5830d8d04fc921f9c1544f9d
2023-11-08 12:56:23 +00:00
drh c78d3b5963 Suppress harmless UBSAN warnings about memory overflow in OP_AddImm. The
exact same machine code is generated by GCC.

FossilOrigin-Name: b0594383b9fa021a8713d640a4606b9053f8e21d64b4ec8ea60a0b6cddfca306
2023-11-08 00:45:14 +00:00
drh ec8f893e06 Changes a no-op call to freeP4() into an assert().
FossilOrigin-Name: 32a7b1bd4d88a6839c2c5061a38b4a28f31b22aa8fa08c743633ff96fc4bf88d
2023-11-08 00:12:38 +00:00
dan d7b0b5ec6f Add -DSQLITE_ENABLE_SETLK_TIMEOUT=1 to a release-test configuration.
FossilOrigin-Name: cd3e38fb0f2f7e134bb6cfbe86b6621017344e4361dc6a09ec1367860ba8773b
2023-11-07 20:56:29 +00:00
dan 275234e320 Fix an assert() that could fail within calls to sqlite3_snapshot_open() in SQLITE_ENABLE_SETLK_TIMEOUT builds.
FossilOrigin-Name: 84634bc268e5c80146f3f3b2e13118f239c9a7e4e4e9dfcaccef2b17252ce53b
2023-11-07 20:11:49 +00:00
stephan 344255e5aa An attempt to work around compilation errors on MinGW reported in [forum:9089d2049a|forum post 9089d2049a].
FossilOrigin-Name: d5658a3ee4f585cc1e96d84425aad2e1f56b3ae507c9e17b71e030f83f0b70e8
2023-11-07 19:39:23 +00:00
larrybr 73f03276d9 Refactor MBCS/UTF-8 translation to avoid extra allocations, supporting non-formatted (faster) output. Some code cleanup. Wrap .system/.shell command exection with restoration of startup console mode and renewing mode setup. Changes to make legacy MBCS build work better (than legacy did, even with --no-utf8.)
FossilOrigin-Name: d5e88fcde53ca7ba05bb164943a9f57bd92080bb7e5eebbbed64b9886ac97338
2023-11-07 19:30:14 +00:00
drh 17a3295387 Update the documentation to the sqlite3_set_auxdata() and sqlite3_get_auxdata()
routines to make it clear that they do not work as one might expect when they
are called during query planning, instead of during query execution.  The JSON
routines misuse those interfaces, so add a special flag to JSON routines that
prevents them from being invoked during query planning.  Fix for the problem
in [forum:/forumpost/a655ee159eca1ea5|forum post a655ee159eca1ea5].

FossilOrigin-Name: 796a23f9ee33da0803844a2f40c1733db894cc4ef7fbaa1fa94af6af2d3b873b
2023-11-07 19:03:13 +00:00
stephan 637922acac JNI: during static init record whether the current JVM supports JNI-level access to java.nio.ByteBuffer raw memory, and add sqlite3_jni_supports_nio() to query that.
FossilOrigin-Name: fb8dbb77a4d8efafd6772333824b4ab589828cf155a63ca6a26730314d0a4bd9
2023-11-07 17:15:55 +00:00
stephan 488125d4ce Add Sqlite.prepareMulti() to JNI wrapper1, for preparing multiple statements from a single input.
FossilOrigin-Name: e4670d68b52233ab376a1725983e148aaf2a2c3658a41f5768e37a0f1f87428a
2023-11-07 15:56:39 +00:00
stephan 6dc368e632 Diverse minor cleanups in the JNI pieces.
FossilOrigin-Name: 35233dd900632b997b5e532170a3b2af0ca7f1dccb8407555b93f2b395b0f7b4
2023-11-07 13:44:29 +00:00
stephan 6ca0cfd87a Flesh out [7a63b5b65a79] to be able to build JNI with or without SQLITE_ENABLE_COLUMN_METADATA.
FossilOrigin-Name: fcee41b3d4d2558299ead28cc17f290b9ff1957a84c3feaa0a24872feeb22901
2023-11-07 13:22:49 +00:00
larrybr 4c78cb50bf Get dependencies into make recipes. Get legacy console I/O (-DSHELL_LEGACY_CONSOLE_IO) working. Due to movement of MBCS/UTF-8 translation into traditional stream I/O simulacra, the input translation does not happen the same way. (It works the same, but fails differently and a bit better.) Added printf() and fputs() look-alikes, and made CLI use them.
FossilOrigin-Name: 1721dc6a434361c4e2b87c6e677b6dc223432b3cdd5b9eecabaa258889fb2d2a
2023-11-07 02:41:46 +00:00
stephan 23056b608e Add -DSQLITE_ENABLE_COLUMN_METADATA to the JNI build, as per [forum:9205518c0568fdf0|forum post 9205518c0568fdf0]. Add tests for the functions that flag enables so that the build will fail if that flag is missing.
FossilOrigin-Name: 7a63b5b65a79d15658a160d0878c7371941c67e9b48a7442762c68c60b77288a
2023-11-06 21:57:15 +00:00
dan a3e8822c4b Fix an fts5 problem where a transaction consisting of (a) a DELETE on rowid X, (b) a prefix query, and (c) an INSERT on rowid X, could corrupt the index.
FossilOrigin-Name: c2058a045b57571b2b5d342adb212fe606717c633a0422755691ae6bf5725d25
2023-11-06 18:40:25 +00:00
larrybr d0046388a0 Move console_io lib to its own subdirectory, etc/consio .
FossilOrigin-Name: 1d0583f2eb69fdca1cbc55763c0e86a7e32cb7771bfbc2cdf02da4e3fedbfa23
2023-11-06 15:31:08 +00:00
larrybr eb1898d6f3 Some renaming, warnings cured, and more coding convention conformance.
FossilOrigin-Name: 58815f0ad259599a674e7d0fe0d4676a1c4fb8e118c08b619580ff655fb3be40
2023-11-06 15:15:58 +00:00
larrybr 56fba47850 Add an fputs() equivalent to console I/O lib, and use in CLI.
FossilOrigin-Name: d661f90724b1bd31948169841bacb80ba4ae91b52191a0b0021a89a22d932a02
2023-11-06 13:24:07 +00:00
larrybr e3a6a60901 Cause CLI to use console_io library.
FossilOrigin-Name: bf66a7c1d330b04fd3c3e50f43ebe1e41307db59d7189798acafb3de77e0c8b2
2023-11-06 03:09:10 +00:00
larrybr a0cd392979 Make MSVC accept it, too. (Cannot use static const int as part of a constant expression?)
FossilOrigin-Name: bb278d2496b27d2e2ee3cedd6fc54394e71ab2ba5d3d51593f97e897b8b306a4
2023-11-06 00:15:14 +00:00
larrybr 8dd4697e48 Make it buildable. Pull test program from testee. Zap stray fprintf().
FossilOrigin-Name: 2b850aca1e76805a0358064318a765fa66ce394d015936fd47683d74ca4c187e
2023-11-05 23:55:41 +00:00
larrybr ff39634625 Input working. No line-editor provisions yet. (WIP, but suitable for testing.)
FossilOrigin-Name: e8568b1d925c2118eb08394dd8aa50cfb521240f87668f535ec4a03e67dc9a09
2023-11-05 19:42:00 +00:00
stephan ed99e7493a Add incremental blob I/O support to JNI wrapper1.
FossilOrigin-Name: 7f1c76fe930d69a0274f70fa7b7e68e0db6226b731a065fa57d0936c8400ffb0
2023-11-05 04:20:04 +00:00
stephan 546db3f14a JNI wrapper1 normalizeSql() now throws UnsupportedOperationException, instead of returning null, if built without SQLITE_ENABLE_NORMALIZE. Remove SQLITE_PREPARE_NORMALIZE from the JNI interface because it's a legacy no-op.
FossilOrigin-Name: d081a126697e214082f3b203f23ea63510080e5c2aac1d8badc9e6e4bfea7c95
2023-11-05 03:37:33 +00:00
stephan 88c90a793e Add authorizer support to JNI wrapper1.
FossilOrigin-Name: 773f9873865b5277a6a682c4695f216bfe1ec05ed5e5a2a70aaa451934ba2dc0
2023-11-05 01:55:20 +00:00
stephan 9a26589940 Add progress-handler support to JNI wrapper1. Correct the return type of the extended_result_codes() JNI binding and expose it to wrapper1.
FossilOrigin-Name: 6c0acfdce2160d8db261a59677cec571b6abc333481525b1ec975d98e88bec88
2023-11-05 01:39:29 +00:00
larrybr 557297ae77 Setup, takedown, mode set and output working. No input yet. (WIP)
FossilOrigin-Name: dfea85be1fb927ea446c9d98bae42ba1197bdab098aa6d95aa512a37d07a1e52
2023-11-05 01:21:14 +00:00
stephan 0ee7ae72a1 Add update-hook support to JNI wrapper1.
FossilOrigin-Name: 6c584cf27179d16deee84e9699493cf29bebef123fa2a7493aad0324bead1618
2023-11-05 01:14:07 +00:00
stephan 2c930e3b86 Add commit/rollback hook support to JNI wrapper1.
FossilOrigin-Name: ff3d44fe42528d96533d22c7807472df89bca18f1def23b018e2f407318143f8
2023-11-05 00:48:43 +00:00
stephan b02ca781ad Add busy-handler support to JNI wrapper1.
FossilOrigin-Name: dcf579ab2de4a3d3a437cde59b2fd60f1729c0bde31df1865117e6a5ea4bab20
2023-11-05 00:02:47 +00:00
stephan 15d38c0dde Bind collation and collation-needed to JNI wrapper1 and correct the callback return type for collation-needed callbacks in the lower-level JNI binding.
FossilOrigin-Name: 0f673140681685ab390ecd7326a8b80d060b7ab23c31a2cfc28ba76fd5096afe
2023-11-04 23:37:11 +00:00
stephan dc8a684c11 Wrap the sqlite3_backup API in the JNI wrapper1 API.
FossilOrigin-Name: 3ee6cc29d2111e7ad90860827c0ea808fdf07bc71defdade7e6794ec4a2a3ce2
2023-11-04 22:47:40 +00:00
stephan ffdb479e7c Reimplement auto-extensions in Java for use with the JNI wrapper1 API.
FossilOrigin-Name: 14ed4c64533622e5faf1aaa59c24885885aad43f1c0d4717773e79440e8e1468
2023-11-04 21:51:34 +00:00
drh d4677f192f Ensure that the YYYY-MM-DD input to date and time functions has been normalized
prior to returning a result.
[forum:/forumpost/6bb476897e|Forum post 6bb476897e].

FossilOrigin-Name: b692eb8ccb2d0645599ad73a8bdacf5df499114244aadeb38aabc580fc4dc7c5
2023-11-04 21:44:00 +00:00
stephan 1b199243cb Expose context_db_handle() to the JNI wrapper1 API and clean up some related tests.
FossilOrigin-Name: c23123af7d40dea24a0848dff987fd58a6703ce04165060533544db85983d566
2023-11-04 16:01:13 +00:00
stephan 348e192ea8 Rework the JNI wrapper1 variants of status() and db_status() to be more Java-esque.
FossilOrigin-Name: 40ad3920673561a06edf0b70a50a40be6cd20817fe22b87b63a9ac80cb2c9df8
2023-11-04 13:37:42 +00:00
stephan 5f3b13d136 Bind the bool-flag sqlite3_db_config() variants to the JNI wrapper1 API.
FossilOrigin-Name: b5cdcb9279d9276f24b67083839f463beecd731f46f2e8bf68fff716df0a3921
2023-11-04 13:16:49 +00:00
stephan aad6808efc Bind the trace API to the JNI wrapper1 API and add a way to map the native-level db/stmt types to their high-level counterparts (required for translating callbacks such as tracers).
FossilOrigin-Name: 702910e0d1cfc897a269b4fb36b255165958edf529ac9553ebc5155e404d4cd3
2023-11-04 12:53:00 +00:00
larrybr a501791d59 Define interface between project command-line apps and a console I/O "library".
FossilOrigin-Name: 64abef8314b8544fdc7b71317d61a4641dc607a1ae42b8ff21543226fd338ba2
2023-11-04 02:22:04 +00:00
drh 5189ef98d3 Back out the ALWAYS inserted late yesterday. The fuzzer discovered a
counter-example.

FossilOrigin-Name: 570635575cc5fbffe910ed992b58393e214117ef3b5370a66f115cd0ee202913
2023-11-03 18:45:26 +00:00
dan ab8a4b2bb0 Fix a case where adjacent tokens are handled incorrectly by the fts5 snippet() function.
FossilOrigin-Name: 8f5e9c192ff2820d8cfb076ab28f30697d10c22710583d6c7fd7019c4a0ea795
2023-11-03 17:20:20 +00:00
stephan b945183195 Wrap more of the stmt API behind the JNI wrapper1 API.
FossilOrigin-Name: 8fea23dc3af023ccf2909f1b4c6f91e7df0ffaac875b15f1fb3e264fba169b6a
2023-11-03 13:00:51 +00:00
drh 7b5123c796 Enhance the JSONB lookup routine with logic to apply edits. The new logic is
currently unused and hence untested but does not create any regressions.

FossilOrigin-Name: b12110276fc15d1b6b0cc455f89747ace7a09650d5ba433d8bb431bb4e5bc951
2023-11-03 12:09:22 +00:00
stephan 073f8f5458 Bind the sqlite3_keyword_...() and sqlite3_compileoption_...() family of functions to the JNI wrapper1 API.
FossilOrigin-Name: b27242414d6023eac7e62cf6120e1f02b0ddc7b8f0a1e4c48111cfe88d197cbd
2023-11-03 11:53:13 +00:00
drh 1d5f2af063 Update the tool/srctree-check.tcl script so that it verifies that autoconf
has been run following edits to VERSION.

FossilOrigin-Name: 52ab3deba4d26ca0e9a84a6dff254195b4a0e2cc6cf948cf7a66bb11117e7002
2023-11-03 11:41:24 +00:00
drh 24f7f5923d Merge all the latest trunk fixes and enhancements into the jsonb branch.
FossilOrigin-Name: b089bf46374b374d02d7654c65eb3e75d1777638b398061e47644af0dab48c9b
2023-11-03 11:35:33 +00:00
drh d706adba48 Update the version number to 3.45.0 to begin the next development cycle.
FossilOrigin-Name: 15b618e92a2708cc83256947736de8c494a9985a77e38bd5efc7e51e72cba344
2023-11-03 10:15:16 +00:00
drh dcd6d21fe9 Mark a branch made always-true by the penultimate check-in with ALWAYS().
FossilOrigin-Name: 268b5984a4263bee245a9bb47ac927bde56cdf4af8795b851dada5622224076f
2023-11-02 22:39:11 +00:00
drh 792103a0f2 In the previous check-in, use a u32 instead of an int, to make it easier to
prove that the integer will never overflow.

FossilOrigin-Name: 51002079e0b499e116415189a913a238e95ed9766a8af9d042928f4263861a87
2023-11-02 22:11:35 +00:00
dan 5e4233a9e4 Fix a spurious "misuse of aggregate function" error that could occur when an aggregate function was used within the FROM clause of a sub-select of the select that owns the aggregate. e.g. "SELECT (SELECT x FROM (SELECT sum(t1.a) AS x)) FROM t1". [forum:/forumpost/c9970a37ed | Forum post c9970a37ed].
FossilOrigin-Name: 4470f657d2069972d02a00983252dec1f814d90c0d8d0906e320e955111e8c11
2023-11-02 21:02:53 +00:00
dan 78fee3f738 Add the "remove_diacritics" option to the fts5 trigram tokenizer.
FossilOrigin-Name: 0d50172477064dce3f7c61641d88aa15d8a1edea9872adaffcc1a7a1c9ab0e93
2023-11-02 18:22:35 +00:00
dan 7b0fd0c564 Fix a problem with amalgamation builds on this branch.
FossilOrigin-Name: 8f046c82c9cf51fc349674577c68d3d2499ee37009deacbf937d711d9930fd49
2023-11-02 18:10:22 +00:00
stephan 06af40b2e1 Merge latest JNI pieces into trunk.
FossilOrigin-Name: e8c97faec339430fa2d1a7e915050d9b363bdba606a4419397cb099141da254d
2023-11-02 18:08:49 +00:00
dan e186fe20f5 Add the "remove_diacritics" option to the fts5 trigram tokenizer.
FossilOrigin-Name: 83da80135b6105f47d1de560232449562ae8ac176c8011a6f75589f62bc9b1db
2023-11-02 17:31:06 +00:00
drh 845a2e0cc2 One more tweak to tool/srctree-check.tcl so that a complete build can be
accomplished from a read-only source tree.

FossilOrigin-Name: cba9f0601ca995ac6952aa3a83f7264f6d25aaaa2ea36b19e90cbf591077de72
2023-11-02 13:10:16 +00:00
drh 8a93ce7458 Update the srctree-check.tcl script and child script so that they can be run
on a read-only source tree and so that if any inconsistencies are found, the
script returns a non-zero exit code and thus halts the build.

FossilOrigin-Name: 58eb5440d75fb13c1a089cb935de6fb94b4322e10e349db8f11f37b9a2fda2fc
2023-11-02 12:05:55 +00:00
larrybr 57a7fdd814 CLI to compile with older MSVC compiler.
FossilOrigin-Name: c89f75f8eef449f6869bda423593f7e14080e805f8ed192aff9463460f11b1a9
2023-11-02 11:04:37 +00:00
stephan 7d52262889 Remove some outdated JNI docs.
FossilOrigin-Name: 52aef3d015bad12c4bd9334ec795276a68a0dabdc6fd0ff2a1068f640171eb77
2023-11-01 20:09:02 +00:00
drh 0577cdec5f Update the windows build documentation to provide the exact command used
to build the DLLs that appear on the download page.

FossilOrigin-Name: 3524a8914c62f7987d948e5ef50cfdcb07254489e80e2257f38d0050c4baf450
2023-11-01 19:35:15 +00:00
dan 54e7a0f360 Fix testrunner.tcl so that it accepts a relative path as an argument to the --zipvfs switch.
FossilOrigin-Name: c1ed7ed02269e5b26142eb307e7889fc636672f101ea7fc192a2445e16a7dd6f
2023-11-01 14:33:44 +00:00
drh 2158864585 Bring the jsonb branch up-to-date with version 3.44.0 changes.
FossilOrigin-Name: a420a4f7ff76b0e9cf5f6d515ccfa31e526d58f4001a4015a367e2aa3c82091f
2023-11-01 14:12:07 +00:00
drh 66476771b7 Version 3.44.0
FossilOrigin-Name: 17129ba1ff7f0daf37100ee82d507aef7827cf38de1866e2633096ae6ad81301
2023-11-01 11:23:50 +00:00
drh 5ba99da869 Fix harmless compiler warnings in test code.
FossilOrigin-Name: d1895dd8f5757a339f619f22b29c8a739398ded673bb9c93f1b8eb8a4b38f510
2023-10-31 17:40:30 +00:00
dan ddd1c97500 Fix a problem with SQLITE_ENABLE_FTS5 builds of the non-amalgamation testfixture.exe in Makefile.msc.
FossilOrigin-Name: 544091cc0f5b7395a89226629267c858ec53f18095aadb3e9307ef8ea718176e
2023-10-31 17:39:10 +00:00
larrybr 739edb99ee Remedy CLI non-UTF8 handling detection flaw [forum:/forumpost/fa0e00d36db5fe0a|noted in a forum post].
FossilOrigin-Name: a4c7954b9380dfd3599e521f7fd40b8b556a7d0268198b302146311d20c3b162
2023-10-30 23:20:45 +00:00
larrybr 7545b0eab7 Remedy CLI non-UTF8 handling detection flaw [forum:/forumpost/fa0e00d36db5fe0a|noted in a forum post].
FossilOrigin-Name: 2666b80d2cf64a14d0482ee1ce0e8b2ff9e0c5bb21cd626470241a73a6555a3a
2023-10-30 23:04:27 +00:00
drh 23f5f62d41 Update the autoconf/Makefile.msc so that it aligns with Makefile.msc.
FossilOrigin-Name: c4f724de279dae1afd10d435ad61ed07c1e4658288457c50e4f9b2f4521bbe12
2023-10-30 20:35:33 +00:00
larrybr cc2af7e68a Dodge a nuisance warning from MSVC in CLI use of Prepared Statement Scan Status. (Yes, C rules are fine.)
FossilOrigin-Name: 166c96d91dcc8c500a084d5703db103176ed5ec7d5f1334050cd66b3f1cd0e4f
2023-10-30 19:50:20 +00:00
drh f3f0bd2155 Add a cast to i64 for an integer in an sqlite3OsWrite() offset
calculation.  The cast is not strictly necessary, but it helps human readers
see that the code is correct.

FossilOrigin-Name: 7564ff1ba2c2fba89106d1aa06cc5379e752f119f22370f2f155f24cc698dec6
2023-10-30 19:03:17 +00:00
drh 383295eb17 Fix another comment typo. No code changes.
FossilOrigin-Name: 3203256c0285ef2fea810d59978603bfd7e38151cbc42e1baddf84b57c56f638
2023-10-30 18:48:03 +00:00
dan 2d0dd57dd5 Add FTS5 to default Makefile.msc builds. And UNKNOWN_SQL_FUNCTION and STMT_SCANSTATUS to default Makefile.msc shell builds.
FossilOrigin-Name: e41fde2fdb999b430561ac6d6ba150666ad1858e99c973f8dcd04a6934991032
2023-10-30 18:17:29 +00:00
drh aa36dcf074 Make the new sqlite3_get_clientdata() and sqlite3_set_clientdata() interfaces
available to run-time loaded extensions.

FossilOrigin-Name: 8fc9aa0f896a2676b73c41cefbff74c822a7106b56324558862dd0a2a1877eaf
2023-10-30 18:09:14 +00:00
dan a238c57eb8 Fix problems with previous commit on Makefile.msc. Also some win32-specific test failures in fts5misc.test.
FossilOrigin-Name: 527f2237a6fb5e2ac97afddd7c65c0afa971efc616cb3b738e7245ee0b8d9570
2023-10-30 18:07:18 +00:00
drh 30c9aca83e Fix typo in a comment. No code changes.
FossilOrigin-Name: 424ef9550abaab32f745dc955e8996d51836eaed737e24c4b4db811aa118ce5d
2023-10-30 17:56:27 +00:00
dan 23e8f976dd Add FTS5 to default Makefile.msc builds. And UNKNOWN_SQL_FUNCTION and STMT_SCANSTATUS to default Makefile.msc shell builds.
FossilOrigin-Name: bf99ebba23b6a72a0c02c5542574f409828343e89501cbe2112596ced063c6ab
2023-10-30 17:13:12 +00:00
larrybr 88d3c148c8 Predicate Windows CLI UTF-8 console I/O on a runtime capability check rather than an OS version check.
FossilOrigin-Name: 0058ea6a69103c1442bf5090553ce9bf77e06011a4960df754eb96144ced497d
2023-10-30 16:38:05 +00:00
larrybr 8a95e92af9 Predicate Windows CLI UTF-8 console I/O on a runtime capability check rather than an OS version check.
FossilOrigin-Name: f89d062f8890fffc957a354e966784031d561d0f8f5c174c1ccdcf77e66c32bd
2023-10-30 13:56:50 +00:00
drh d2147bd32b With SQLITE_ENABLE_BLOCK_ATOMIC_WRITE enabled, if a transaction is committing
and there is a new freelist page at the end of the database file which would
cause the database file size to grow, ensure that page is written and the
file size grows before the block-atomic-write commits.  Fix for the
problem identified by [forum:/forumpost/3bd8d497b2|forum post 3bd8d497b2]

FossilOrigin-Name: c9fdd6805df04f05ef347e5a43506fd37a729c5924abb6e1103e871c4ac2d6dc
2023-10-30 12:09:48 +00:00
larrybr 7c2d3e8a06 For Windows CLI, institute a version check to determine default MBCS or UTF-8 translation on console I/O. (Default to UTF-8 where known possible.)
FossilOrigin-Name: ddc6ead6453e0f98943bd07aedd90d47bc2e9e9e27b008d493491168bea2b3f1
2023-10-29 20:05:18 +00:00
larrybr 84eab13df9 Simplify code slightly. Improve comments on added code and its use.
FossilOrigin-Name: 046c84296627382ee416f64b02b77a937b368e30b32e6b800de5a854810766f6
2023-10-29 19:55:22 +00:00
larrybr 060c097e0d Properly close a handle. Use a putatively effective runtime test for UTF-8 console I/O capability. (This makes the version test useful mainly for avoiding a warning that UTF-8 console I/O could not be setup.)
FossilOrigin-Name: dc91eb91725f3db65c73725f1fbcf18a711cafb65b4fea3277aa0905a24df353
2023-10-29 16:26:12 +00:00
larrybr 3017a6851f Condition default UTF-8 console I/O for Windows builds on OS version 10 or more. This is to accomodate an IsValidCodePage() API which may happily report CP_UTF8 as a valid code page when the stock console cannot, in fact, do UTF-8 I/O.
FossilOrigin-Name: 6b9b2a886fd4d239c2e87c3f3809c011f77c0f60e0c279bbe4e1d1b53c609e2d
2023-10-29 00:24:22 +00:00
drh 5269e846dc Limit the range of integer unix timestamps in date4.test, since some systems
(RaspberryPI) cannot deal with timestamp values that exceed pow(2,31).

FossilOrigin-Name: 765290663b28e90a0494997baf023f9610a4ed32f0ff0099bf9fc3d485733fca
2023-10-28 16:19:21 +00:00
drh 50862112b9 Update documentation for sqlite3_errmsg() to try to make it clear that if
the schema of a database contains table names or similar with invalid UTF,
then the error message generated by SQLite might also contain invalid UTF.
No changes to code.

FossilOrigin-Name: 33ba13c7c4b6f9c5e64ea30c819718f2caea214afca945c9ed7075864f4aaa40
2023-10-28 11:53:14 +00:00
drh 1b772cae2d Since SQLite considers NaN to be like NULL and NULL sorts before any integer,
make sure the sqlite3IntFloatCompare() routine reports that the integer
argument is larger if the floating-point argument is NaN.

FossilOrigin-Name: de1cf31a34b0f21288e7e30434a06baf25ee579929107c22e65c57236577fc4a
2023-10-28 11:40:33 +00:00
stephan 793e872803 Merge trunk into jni-post-3.44 branch.
FossilOrigin-Name: 9670eb2496b4005cf718d9fd12dbd11733f7cf6704a967ebb316504ea6a51e82
2023-10-28 03:56:14 +00:00
stephan a9fea28761 Remove the SQLITE_MAX_ALLOCATION_SIZE flag (set to 536mb) from the WASM and JNI builds because it can unduly limit db exports via sqlite3_serialize(), as reported in [forum:75524f7342c1ba45|forum post 75524f7342c1ba45]. It now defaults to whatever sqlite3.c uses, which is currently just shy of 2gb.
FossilOrigin-Name: f6e1137919243c5ce86725df64b40b7e12e82cbceaff210ca41616d620f0dd1b
2023-10-28 03:54:28 +00:00
stephan 95c11b1158 Add missing docs for the JS Worker1 export method, as pointed out in [forum:75524f7342|forum post 75524f7342].
FossilOrigin-Name: a7b267bd11216ee990cdd855044fbc18c300dff07cf25b317f27c1bbbc340dcc
2023-10-27 23:02:01 +00:00
larrybr 4b8010a8ee Fix cosmetic flaw in SQLITE_TXN_* doc. (no code change)
FossilOrigin-Name: 72d7c18f80f41529811f74855ac198681a0cfe7634225233ec4b8df219c2e73b
2023-10-27 21:44:53 +00:00
stephan 26422f823f Export sqlite3_get_autocommit() to WASM and add tests for it.
FossilOrigin-Name: 7cb6d3cd3926882240c91a4b90fdf237b931653295d8d94b05ed885dcae59608
2023-10-27 21:05:50 +00:00
drh 6572f7799c Disable two rtree test cases for Linux-i686. We do not have that platform
available for testing.  The test cases work on all other 64-bit and 32-bit
platforms that we have at hand.  So there is no way for us to debug this
problem.  Better to disable the test.
[forum:/forumpost/ffcbf789b5386573|forum post ffcbf789b5386573].

FossilOrigin-Name: fed4d6e90baecca716429f95e1fab7930297afa2a191a87a0a87cb93910d1bce
2023-10-27 18:16:15 +00:00
drh 2e0d7b28c1 Add makefile targets for fuzzcheck-asan and fuzzcheck-ubsan.
FossilOrigin-Name: 27fc14303a0c6942e8cf5017d66c7ff5fcbc700a6109a93768263370f608397e
2023-10-27 17:40:46 +00:00
dan a03932a790 Fix to Makefile.in so that the "tool-zip" target can be run from outside the source tree.
FossilOrigin-Name: cd118da9db0680fab8761ff44384639b9d564b67a5dfb1280ab72e44ff57714d
2023-10-27 15:31:44 +00:00
larrybr 1b2a93ea36 Cleanup shell.c comments. Hide -utf8 option, as it is undocumented and accepted only for backward compatibility. No functional changes except for -help content.
FossilOrigin-Name: 3a87995560b5acbebeb5af407aa9eddf4c37c6a27ec9429ece0ea931918f9d9c
2023-10-27 13:59:05 +00:00
drh c42276f63e Back out [2904fcbeebba9189], mostly. The page bitmap is needed even for a
partial integrity_check in order to avoid an infinite loops while walking the
database.

FossilOrigin-Name: dd6e03b37356e2961abd97f636402c09744c98c3033f3435503958c6219f9121
2023-10-27 10:49:17 +00:00
larrybr 1ecf531fb0 In CLI for Windows builds, do MBCS/UTF-8 translation independently for input and output. (And cure an MSVC warning.)
FossilOrigin-Name: 80f0b6f9c76c049edaaaeedc1745978b9d5b02a9c34bbf64209266a154b2b0e3
2023-10-26 18:41:13 +00:00
larrybr d7969e5274 Handle utf8 mode restore better for redirected input or output. Move utf8 mode setup to before any invocation command processing. Sync w/trunk.
FossilOrigin-Name: 59452e7c72de02c2b6d40974c91173d87fd5f6f2e28c0b1f6f53eb1a09693fb6
2023-10-26 18:24:41 +00:00
drh 2473180bb1 Enhance the "tool-zip" makefile target on Windows so that it uses more
universally available commands to build the ZIP archive.  Fix other minor
issues with that makefile target at the same time.

FossilOrigin-Name: d2f2f9126562c24966b0e9bc3629b85950c20199c41250478ba1da68eb7a0046
2023-10-26 17:30:54 +00:00
drh 113e15fbbe Further enhancements to make sure the correct returnning trigger is run when
there are nested INSERT RETURNING statements.

FossilOrigin-Name: d83405e870b071540b56e76291bdce1388db80b860afd68a34e4233f6ff2a3ab
2023-10-26 16:59:22 +00:00
dan 94331d406a Handle the case where a virtual table xBestIndex method called while coding a trigger fired by a top-level statement with a RETURNING clause prepares a statement that also contains a RETURNING clause.
FossilOrigin-Name: ec6bffa3b0fbf7cdee852cb0208de1f5869dafd88e2ee64c23cec89634219adc
2023-10-26 16:05:57 +00:00
larrybr 69d40f8e1f Ready for merge, but could use testing on some dusty old Windows machines.
FossilOrigin-Name: 20220615621e01ff83fd26133840e2553a6087bcb56bbb8e9bf1408dd881cd9b
2023-10-26 15:12:41 +00:00
drh 3c75a03615 Deal with the case of a reentrant INSERT on a virtual table where the
outer INSERT has a RETURNING clause but the inner does not.
dbsqlfuzz 3ac9a1e33f676254e02c0f297263b0a7aeb0c1a5.

FossilOrigin-Name: 8aba78e4dbf5c196aa50a28579f4bcd7c96103661c5968d62b2fded075bc73e4
2023-10-26 12:59:41 +00:00
larrybr 634b037fa3 Cure a harmless MSVC warning. (no other changes)
FossilOrigin-Name: 02234ef85ed9349ddf6225f67b7e941b0b68cbd2e58c9d7d84a8324393349853
2023-10-25 20:44:01 +00:00
larrybr 63aff62e69 In CLI for Windows builds, do MBCS/UTF-8 translation independently for input and output. (WIP)
FossilOrigin-Name: 47b13e66e07be3a26cf7fa0b7b65ad8576c3b4be50fc422be8fe93be93d681ac
2023-10-25 20:27:18 +00:00
dan 799e743333 Updates to the mktoolzip.tcl to work on win32.
FossilOrigin-Name: 166b775bb523362ffe17722ce175278726303c319fbe7e4a1e9291736db38a5d
2023-10-25 20:08:21 +00:00
drh 208f5c65d4 Fix harmless compiler warnings.
FossilOrigin-Name: cc8efe0494b8fe0df18aa67b1675779bf704d1ac53647fe1f7f55d8048041680
2023-10-25 19:06:23 +00:00
drh d91884586a Provide dummy parameter names in the prototype for the new xIntegrity method
of the sqlite3_module object.

FossilOrigin-Name: 131374d720a365cbbe3031a2bc0a34ba1c74e751e39c66e484dbce07ab49fb62
2023-10-25 18:53:00 +00:00
drh 1d91f8619b Fix minor problems with the "showwal" utility program. Add a missing header
to the zipfile.c extension.

FossilOrigin-Name: 7b3f866e619c2b781b613493727cd111335ceec690d984408725c756feff26e6
2023-10-25 18:35:48 +00:00
drh b7086c48b6 Test module altermalloc3.test requires DQS_DDL to be turned on.
FossilOrigin-Name: c4d02f4b0761a2836aa34f204e725f49478a19b02160e694e01042e6e7766167
2023-10-25 17:26:11 +00:00
drh 47a484d8ed Add an ALWAYS on branch that was made always-true by the DISTINCT ORDER BY
fix.

FossilOrigin-Name: c29ebcb0be8e969c359c077bd4ce752a9c952d2659062868f12efdd2cc77186d
2023-10-25 15:30:11 +00:00
drh 8f99cb9ad6 When doing a DISTINCT aggregate that contains an ORDER BY, only the
arguments to the aggregate need to be distinct, not the ORDER BY terms.

FossilOrigin-Name: d2dbbdf7194bab4e5e3b74d3dbffb012a335829824c775c72dd7347c013d2125
2023-10-25 14:54:16 +00:00
drh a9a7d118f6 Fix the amalgamation generator so that when it is reporting the Fossil
version number in the header comment of the amalgamation, it does not
elide the first hex digit of the version hash.

FossilOrigin-Name: cc66f526bd58c110cac027583f636099fa05b2b56d7ea9807b554cd950c1bf3c
2023-10-25 12:58:06 +00:00
drh 9f20bde65a Enhance the new xIntegrity method of the sqlite3_module object with new
parameters that provide the name of the table being checked and
a flag to indicate a "quick_check".  Based on feedback in
[forum:/forumpost/965c0d02ea|forum post 965c0d02ea].

FossilOrigin-Name: bc8afa3f15954bb35f65dbf940bf069de5e14d333036676c24430cf17b658d05
2023-10-25 10:37:11 +00:00
drh f4154879eb Implement check-in [477577120b897bf1] differently, so as not to disrupt
non-standard build configuration. See
[forum:/forumpost/c11523ca2df50293|forum post c11523ca2df50293].

FossilOrigin-Name: 50448fe4fdc8fd93303fe26bdcd885ecc606080c8e66e69d5be8dac28a77492b
2023-10-24 19:56:57 +00:00
drh 85b1f5c2f6 Test cases for COLLATE and NULLS FIRST/LAST in the ORDER BY for an
aggregate function.

FossilOrigin-Name: 641f928feb8b6cfd64cb03992bc18c1653960b6eaeb35145d17df494727c7a11
2023-10-24 17:45:11 +00:00
drh 9d486435f1 New #ifdefs to fix certain compile-time options.
FossilOrigin-Name: 688c6279ef1d4ff8d58e83a1c73ca0221ab8570a7cd89459946a86afdad6ec1c
2023-10-24 16:16:27 +00:00
dan 3da982635c Fix a problem with the fts5 highlight() and snippet() functions when used with tokenizers like "trigram" that output overlapping tokens. Forum post [forum:/forumpost/63735293ec|63735293ec].
FossilOrigin-Name: e952db86faaafd2ef8558175ebcae6834d0b0002813cc78c7ae7da713604cbec
2023-10-24 16:06:56 +00:00
dan d548f74024 Fix a problem with the fts5 highlight() and snippet() functions when used with tokenizers like "trigram" that output overlapping tokens. Forum post [forum:/forumpost/63735293ec|63735293ec].
FossilOrigin-Name: d570aa02f79b1d7d3889e33f9eebab1b7edcf5231b1357451eed9a538607de54
2023-10-24 15:53:02 +00:00
larrybr bdcae89358 Spelling improvements, in comments only
FossilOrigin-Name: 9ee25eeaf17124dbe34b961384e74c1ae239ee1798bb01907938600ed6b4066f
2023-10-24 15:11:55 +00:00
larrybr da67619d97 Spelling improvements, in comments only
FossilOrigin-Name: 65bcc583784476f8cc52dfe72ecc52df93d0ec5817c313ad7a83905c0047dd8c
2023-10-24 15:07:28 +00:00
drh c0ba6a97ec Fix various harmless scan-build warnings.
FossilOrigin-Name: 54be9af4469d7e31ee852f67e5aa32996557c10de654a60103fd165d2fedf311
2023-10-24 11:06:44 +00:00
drh eb9882d7d1 Fix a harmless compiler warning in the expert extension.
FossilOrigin-Name: 0c4907ddf9abd1ebfef31c1a53d702c4dcaa015c0032b8b52774c9e260b6cfd7
2023-10-24 09:57:54 +00:00
drh 13e69440fb Fix a minor typo in the previous.
FossilOrigin-Name: 9d388267e4e6724e2df333fe09d509e87defcfe984c5c2ebe031152d320812d0
2023-10-23 23:34:53 +00:00
drh 043768ce5a Add notes on how to build a statically linked version of sqlite3_analyzer.exe.
FossilOrigin-Name: 99ce9939c861ecc07877e640f75d395afb6502130bf4b5ccea8b98a80a1569ce
2023-10-23 23:34:24 +00:00
drh 811bece3f7 Add the sqlite3res file to the Windows build of sqldiff.exe so that the
resulting binary shows the SQLite icon.

FossilOrigin-Name: c696a4963bdd010c8161acbf7601078313e7da5ebb3f60381a489a70e5728dea
2023-10-23 19:51:16 +00:00
dan 4fecc81e20 Avoid attempting to delete a file while it is open in test script sessionalter.test.
FossilOrigin-Name: f20b4a96f747adeb2ed486f06dd7d911f0f24b37df87dcd32d2c3e4b687da9a6
2023-10-23 15:30:38 +00:00
dan 2db02ec245 Revert an earlier change that considered passing a NULL callback to sqlite3_preupdate_hook() to be a misuse. This is required to clear the preupdate hook altogether.
FossilOrigin-Name: 1bf6f0f3bb54933e4e6c730554144934f73a7fac87c97a916b275afd7379f397
2023-10-23 15:24:44 +00:00
drh 8c86f56e90 New test cases for PRAGMA integrity_check against FTS4 and FTS5 tables.
FossilOrigin-Name: 3950d20c3a4433543546ce4495ae07aab3d6cefb163b1f30813a9776c5b962e8
2023-10-23 15:23:10 +00:00
dan fd05978a1e Fix mutex1.test to take into account that the prng mutex is not (as of [c84e4483cb4]) taken when writing a header to an in-memory journal.
FossilOrigin-Name: c9e37417fcc41f1e975290bf583113ff8f4144c056e8fcd9fafcb432c1a26b18
2023-10-23 14:24:09 +00:00
drh 45b2e946cc Improved error messages for the xIntegrity method in FTS3/4 and FTS5.
FossilOrigin-Name: df69abc0823f4dd9c051f8d5993d2f447ee0dadab5cef73d670107b1071db255
2023-10-23 13:46:22 +00:00
drh bd8941a4f8 Fix sqlite3ExprDup() alignment assertions so that they work on 32-bit
platforms.

FossilOrigin-Name: 143f95efc7114455b1920f1d2f9a867392740f608d1dda4306005f01238103f8
2023-10-23 12:16:27 +00:00
drh f49d52d642 Temporarily disable the ext/expert/expert1-6.0 test, which does not work
on my configurations.

FossilOrigin-Name: a41b903d0fe489d20c4a7118a2c91f7c8a78568ce552225f64c2945bb811ea16
2023-10-23 11:41:41 +00:00
dan 3f81c3264a Add an SQLITE_OMIT_VIRTUALTABLE block around the new vtab code in "PRAGMA integrity_check".
FossilOrigin-Name: ab99eb2b6c50f4f0bb42918b9bdda37e2ef9316bed07de18fa4bd373c49a8b00
2023-10-23 11:34:24 +00:00
dan 06700d77c2 Add missing test file fts3fault3.test. Should have been part of [7f41d700].
FossilOrigin-Name: 92e64ab63a78445fe20863f9679c55a630a2295ff10c4ce33eab6dfaeef09622
2023-10-23 11:18:13 +00:00
larrybr 47b496868c Merge fixes for expert handling of UDFs and other UD-whatevers.
FossilOrigin-Name: b5d7d07df58bb8e4013b4047e4019b96e506e8ef03ca0abfc9aae06a122b353a
2023-10-23 02:08:34 +00:00
stephan 334b6fe418 Add missing JNI sqlite3_result_subtype() binding.
FossilOrigin-Name: ac920b5386db525c794f4cae332dedcc709cac356f6fb85f3e92bc0a34602511
2023-10-23 02:06:27 +00:00
drh aa040d891f Fix the shell1.test test so to align with the new behavior imposed by
[bce807cd48763273] - that backslash escapes only work without quoted arguments
in dot-commands of the CLI.

FossilOrigin-Name: 0db82b4281a0e0d5e365553df11e0347f60c00c861c0fb96227059edff3a0ef6
2023-10-23 02:01:14 +00:00
larrybr 81be0d26bb Clear some picky warnings, sync w/trunk.
FossilOrigin-Name: ee58425904b36319e016dc69bb5f141bb3565b1723a97490a4b2cfa16b89fbdf
2023-10-23 01:55:35 +00:00
stephan 90a5617b55 Add many more high-level wrappers to the JNI wrapper1 API. Correct the JNI bindings of sqlite3_strglob/strlike() to compare as the core lib does if their glob argument is NULL and the other is not.
FossilOrigin-Name: 55c4b1dc402b358d53d65fa1f6ec063e9e38e95c81a05d98dae3cb58c52ef55c
2023-10-23 01:34:17 +00:00
drh 5e6dde3b35 Fix [f5c01676fd281e93] so that it always preserves 8-byte alignment for Expr
objects.  Add new assert() statement to verify this.

FossilOrigin-Name: 678a9728dc6b88d8ef924c86603056df18204bc9a9c4776b9baffd7c5b10c5f2
2023-10-22 23:44:32 +00:00
stephan 96aa4d344d Add high-level window function wrapper to the JNI wrapper1 interface.
FossilOrigin-Name: a27e7471231a24864cbd04b77cbc4b336ce180d738a36ce4318543e2666ed708
2023-10-22 23:36:16 +00:00
drh a63539143d Do not do backslash excape processing on any unquoted strings in dot-commands
in the CLI - on Windows or on posix-like systems either one.  This brings the
processing into alignment with the documentation, allows backslash-delimited
filenames on Windows (as long as they are unquoted), and causes the CLI to work
the same with regard to backslash escapes on both Windows and posix.

FossilOrigin-Name: bce807cd4876327396b4ffcdf77f6931dd3bbd3314336eedf38bcf01d17af32c
2023-10-22 17:27:55 +00:00
stephan 166c8d0067 JNI: flesh out and simplify the APIs for binding and fetching arbitrary Java objects.
FossilOrigin-Name: 89fecf1dd8b97941f9b45130a3c8a67af36ec65cc6f70f5026c569c058a4963f
2023-10-22 14:25:37 +00:00
stephan 32a0d6129f JNI: add column-get bindings to the wrapper1 Stmt class and extend the AggregateFunction tests to ensure that the aggregate context is honored.
FossilOrigin-Name: 60a0e82db26270af9d0a5f55c6173e4fd0bdc90a885e838480ed75f8ef193287
2023-10-22 13:54:26 +00:00
stephan 4598b6e5ae Add API_ARMOR to sqlite3_clear_bindings().
FossilOrigin-Name: f3fb4d345bbf5ae4a35d8076043df601b1bf7dfd68760a416440139eb3e5eb9a
2023-10-22 13:09:37 +00:00
stephan 4bcde614d8 Add JNI wrapper1.SqliteStmt.bindXyz() APIs.
FossilOrigin-Name: 54fce9bf04a7517cdc8e96fe2efec66f03b0d03983c3a45d0ae7e1f16aa5a6c9
2023-10-22 12:43:30 +00:00
stephan 28f45b1ab1 JNI: improve UB protections in sqlite3_bind_blob/text/text16().
FossilOrigin-Name: 5c8383210a87d7f9d37a27053b5b1b6f41794fa8612826c68c1ca49c495cbd97
2023-10-22 12:33:05 +00:00
stephan 6e4ef06ca1 Add missing ScalarFunction.java to JNI build.
FossilOrigin-Name: b8258103fb433d9f5cfa15661b5edf4e60128fb4161d8a18e5cc3253e5aed72b
2023-10-22 11:11:54 +00:00
drh 06f4dc7edb In the CLI, do not translate unquoted backslash escapes on dot-commands
as that can damage filename arguments.

FossilOrigin-Name: 4a0e2c6e17eefb296b7e91a13305b2260d6eb869a37bc5e5b79edaf3c19c810a
2023-10-21 22:31:00 +00:00
drh d2c737f2e4 Remove a branch that cannot affect the outcome from FkNoAction processing in
fkey.c.  Replace it with an assert().

FossilOrigin-Name: 023a9dbe83c0042e9d500e3ae6c0592a468982e4ac278d08c9201967506c7555
2023-10-21 20:34:57 +00:00
drh 76bd238a66 Replace the patch at [5cb61c6788d7c017] with a better way to avoid failing
PRAGMA integrity_check if a module implementation is not loaded, as the use
of suppressErr can lead to problems.

FossilOrigin-Name: be5737ee4f4f1d5f8b3126f6eb70210bf3d8b1cdaef51aae818eaaba43ccf722
2023-10-21 20:03:44 +00:00
dan ef6bf1bbe5 Fix a problem allowing a COMMIT following an OOM to cause fts3/4 corruption.
FossilOrigin-Name: 7f41d7006db4225cf9b3d197d3a76842778669ac079e76361214a8023c9976e6
2023-10-21 18:12:07 +00:00
drh da4cfde031 Add the SQLITE_TESTCTRL_FK_NO_ACTION test control. Make it available in the
CLI.  Fix a minor problems with SQLITE_CHANGESETAPPLY_FKNOACTION in sessions.

FossilOrigin-Name: 91b64c6a70744704fc3285be7d8d46ba679ea5f0f69bd77b0e093eeb7447947d
2023-10-21 16:33:20 +00:00
drh 5c8cfe9617 Extra comment on the implementation of SQLITE_TESTCTRL_FK_NO_ACTION,
warning about the need to reset the schema in order for the setting change
to take full effect.

FossilOrigin-Name: a50a333ae11ba5d92f432108308ac0bec9afb00f466b78c8d3f3aa7e2851ef21
2023-10-21 16:25:44 +00:00
dan fcef73a787 Add missing "finish_test" to sessionnoact.test.
FossilOrigin-Name: 9a1d37f5479adedb93f40e8189ad0efa891a438805d80774ee992d4dfd71150f
2023-10-21 15:21:06 +00:00
dan ca9dc173a2 Fix a problem with SQLITE_CHANGESETAPPLY_FKNOACTION and the foreign-key-trigger-cache.
FossilOrigin-Name: 95304ec6c9654b4366dc58ae263270cb42861cb5017174936f816922b5eb61e6
2023-10-21 15:04:24 +00:00
drh 51b358e5e5 Add SQLITE_TESTCTRL_FK_NO_ACTION.
FossilOrigin-Name: 563cf5f782cdddbbd7f727c65118edfd109aeb731c8aaf0d6ee5ee7030e61ea9
2023-10-21 12:54:37 +00:00
drh a35a54b824 Adjust an EXPLAIN QUERY PLAN output in scanstatus2.test to account for a
different loop order taken due to the removal of the view-scan optimization.

FossilOrigin-Name: 9deb8b6915e794bf9c5de88f2727ef9857351eaf905f8ab56754bc4d399c88b4
2023-10-21 11:43:54 +00:00
drh 813159e2ea Compress the content of the status line in testrunning.tcl so that all the
same status information appears but with less punctuation and with "zero"
values omitted, so that the status line will fit comfortably on an 80-column
terminal window.

FossilOrigin-Name: 966351311682a319ec796c07d407ce90cd1aa0cea9e904ba1085a4ebadbdf925
2023-10-21 11:34:59 +00:00
drh 531ca6058e Create the new string_agg(X,Y) aggregate function which is an alias for
the two-argument group_concat(X,Y) function, for compatibility with
SQL-Server and PostgreSQL.

FossilOrigin-Name: a3b3df8a9e43f1880a904b76027db56ed504a84622831c96ba1962a19bbed762
2023-10-21 11:13:47 +00:00
drh 266e5d034b Avoid an unnecessary malloc() for the page usage bitmap when running
a partial integrity_check.

FossilOrigin-Name: 2904fcbeebba9189cebc48e58d12fc46f78ee23d8c4d46644606691f8cabb80c
2023-10-21 11:06:03 +00:00
drh 51bb5dec6c Fix a test case broken by the previous check-in.
FossilOrigin-Name: 1e61864c3bffec133632f4575d59d11936e8bafc089c4d84a129dcaf7e9b6d7d
2023-10-21 11:00:57 +00:00
drh d5e040b6f0 Add string_agg(X,Y) as an alias for group_concat(X,Y), for compatibility
with SQLServer and PG.

FossilOrigin-Name: b91c19bf2680f60d7826ab5d9e7902e2dc2a55d847bbea565a6489d47f2cc8f1
2023-10-20 20:19:30 +00:00
dan 71bae9f19f Fix a problem allowing a COMMIT following an OOM to cause fts5 corruption.
FossilOrigin-Name: fba3129d1362dd3ee04d61cb3d3c48518cd98dde3febc85298f411776d60073e
2023-10-20 19:59:12 +00:00
drh 81656b15c9 Fix PRAGMA integrity_check so that it does not raise an error if the
schema contains a CREATE VIRTUAL TABLE that does not have its module
loaded.

FossilOrigin-Name: 5cb61c6788d7c0170b587e4667ee57d588c991a9901951762e33b97c6a135a82
2023-10-20 19:06:04 +00:00
drh e98f62fb3f Omit some redundant calls to strlen() used to find the size of
Expr.u.zToken in sqliteExprDup().  This inefficiency was seen while working
on the previous check-in, and I thought it best to fix it while it was
fresh in mind.

FossilOrigin-Name: b7a84eff5dcdf5b2ff81331097cdc64d0c5d16a4ae224320f39b64541ec72dc7
2023-10-20 18:09:08 +00:00
drh ab3eb5b770 Improvements to the sqlite3ExprDup() logic for faster performance and better
run-time error detection.  This check-in fixes the 5x oversize memory allocation
bug from [f371e4c0f8ea73ae] as well as all other known issues that result from
handing the ORDER BY clause of an aggregate function off of the pLeft pointer
of the Expr object.

FossilOrigin-Name: f5c01676fd281e938181b846dd2024d050f597dc6a7a91928beab9d8553dfdb5
2023-10-20 17:15:15 +00:00
dan 17c3408e0e Add the SQLITE_CHANGESETAPPLY_FKNOACTION flag to sqlite3session.h, for passing to sqlite3changeset_apply_v2() to cause all foreign key constraints to behave as if they were declared NO ACTION.
FossilOrigin-Name: fc9f82ea084159eaf3dd1757b96d17d1201b00c4e06455a7dcd8067172b25f28
2023-10-20 17:06:39 +00:00
drh 90e4a3b7fc Simplifications and optimizations to the Expr object duplication logic.
The 5x multiplier crutch from [f371e4c0f8ea73ae] is still present.  More
fixes are still needed.

FossilOrigin-Name: 56142a78163b755f16afc05201f623a7a19d9a4b0620a67f7fa20d2a965a288d
2023-10-20 15:47:30 +00:00
drh 0a261eb4bb Omit an unused constant from sessionfuzz.c to prevent a compiler warning.
FossilOrigin-Name: a0cf7e24f928183866ac54f0d6cd83c859d487a19dc87572ab0188d90d2ff87d
2023-10-20 14:05:26 +00:00
stephan 11223e0cff Correct the opfs-sahpool VFS's xGetLastError() method to return the previous error code, not 0, on success.
FossilOrigin-Name: 95a1dde63117d696323c775580b9c04f044a5b8d609e9174b739ac03ecc1336c
2023-10-20 12:23:49 +00:00
drh d707e012f0 Use sqlite3ParserAddCleanup() instead of calling sqlite3ExprListDelete()
directly when disposing of an unused ORDER BY in an aggregate function,
to avoid disrupting ALTER TABLE data structures.

FossilOrigin-Name: d083e42086733ecd79aba8c268e020b01782bfe1cfa9684ce1c277af9c8bf92a
2023-10-20 10:18:03 +00:00
drh f5e487d168 Merge recent trunk enhancements into the jsonb branch.
FossilOrigin-Name: f47a8d0a207a8442a7f621b070ce9dd1d6013ce26bcf68165d20bb379bd478a0
2023-10-19 20:46:22 +00:00
drh dcf76a8bbd Remove a faulty assert() that was added just a few check-ins ago by
[8b6fffb552c30f9c].

FossilOrigin-Name: 18e7c826f08bce51719ef045daa60200b33790f2fe312c6853f9ef6c7e9d5030
2023-10-19 20:06:20 +00:00
drh 5f53f85e22 For TK_ORDER expression nodes to always be full-size.
FossilOrigin-Name: a5c73b46f4772f214ffbfa31cb87dce82ebd690addabef539bd09def26b7744d
2023-10-19 19:57:57 +00:00
drh 5080397e67 Fix an adverse interaction between the new aggregate ORDER BY logic and the
expression compressor.

FossilOrigin-Name: f371e4c0f8ea73aee9ea0645f396e3da20f1eb97be34c83de9d94c9cbb959934
2023-10-19 19:19:59 +00:00
drh d8b840a4f4 In the TreeView logic (used for debugging only) do not show unnecessary fields
of the Window object when a function has a FILTER.

FossilOrigin-Name: d2c6b82c346dbf39e07f09d187e4a8216a56caa7df3dcfafc82e32f676a1db8a
2023-10-19 19:08:33 +00:00
dan ee3c55471c Fix an fts3 problem caused by reducing the page size using the undocumented "nodesize" option, then running an incremental-merge.
FossilOrigin-Name: 2875dcb1ac86db2704ad377d719f308c8837e196b49c98b20ceda1324f2d27ea
2023-10-19 18:23:17 +00:00
drh 4e254641ff Simplify the Expr compression logic slightly by adding the new EP_FullSize
property to expressions that are exceptions to the rule and should not be
compressed.

FossilOrigin-Name: d5ae82ec52eafed5e3dc8c9d99685f6523fce7d973ef7e8d9d75ed9b8912426a
2023-10-19 18:07:58 +00:00
drh 0588385799 Simplification to sqlite3GetVarint32() to avoid confusing gcov.
FossilOrigin-Name: 89862c51ad9715bedf5b029db484602e740f0db7404970d482ce503c1b9a0ed2
2023-10-19 13:35:22 +00:00
drh 11877c6ba4 Fix an error in a debugging routine used inside of testcase(). Does not
appear in production.  Problem discovered by scan-build.

FossilOrigin-Name: c6c9d7ed6dbc71b998aeaaa1bdeb36b1bb9b902f9d6088ad36db29f8ea8b7ef6
2023-10-19 13:00:41 +00:00
drh 266178f369 New assert() statements to help verify correctness of the ORDER BY aggregate
code.

FossilOrigin-Name: 8b6fffb552c30f9cbf7102c2e26a7b486c4c9edd9f494d803de7004db3018277
2023-10-19 12:36:59 +00:00
drh 519017f25a Fix the sqlite3ReferencesSrcList() routine so that it recognizes columns
in the ORDER BY clause of an aggregate.  Fixes a problem with
[634286828dad873d] discoverd by dbsqlfuzz.

FossilOrigin-Name: 3d26f1aaa4876f21f2c3abf13bbc37933c8f32471153e29019880cc9530cb011
2023-10-19 12:12:49 +00:00
drh d9451051f1 Add support for ORDER BY on the argument list of aggregate functions.
FossilOrigin-Name: 634286828dad873dba244751441aa729cd37b6ed5899fa5875643d4af03c006c
2023-10-19 01:26:34 +00:00
drh 0b633640fa Merge the latest changes from trunk.
FossilOrigin-Name: d18bc400146812c8c5048af7eaf4a5371ddd80fc9ce895d253b6114b9eae7fc3
2023-10-19 01:09:49 +00:00
drh 69ff726d4d Fix a false-positive in run-time error checking.
FossilOrigin-Name: cd63eec0758960d9ee63d7b964ec62e2bd622f8c94a58bd9556046381ffa18d0
2023-10-19 00:25:35 +00:00
drh d05e54efff Changes for test coverage.
FossilOrigin-Name: ddfa09c6031afd4391a9888381bf09214cd542a826b431eeb1537a070f65c5f9
2023-10-18 23:48:24 +00:00
drh 42f5ea354e Test case for alter column that is used in the ORDER BY of an aggregate.
FossilOrigin-Name: 648ddb3a8eb358fc6c95aaa561ae68476c77996abda23890054ac07166278cef
2023-10-18 22:53:22 +00:00
drh ce51b31367 Deal with an OOM condition while processing aggregate ORDER BYs.
FossilOrigin-Name: b6d44427ec0184880ad14919667e2f526777c5085ffa636f6734cb5951a31225
2023-10-18 22:27:59 +00:00
drh 20b95f8d3f Make sure all terms of the ORDER BY within an aggregate go through aggregate
analysis.  Do not attach an aggregate ORDER BY to a window function.

FossilOrigin-Name: 16f3805514a741405f70e0ee3b5a6b67720bc75719372e82daa4136fe411ea2b
2023-10-18 22:03:48 +00:00
drh 49dc1be96f Basic test cases.
FossilOrigin-Name: 6cccf86c362631ada0c6cbaf661520648f8bbf7ae2b137b3d7d959647387e5c2
2023-10-18 19:44:59 +00:00
drh 0021079ceb Fix one minor error in order to get all legacy tests to pass.
FossilOrigin-Name: 5a885139c433a143302ab43dc5aaac1fed9db61548b4ff7d4b19b3a6a46c8fcc
2023-10-18 19:07:45 +00:00
drh 59a0d0bbf9 ORDER BY on aggregates seem to work, at least for simple smoke tests. Lots
more testing is needed though.  Surely there are many bugs.

FossilOrigin-Name: 64c12a835b6f1df8f2f5f4a41de083f6b3fc7f8030042c6aac0082382cd9cc4d
2023-10-18 18:11:11 +00:00
drh db19f48b69 Basic error checking. Resolve symbols in the aggregate ORDER BY expressions.
FossilOrigin-Name: c83a53a574d312130d1238c05ffa449d8bed2535d5ef5b5d9cf02f894494cca4
2023-10-18 13:58:31 +00:00
drh f8202f1ff3 Enhance the parser so that it can accept an ORDER BY clause on a function
invocation.  For this incremental check-in, the ORDER BY clause is currently
ignored.

FossilOrigin-Name: 3a98ff24bf468ed42d410a9a12d9f9b2ca154c7babe99fd6bc6f7b0565e0d132
2023-10-18 13:18:52 +00:00
stephan 0b4de1acac JS: replace one errant reference to 'self' with 'globalThis' and remove a separate dead-code 'self' reference.
FossilOrigin-Name: da1a47932ea96755b1e11fc2547ce11780b87846af6885857318ff18f1e62c31
2023-10-17 23:35:47 +00:00
drh 43dc31cf1f Fix a harmless compiler warning about variant types for a pointer function.
FossilOrigin-Name: 37ff0d8e7f91c32e8c53bb015280be47c66bf599281b640a8d3fd41335b55289
2023-10-17 19:33:52 +00:00
drh 6db4e5ed4c Make a new branch in the SqlExec opcode reachable by tests.
FossilOrigin-Name: 023ad470b68b7c09e159244493a6d0285d706efdecd8719a71e6885509d4750c
2023-10-17 18:59:13 +00:00
drh 42b49a3138 Omit an unreachable branch when SQLITE_ENABLE_API_ARMOR is not defined.
FossilOrigin-Name: 5f8b807b568f4f9ec65f1c924e0e1ac2f996f405e045204145ad7116be7580ee
2023-10-17 18:28:27 +00:00
drh 4c827feba1 Changes to sqlite3IntFloatCompare() in an attempt to better measure
branch coverage in the face of aggressive compiler optimization.

FossilOrigin-Name: 5781d043ffeccda03357ee530564987443dc1deb6111a4028701b4a98fdfe16d
2023-10-17 17:53:46 +00:00
drh 52d934e316 Fix a JSON bug introduced by the optimization of [df099ad713011b67] and
first appearing in 3.43.0.  The problem occurs when doing a JSON_EXTRACT()
on an array element that was added by JSON_SET() without first reparsing.
Reported by [forum:/forumpost/fc0e3f1e2a|forum post fc0e3f1e2a].

FossilOrigin-Name: e5099c549a1d8959d4015516f090b8e6438e517a64b20651175bf5413d94fb58
2023-10-17 13:41:41 +00:00
drh 5e0b11bda7 Enhance the documentation of sqlite3_deserialize() to make it clear that
the input database may not be in WAL mode.
[forum:/forumpost/a7e272cee9ac469f|Forum post a7e272cee9ac469f]

FossilOrigin-Name: e754789971e019647e7068f76dc6f9778675e4849fe51d8b25b39d3edf2379e5
2023-10-17 11:57:36 +00:00
drh 11095bcb89 When an I/O or similar error occurs during a transaction with
journal_mode=MEMORY, attempt to rollback before closing the journal, as
all rollback information is forgotten when a memory rollback journal is
closed.

FossilOrigin-Name: 1d67f75de259e5a26b751a50432822a268ebe367cda6510891ab81a15e5daa1c
2023-10-17 10:47:42 +00:00
stephan b866f98346 Harden API_ARMOR checks for sqlite3_stmt_scanstatus_v2().
FossilOrigin-Name: d3f38e813ba3b887e973af034713cd58fdfbe07da375b154accc7b5790cfaeda
2023-10-17 02:15:49 +00:00
stephan 08747d44a2 JNI: add aggregate function support to the wrapper1 API.
FossilOrigin-Name: 15b28b340a5c5efdbfe3fbed16ee0b699561edaeebb77446addf2374bdf9357e
2023-10-16 16:04:23 +00:00
stephan 626d0a9fda JNI: add scalar UDF support to the wrapper1 API.
FossilOrigin-Name: a850535766d2243d9475e1523c753615875a2da9c9d82a41a9fb61b141c6334a
2023-10-16 14:31:13 +00:00
stephan f2d7e961d9 JNI: initial draft (untested - requires more infrastructure first) of a UDF argument/result-handling interface which completely hides the C-style API from the client.
FossilOrigin-Name: 43b10a5cf9cb8be53d62914f340d533e60a70bf4caa8b9b91c0f867fa0f70493
2023-10-16 13:04:42 +00:00
stephan 7e540e5a2c JNI: cleanups in Tester2 and update the jar makefile target to account for [9fcdf96adca2].
FossilOrigin-Name: abc82bf4b800dde1b6e6172c7be816edb391fdbed5dbd2749f54623fdf3bf8e6
2023-10-16 10:38:34 +00:00
stephan 22fd153e43 JNI: move the C-style API parts into the capi subpackage and the higher-level wrapper into (tentatively) wrapper1, so that CApi.java can support multiple independent higher-level wrappers without name collisions, and CApi can be made public but have the option to be elided from wrapper-level javadocs for wrappers which do not wish to expose it.
FossilOrigin-Name: 9fcdf96adca25cc2d2f4b75ec4eea94254fb9671c5ba63b88213d7f62dedff1b
2023-10-16 10:27:19 +00:00
stephan 19179722d6 JNI: after calling a Java-side UDF, zero-out the pointer of the Java-side sqlite3_context and sqlite3_value array entries to avoid misbehavior if a client makes the mistake of holding a reference to one of those objects.
FossilOrigin-Name: 9fc3104f76a83d600beb11d91feb97bcea8bc7f7cda8cd73e7a6b81fbba879df
2023-10-16 08:10:11 +00:00
stephan f83a4d850c JNI: do not expose SQLITE_OPEN_... flags which are specific to VFSes.
FossilOrigin-Name: 2b4e53d8be42a3bc098317abd8bb58b8ddc25094d80787f784bbc896f4f7b976
2023-10-16 08:05:51 +00:00
stephan 24f6baca16 Change 3 instance of #if SQLITE_ENABLE_API_ARMOR to #ifdef for consistency with how it is normally used.
FossilOrigin-Name: dd766eeb59fec71627dd8ad8f120875b96fda455c6401e5671e086b785e2b2bc
2023-10-15 13:36:21 +00:00
stephan 5698e62f9e JNI: enable all optional components in the default build and fix a test broken by an option toggle.
FossilOrigin-Name: 5586a9df58a400c20a6815e8b0fbc7a7c49e5ef89daf92c452f1990586ca78ae
2023-10-15 13:32:38 +00:00
stephan 29aeb620e7 Correct non-void return from sqlite3_preupdate_hook() when API_ARMOR is enabled. Broken by [6cb77503484e].
FossilOrigin-Name: 40f5906f48d3f1f44d54e5b271fe04e7082166e50e3ed250bb57a4760f112e8a
2023-10-15 13:25:39 +00:00
stephan 6089e5bfcb Add coverage of more functions to SQLITE_ENABLE_API_ARMOR builds.
FossilOrigin-Name: 6cb77503484e104a51f08690974d6e19c336ab5467d327f84be1d042615f008c
2023-10-15 12:11:10 +00:00
stephan 8a273bb6e8 JNI: extend [baf220e78a46246c47] to include macro-generated sqlite3_value_...() bindings.
FossilOrigin-Name: ad34b2542e330c22fcf331810ab43e66346b1bad8de1a3989e6208cfa0fc2f92
2023-10-14 20:44:13 +00:00
stephan 4b1e42ef94 JNI: make the sqlite3_value_...() family of bindings resistent to NULL arguments.
FossilOrigin-Name: baf220e78a46246c476cd429ac131698db5fa66f62fb5725d4f736fe5246283c
2023-10-14 20:34:40 +00:00
drh d718bde6da Do not allow an ALTER TABLE ADD COLUMN on a STRICT table if the added column
contains a DEFAULT clause that would violate the type of the added column.

FossilOrigin-Name: 75b075863eaa56e36635a1d27740d37de8600ba92099b3fad9378a1e6ce12c0e
2023-10-14 20:24:52 +00:00
stephan eb62ccdab5 Add API_ARMOR support to the scanstatus family of functions.
FossilOrigin-Name: 2f3101f016d5f8d954a262befdf5e0f938a223e444bc534e1d3655b733f4663d
2023-10-14 20:01:55 +00:00
drh 153790d9ae The rtreecheck() SQL function should not invoke BEGIN or COMMIT as this
causes issues for statement transactions.

FossilOrigin-Name: b7b2e30b570efda338c62e88446dedb6667640f33855b2422ad20c7b569b025a
2023-10-14 17:14:53 +00:00
stephan 4213cc78cd JNI: add a missing result code check to sqlite3_bind_value(stmt, null).
FossilOrigin-Name: f1a357f42de58927c33e7047dc4417e3b9549d7529625f30c583e66b7d603b00
2023-10-14 16:57:48 +00:00
stephan a9000d424e JNI: add missing sqlite3_bind_value() and minor memory-safety-related cleanups.
FossilOrigin-Name: 3900031763abf4b2aab7b57e886cc407571d0d5928b74f24dcea8fb7371655af
2023-10-14 16:56:12 +00:00
stephan c76e450908 Add API_ARMOR support to the sqlite3_result_...() family of functions and sqlite3_bind_zeroblob64().
FossilOrigin-Name: afabe3e35a66625527e2881749cdb6e13300888ab57bc0f05889d0e3ee203d73
2023-10-14 16:29:36 +00:00
stephan 7a54b12695 More API_ARMOR additions.
FossilOrigin-Name: 78ebf838f645742f87733665cd72af736df345683b27377a2c8310c893b1769d
2023-10-14 14:53:18 +00:00
stephan a17f63290c Add column name to API_ARMOR check in sqlite3_blob_open() to avoid a null-pointer deref.
FossilOrigin-Name: 0114a6622afc4588c47e98d804340449417b603dc4831513eab4d8e4ccb15d42
2023-10-14 13:24:30 +00:00
stephan 067a13c395 Revert [f6cd88e6b234] - the NULL callback case is perfectly legal.
FossilOrigin-Name: 718ab67607895176e529eb7469832d262a347d030e83e7ee66d3b4704bf933de
2023-10-14 12:45:11 +00:00
stephan 6d035d481f Bring JNI-side sqlite3_last_insert_rowid() and sqlite3_table_column_metadata() in line with the core's NULL handling.
FossilOrigin-Name: c2afc1c2a2fc9bdf3d7b7701004e0fa40965cf8c6b7cf44b46f2ca37cfa1d2aa
2023-10-14 12:40:57 +00:00
stephan eaebd7fb6a Extend API_ARMOR checks on sqlite3_commit/rollback_hook() to include a check for the callback pointer.
FossilOrigin-Name: f6cd88e6b234560f729ff00da86144a0121ad96d5a07d227c9ffa3f43c22f72d
2023-10-14 12:20:55 +00:00
stephan 2ba1acce81 Add missing JNI bindings for sqlite3_db_readonly() and sqlite3_db_name(). Code-adjacent tweaks for the API_ARMOR audit.
FossilOrigin-Name: e5eace747ae656f1e05a346e53a2827242e57107567d2c3470e292cf0b40858d
2023-10-14 12:19:59 +00:00
drh c84a0b2195 Bug fix in sqlite3_analyzer: for databases larger than 1GiB, take into
account the lock-byte page when calculating the number of freelist pages.

FossilOrigin-Name: 26a909cdd32afee0f15968ca6f611eb259373b2a2195d0b3d47cfba103f1e6d6
2023-10-14 10:54:37 +00:00
drh 9132b8816a Earlier detection of a host of errors in CREATE TABLE, such the CREATE TABLE
statement itself fails, rather than generating an error on the first attempted
use of the created table.

FossilOrigin-Name: 348fa7aaf7958b3fb689ed023d946064ae8d92718a497a346e95114a2410cbf5
2023-10-13 22:19:23 +00:00
drh cc67a62fa0 Apply the correct affinity to DEFAULT values that are TRUE or FALSE.
FossilOrigin-Name: 4958db70c8103e249748fdeffc35f1c385e2e550e8a48697b82fcd797f3016be
2023-10-13 19:41:20 +00:00
drh 6f12e51313 Do not allow triggers on shadow tables under defensive mode.
FossilOrigin-Name: 44b932ff16e35814febd842d11215cf243393f173aade33ceded473598221e88
2023-10-13 18:29:18 +00:00
dan d7cdfe74f6 Fix an fts5 problem caused by a 'rebuild' followed by a DELETE in secure-delete mode.
FossilOrigin-Name: 18be505c628d9b13431ca6cfe822d4aeae119c53ae08aef26a67f310a8bd7bd3
2023-10-13 15:59:11 +00:00
drh af527231c1 Immediately fail a CREATE TABLE statement that attempts to create a
table that has a generated column loop.  Legacy allows the table to be
created but the table would not be usable for anything.

FossilOrigin-Name: 3237bf964117c1ef71143042837ef21872bb3d04bfd682075672e768953ec802
2023-10-13 13:49:46 +00:00
drh 9817065340 Actually prevent PRAGMA writable_schema=ON from being set in defensive mode,
rather than just preventing it from functioning.

FossilOrigin-Name: 2b574d9e7e922d0961ea27bfc0148d3098157ff8d4feefba469489c81d43f3e0
2023-10-13 12:57:23 +00:00
stephan 7dc0cc45f3 Round one of an audit for SQLITE_ENABLE_API_ARMOR for functions exposed by JNI and those functions missing armor, as [forum:5e3fc453a69b49ca|reported in several forum posts].
FossilOrigin-Name: 8c25c4b18ad07861bf0e47f99f3db04b569b9b859ad0690602f748ddf3576939
2023-10-13 12:48:35 +00:00
larrybr 99d43979d3 Avoid potential overflow in hex(). [forum:/forumpost/7ac0c9c5ea|See forum post 7ac0c9c5ea.]
FossilOrigin-Name: 2e968114fd0e60eebcc088dec02684e10b06ac1ec42d6bedb5287362f5cbf032
2023-10-13 01:39:47 +00:00
drh 5dc4a405c5 Make sure virtual tables have been connected before trying to invoke
the xIntegrity method during PRAGMA integrity_check.

FossilOrigin-Name: 4a4eccb681741917c6771101d4e329f91eae9b035f4d345218e522bb55fff6da
2023-10-12 20:51:17 +00:00
dan 3046039309 Fix a problem with an fts5 secure-delete on a rowid/term pair that follows a legacy delete of the same pair.
FossilOrigin-Name: 579aea0c28e01a79620ac758edc02db3a01baaa073e7773b8f0b6f610479520b
2023-10-12 19:46:58 +00:00
drh 4b136ed70a Enable SQLITE_VTAB_INNOCUOUS for FTS3, FTS5, and RTREE.
FossilOrigin-Name: f34c533b6c1f8ef3f69c75c1039406c12751cdde1fbdcb4d9776b24455facf8a
2023-10-12 18:46:27 +00:00
dan a11499a156 Add test case for [29937081].
FossilOrigin-Name: d3b983edf7164d30ddb8b4b745a3feff9737149d85b1c428d6ad488ac6a1960a
2023-10-12 18:09:27 +00:00
drh c7ad9f65fa Merge fixes from trunk into the jsonb branch.
FossilOrigin-Name: a760bd7bcc63aac41c989bb5f4fbc927f9a93fe9c0aa18da2f0131483ec3f189
2023-10-12 17:47:50 +00:00
drh 5d9a6c6734 Make sure the journal file is closed when transitioning into MEMORY journal
mode, to avoid an assertion fault in the new sqlite3_randomness() avoidance
code added by [c84e4483cb44f827].

FossilOrigin-Name: 29937081a986d88f495ad48748c35ff5829f0ac31dd4ad3e48d180ae2fcb9a0c
2023-10-12 17:41:18 +00:00
larrybr 35d6c24fd5 CLI to check for rare prepare failures. [forum:/forumpost/5be6395182|See forum post 5be6395182.]
FossilOrigin-Name: 286d1e8012a5f5d9e1fe6dce1a732f725b8a1837f66daf6aefb792e19e96f2d4
2023-10-12 15:12:38 +00:00
stephan 6a0f297da9 Unconditionally force use of SQLITE_ENABLE_API_ARMOR in the WASM and JNI builds. Their corresponding test suites still work.
FossilOrigin-Name: c11394b50d0687b6d6abad9c80d5cd3cdce77e9b9c278f40601c460f463744a1
2023-10-11 20:37:42 +00:00
drh ac4aea5102 Fix the use of an uninitialized value that occurs when doing a json_insert()
of a string value that contains embedded U+0000 characters.

FossilOrigin-Name: d3c0cbb90966316be9cd25e3edb501da42731e8a83c13227b90ce17d3975a2c3
2023-10-11 17:24:31 +00:00
drh 7d1c9da62d Fix the use of an uninitialized value that occurs when doing a json_insert()
of a string value that contains embedded U+0000 characters.

FossilOrigin-Name: fc5ee9e51ad4556af526a6cefca5ae5a3b1b7affc4edf09832491d6b4f4ba366
2023-10-11 17:21:16 +00:00
stephan 582d65cce3 Add (prepare, step, reset, finalize) parts of the JNI level-2 stmt wrapper and associated tests.
FossilOrigin-Name: a7082f186f2b9b6666bbc65f2eadeb74d91fa0a681e3b2468b261ffd322bd249
2023-10-11 13:52:05 +00:00
drh 58c7b770de Simplification to sqlite3ApiExit(). Generates identical machine code, but
easier for humans to read.

FossilOrigin-Name: 793bbfa5af9721bc3a61e8e5eda46dfce2f5ff3f223a7564c9e1b09f11e53cb3
2023-10-11 13:34:18 +00:00
drh 86db4555ca Fix a missing zero-terminator on a string when processing
JSON aggregates into JSONB.

FossilOrigin-Name: fb81d570a3d9b8f55e9f278d5240605c5d0f3c5abde51550797999d03cf193a7
2023-10-11 13:19:37 +00:00
drh 2b7a1f5926 Fix an issue that can arise when processing corrupt JSONB.
FossilOrigin-Name: e50045c22296be84c6bea82bb8b310f07bca820c84d4a7349b16da0cf5d2657c
2023-10-11 12:44:17 +00:00
drh 33b56217cb Improved robustness against corrupt JSONB.
FossilOrigin-Name: 0fbda92bb0eeb40f95c83f717e4e8f5bff1ac82f1c899e9f6d400d67df67214e
2023-10-11 12:21:29 +00:00
drh 2a96a1584d Fix an assertion fault in json_patch() that can occur if the patch JSON
is malformed JSONB.  Report the malformed JSONB instead.

FossilOrigin-Name: a72d54645ca0dd80c60a5ed586049dead3ea7f5fa9ad05c6610a506242a7032a
2023-10-11 11:42:06 +00:00
drh 71dc3c714b Improved robustness in the decoding of JSON5 text escape sequences found
in malformed JSONB.

FossilOrigin-Name: 35e0108af2bdd830375c31c525f8ed0e8df64959d89649a88402dc1a5c376612
2023-10-10 23:02:31 +00:00
drh 6b7b23c580 Fix an off-by-one error in the changes from the previous check-in on
the jsonb branch.

FossilOrigin-Name: 216191b113da43516d31301fb133173add66358d503677ab997bc82cc88bfea4
2023-10-10 18:55:29 +00:00
drh 59ded6b5f1 Improved robustness when translating corrupt JSONB into JSON text.
FossilOrigin-Name: 0caa320d9099adbaf98e3719003dbdc4d158abcb3d8a1af20fbcd4c08c970f4a
2023-10-10 18:42:08 +00:00
drh d88f378c7d Fix a potential buffer overrun due to corrupt JSONB.
FossilOrigin-Name: 5cbb861fc6cb7421a81c957ba9d576e0ae92cb4ed725cdb539b364d66f4ee145
2023-10-10 18:32:14 +00:00
drh 064c1688da Fix bugs uncovered by the fuzzer.
FossilOrigin-Name: c96eb7fb618dc0a5aeec8a5e85076475b77dcd56309438aba1f9bddfc8921e3c
2023-10-10 18:04:40 +00:00
drh cc5e612db4 Merge the latest trunk enhancements into the jsonb branch.
FossilOrigin-Name: f93f16c94d0a58dfa90939e84aba048be958c7b085b45a6f25c2243fdf79b3aa
2023-10-10 17:34:41 +00:00
drh 853fb5e723 Fix a bug in the mkautoconfamal.sh script, introduced by check-in [3308fdda4b81c110]
so that the "snapshot-tarball" and "amalgamation-tarball" makefile targets
work again when run from outside of the source tree.

FossilOrigin-Name: 65ccf5fef812d43aed9e00af36c90e1a499d197e30148753790445e25ee1324c
2023-10-10 15:33:05 +00:00
stephan 0b29d37ef9 Start adding tests for the second JNI layer.
FossilOrigin-Name: 0177f4dfe8a5c03c1d5dd279c6babe5a87b596c2351373f845b89fc6abf7947b
2023-10-10 12:16:04 +00:00
drh aaf1341dcd Use snprintf() in place of sprintf() in a debugging function in the CLI.
FossilOrigin-Name: 5e5e4d2b4958c329578d1e5af4ed25c1b774f0ed2d15fa6747d7d7fbbba4fa46
2023-10-09 22:45:10 +00:00
drh f32aa34346 Systematize the names of some of the translation function in the JSON
implementation.

FossilOrigin-Name: db44bd1d62084ef69c808f7d07e0a25d5a089dcb8b98b1b5d026777591bbdefc
2023-10-09 18:33:01 +00:00
dan 6c96bf2c3f Avoid calling sqlite3_randomness and taking the PRNG mutex when writing a journal header in "journal_mode=memory" mode.
FossilOrigin-Name: c84e4483cb44f827416d8caafa22f076b2f31b2024fe8c5b5bcb0c9955149d11
2023-10-09 17:54:34 +00:00
drh 01a5a19919 Add a TCL script that does various verification checks on the source tree
to make sure that generated code has been updated correctly.

FossilOrigin-Name: 1f1a358af77f4386f98010eeae8487e6d39548a6dfe58c2664552490e7661122
2023-10-09 14:56:15 +00:00
drh 7e60f6dbf1 Fix a compiler warning caused by the previous check-in.
FossilOrigin-Name: 7e8768bf8b4002b1c287f2bc95262548e2ae81b437936154f2bb1ea1f739a904
2023-10-09 14:47:25 +00:00
drh 50da20d5b1 Turn on SEH by default when building using MSVC.
FossilOrigin-Name: f57e8275cf1ab3ad8027044cda0893282d1c403a5aa9fabbb1c7de280597b5cd
2023-10-09 14:05:21 +00:00
drh 0dead8d3d1 Merge the latest trunk fixes and enhancements into the jsonb branch, and
especially the JSON cache spill UAF fix.

FossilOrigin-Name: 9422c24f4a8b290dcae61e50ec81be5b314b22c61a2bca1e194e47da1316b6e6
2023-10-09 12:57:03 +00:00
drh faef28e6bd Fix a potential UAF caused by JSON parser cache spill.
[forum:/forumpost/b25edc1d46|Forum post b25edc1d46].

FossilOrigin-Name: a163fecca90cab9d1b7bf8ebac78d498775eed7b6d81e7920e3401633c3a4b60
2023-10-09 12:46:53 +00:00
stephan 66dacae4c3 Flesh out the error state captured by SqliteException.java. Doc additions.
FossilOrigin-Name: 5c5397ff15543f4b3620244d9e57e15708eafcab1d42c9f87b4a60f0c01e8858
2023-10-09 12:45:28 +00:00
drh 4a11142072 Fix harmless compiler warning in the test module for sessions.
FossilOrigin-Name: 3d5ec99397c183cfb8c5d31e1967e0df8a4d6edbea92bbdf728571e3a6a36d68
2023-10-09 12:39:59 +00:00
stephan c8f829cdac Add JNI Sqlite and SqliteException classes. Add Tester2.java as the main test app for the high-level API.
FossilOrigin-Name: 6acf52be7abce8dcf434c5ebf0d5e88859b033e6418077846247ecf00ccf9381
2023-10-09 11:46:32 +00:00
stephan fd013682d6 Minor JNI doc and public/private cleanups.
FossilOrigin-Name: c49d36ece283274963ce2e5a4db1e8f586dffa22e47f4adb93c625f918c3fd5d
2023-10-09 10:44:10 +00:00
stephan 68585ea342 Rename SQLite3Jni to CApi to (A) reduce name prefix collisions with incoming classes and (B) align with its counterpart in the JS build. Remove the Canonical annotation because (A) the new code separation will inherently make that distinction and (B) the line between truly canonical and semi-canonical (e.g. differing in signature overloads) is blurry enough that consistent use of that annocation is becoming an unnecessary burden.
FossilOrigin-Name: ca216b4486aff7d206ebfc3a5e84d48919c282425d4313396bc19313ffca8a0e
2023-10-09 10:30:54 +00:00
drh a7e9386e88 Remove some unnecessary code. Report errors for invalid JSONB input on
an extract.

FossilOrigin-Name: cbea16c29eb0507f39b5a1cf744a3bb9bb7c71ac156e84a19d03a37cb1816891
2023-10-07 23:35:07 +00:00
drh de8ccf00fb The return from sqlite3_value_blob() in jsonFuncArgMightBeBinary() might
be a NULL pointer.  Check for that case.

FossilOrigin-Name: 7b52b266b066f1385144c1103a3a411306db5f44568366ae1e93cd8cce799bbc
2023-10-07 19:46:22 +00:00
drh 3efb2c4791 Improved detection of malformed JSONB when parsing it into a JsonNode array.
FossilOrigin-Name: ed99a788415e1f8375bd5ec004dd18b1cd0fae4aa94558170882ca487f6dff93
2023-10-07 19:40:20 +00:00
drh f26833d783 Fix the jsonbChangePayloadSize() routine so that it shifts the payload in order
to always render the most compact encoding of the payload size.  This is
necessary as sometimes (as discovered by dbsqlfuzz) the payload size can
grow significantly due to json_insert() or json_replace().

FossilOrigin-Name: 8d6d04ca975ec55c419d40d8463c433b0db698c9fb4812ab9f16d4ee5bee460e
2023-10-07 19:05:10 +00:00
drh 1244b6cbf6 Improvements to error messages returned when the ->> operator fails.
FossilOrigin-Name: 2f3388f14c843f1c02926e8b929365c06c1f1f4ea6fe6316092c3799c14549d3
2023-10-07 17:50:06 +00:00
dan 3b381b4afc Fix a problem with sqlite3changegroup_schema() and patchsets.
FossilOrigin-Name: c4ab4200dc2538e1132d87d70fd309b26b0da8a918fede1cb09b567ea40ec889
2023-10-07 17:29:10 +00:00
dan 4bdcfc6666 Add the sqlite3changegroup_schema() API. To allow changegroup objects to upgrade the schema of a changeset/patchset under some circumstances.
FossilOrigin-Name: 03f9e50b15c9c2fb4636dee24c1491c45322578b9fed08c6f4360be07c5c3830
2023-10-07 16:32:02 +00:00
dan 18efe17553 Fix a problem with the changes on this branch and tables that use an implicit primary key.
FossilOrigin-Name: 1f9d10235ff32e72074be695049567135b02e681068ce117b66adc14a6214449
2023-10-07 16:07:02 +00:00
drh 502b11c6d3 Merge the memory leak fix from trunk.
FossilOrigin-Name: 358de1b09f3d5ec0fe459775b0a2a99dfa235817327016b472aaa1ed56d952e6
2023-10-07 11:37:00 +00:00
drh bae197c175 Improved error messages from search on JSONB.
FossilOrigin-Name: 96cfdc31e305406123f6ef1b920423f71902dc4b34381c3efb720274343d37cc
2023-10-07 11:36:16 +00:00
drh 522e880bc2 Report unknown JSONB element type when parsing JSONB into JsonNode.
FossilOrigin-Name: 1f4d3268257bda234c66c3767451b3d39d7b4c7c73a779ad396d9dfc5315905c
2023-10-07 09:13:59 +00:00
drh 8f0634549c Add a test case to prevent a regression of the leak fixed in the previous
trunk check-in.

FossilOrigin-Name: f99ff655d09763c4a22d065041644ece793d84c82c644931e89ccf50c4f4564a
2023-10-07 08:15:47 +00:00
drh 43e232dc47 Fix a resource leak: sqlite3_blob_reopen() was failing to invoke the
xDel() distructor on a register when it sets that register to the new
rowid.  This was never a problem prior to the large-column cache of
[771fe35074b50b8d] as that destructor was always a no-op.
dbsqlfuzz cd96368deaece480fb94d42427dde053737a650e.

FossilOrigin-Name: 545edd5ecf20a7feb99dc5ce6f338fa1f172c2e17b240cbede497f0729dbd3d7
2023-10-07 08:00:54 +00:00
drh 11aa2adca3 Correct handling of "raw" strings in JSON. This requires three test-case
changes in TH3 to add double-quotes to the path outputs from json_tree().
The new behavior is correct, I believe.

FossilOrigin-Name: ab2bf3e3596dfa7566f4a130adeccbef4056c7321181112a875477193583f30c
2023-10-06 23:02:06 +00:00
drh 6b8aa95c3c Improved error detection for JSONB inputs.
FossilOrigin-Name: 6945e11aa441ace52b93cdbb81ac45b6f809987f3dbc522251e63f8cb948fddd
2023-10-06 22:16:09 +00:00
dan 83e43dc757 Add missing source code comments and fix other issues with the new code on this branch.
FossilOrigin-Name: df39fbe9ab87937beb77af353cd55602290b185e222e0a92a3ebf9a9a0b9e2e7
2023-10-06 20:39:42 +00:00
dan 9fc208aada Update this branch with latest changes from the trunk.
FossilOrigin-Name: 897e926a5eaa0eab7e09e5003e4c15c974897be298a18ae85345962dc3f410a1
2023-10-06 19:46:19 +00:00
dan 53e91a5e22 Add tests for the sqlite3changegroup_schema() API.
FossilOrigin-Name: 5dab481c101b1523b1cfde92678cbc654ea26d946bf29da372d71c1f89cbaf46
2023-10-06 19:01:43 +00:00
drh 7b4349735f Incremental improvements to the JSONB logic.
FossilOrigin-Name: fe326829c27715e249f727ba797c7df6491e874ec205a0a82ee09c78d61c6e1f
2023-10-06 18:21:47 +00:00
drh 98dbfa69d3 Fixes to rendering JSON5 extensions encoded as JSONB.
FossilOrigin-Name: 5a17e4479aad2d8313170e5de83a1c52f30b55d9d4fb776024fa6622e175c63b
2023-10-06 15:35:42 +00:00
drh e690d5ad80 Fix compiler warnings.
FossilOrigin-Name: 5227add3c8d509de2e081249163fafdf30ac3173a6d710957f3c3b6f03e7017e
2023-10-06 14:59:40 +00:00
drh e1e2f2dcc5 Improvements to error handling.
FossilOrigin-Name: b41dd237fb6c0dd7daedeaaf81dbc4fdb31cf511fd32cd3d4ee69db34e389915
2023-10-06 14:52:49 +00:00
drh 91f3cd9874 Merge compiler warning fixes from trunk into the jsonb branch.
FossilOrigin-Name: 6409d307915f8969f12df2d5ffa961645bd4db7ccfbd6f52237a217b6a867252
2023-10-06 13:05:42 +00:00
drh 00eee7a786 Remove an unused parameter from the recomputeColumnsNotIndexed() routine in
order to squash a harmless compiler warning.

FossilOrigin-Name: 9bf4bfd68080367b58594e0d44b110b3ee9766420f648537fd7bc638dacefb72
2023-10-06 12:55:53 +00:00
drh 1935887a68 Ensure that all fields of static sqlite3_module objects are explicitly
initialized, in order to hush-up nuisance compiler warnings.

FossilOrigin-Name: f3b3d712d6e58b1cb8fdebd2b6b3125080b6b3ac8c7c849a8cc1e5e778d62fe7
2023-10-06 12:51:05 +00:00
drh 03f7e42538 Remove and unused static variable from shell.c.in in order to silence a
harmless compiler warning.

FossilOrigin-Name: 477577120b897bf15edc2a85e220d5c64a1d5a245354939269f6753d8140ac95
2023-10-06 12:22:14 +00:00
drh 568643fd15 Increase the size of some variables associated with the PMA sorter in order
to avoid any possibility of a signed integer overflow.
[https://bugs.chromium.org/p/chromium/issues/detail?id=1489025|Chromium fuzzer issue 1489025].

FossilOrigin-Name: dc3be3af471d90dd810c1e2cc59e83cf6a57f01971b1258ea0da402e4d577ef2
2023-10-06 12:15:01 +00:00
drh dd7f09e6c0 Improvements to coping with malformed JSONB.
FossilOrigin-Name: 563cde404cec4c6559ae0c5fc5edbc878fee874b36562ce2fac5049cc8349343
2023-10-06 00:06:26 +00:00
drh 27401d38d8 Better error detection when doing a lookup on a JSONB.
FossilOrigin-Name: 6e8e0eedbf06e175af6ef230f1ce9a5ad84698b1c6dcd2794d878c746e7b6be4
2023-10-05 23:05:35 +00:00
drh f7af8f3270 Improvements to comments. No changes to code.
FossilOrigin-Name: ac74d7877685006e43684edd6a2d22be8c9857f9f9eb52fc5b3c182d5e2fdb8d
2023-10-05 22:52:43 +00:00
drh dac2707152 Turn an unreachable branch into an assert().
FossilOrigin-Name: 0f75199160e48fa8c44f986f1af777adf19e40fbd114a6f58e24d5e6dede779d
2023-10-05 20:17:01 +00:00
dan 16381d062a Add the sqlite3changegroup_schema() API. To allow changegroups to handle differences in schema created by ALTER TABLE ADD COLUMN.
FossilOrigin-Name: 309deee2dd8dd07623fce79f6bb62d5279d140dd0be3b34bc42af20b0507726b
2023-10-05 19:09:23 +00:00
drh 51d507d432 Fix the parsing of non-standard "Infinity" and "NaN" values from text into
JSONB.

FossilOrigin-Name: df1fbbeb83a2b4a496c9de0d86c7c8c677f8fe3bc770da163dcc1d338a17b58b
2023-10-05 18:33:19 +00:00
drh 5b6cfb7906 Change the json_valid(X) routine to return true whenever X is a blob that
could plausibly be a valid JSONB.

FossilOrigin-Name: 425f0b85a6f8ad0604c4a5faa18efb90436fedb1fe2612a559147c62cff8b7e7
2023-10-05 18:09:12 +00:00
drh 14fce24a9b Fix the text-to-JSONB parser so that it handles some JSON5 floating point
literals correctly.

FossilOrigin-Name: 564edb3b6dd752e09e445e3a25c2f5394ceede2a2cdff251d6433dadc0b3644f
2023-10-05 17:52:39 +00:00
drh 0585d9f08f Slightly stricter testing for when a BLOB is valid JSONB: If the element
is a null, true, or false, its payload size must be zero.

FossilOrigin-Name: 487781be8a93d9b8d870efb9f25822bb4bbcf9d39d571d1017e9c29043ea515d
2023-10-05 16:33:00 +00:00
drh 01acfd5805 Fix a memory leak following a syntax error in jsonb().
FossilOrigin-Name: bf4b36eda8e200b67041ebdf60fc31d0325bd84185272f8d390c56a91d6ac1ee
2023-10-05 15:05:33 +00:00
drh 9061e22a05 Allow the PG-style syntax for the PATH operand on the right-hand side of
the ->> and -> operators.

FossilOrigin-Name: bae5071b0834aa3b7fd4bd871f863e1b148c6558989c8f6cdd02dc4da4770953
2023-10-05 15:02:00 +00:00
drh 0f921e2bac Merge trunk enhancements into the jsonb branch.
FossilOrigin-Name: be5907b648386e05530d1e321e2a3e563e07e99f08373aaf9a3452adc9228dc3
2023-10-05 11:22:16 +00:00
drh 5d9446d100 Improved detection of corrupt databases in balance_nonroot().
dbsqlfuzz d1cf013c50a620b68c2a5bd240d29afd65e5f58e.

FossilOrigin-Name: a2464bbb825b5976ef974a2e6c17ea150f5e6fcd0dd0f144b9f9c1c22a9c9c82
2023-10-05 11:06:10 +00:00
stephan a357bdbfc4 Do not squelch exceptions from SAHPool VFS importDb() - rethrow them. Problem reported in [forum:c80fc578809b80a3|forum post c80fc578809b80a3].
FossilOrigin-Name: 325bcdea4a436e514d83095d3893caf07a714b718e1b643507179b101c108e3b
2023-10-05 11:04:49 +00:00
drh 3a3912f21c Remove a stray blank line from the beginning of btree.c. No functional
code changes.

FossilOrigin-Name: c89983dcfc208faa86b31717144c069572598e9770228264d69a044a84bde277
2023-10-05 10:35:51 +00:00
dan 6d8e91be9d Allow a session object to generate a changeset, even if columns were added to one of the tables using ALTER TABLE ADD COLUMN while the changeset was being collected.
FossilOrigin-Name: a3f435eccf3a2aa11cb7420e94af5efcdfa04e9c169c5aaf61fa5cdcb165ceef
2023-10-04 21:15:24 +00:00
drh 10e751937c The Makefile distinguishes between tcl8.4 and tcl8.5. Some makefile targets
require tcl8.5, but others (ex: "sqlite3.c", "shell.c", and "sqlite3") require
only tcl8.4.

FossilOrigin-Name: 770308db9776b8c0a70b8807463e89a9eddfe5552e25e67324cd303dc974f50e
2023-10-04 12:49:08 +00:00
drh 57996f53e4 Modify the main makefile so that it fails with an error if the tclsh found
by ./configure is not tclsh 8.5 or later.
[forum:/forumpost/4380363682708ece|Forum post 4380363682708ece].

FossilOrigin-Name: 1765f3b5a00a8ca20a7b9e18ac7f9d7de0679470b234b83aea83aa5b4d4d34e6
2023-10-04 11:52:44 +00:00
drh 4500d87e76 Fix a memory leak in JSON group-aggregates when the output is JSONB.
FossilOrigin-Name: 08e7db138b636890cb29a76d65c2069c6e2ff44470fa4bca14f4526fe5f195ae
2023-10-03 22:40:22 +00:00
drh f362731c1c Fix a bug in the jsonReturnFromBlob() function that causes a positive result
for a negative value for when a JSON integer is too large and needs to be
converted into double.

FossilOrigin-Name: dca684da0c29ec78460362f972ea7747be42c13c4d1325da9d62c1ea58022e39
2023-10-03 21:54:09 +00:00
drh 51cc3041fe Fix problems in the "json" output column of the json_tree() virtual table
for the case when the input is JSONB.

FossilOrigin-Name: fefa4475c496aab32fe7b9cc203747ce4faf7448da4f29b807e5486c7686238d
2023-10-03 20:58:39 +00:00
drh 6e737b9252 Improved handling of OOM while translating the JsonNode representing into
the BLOB representation.

FossilOrigin-Name: ef5956710bb542d6045c82937d02218a7ed45af94cf3959b0c180268e04d14e1
2023-10-03 19:37:19 +00:00
drh f0c129f247 Merge the latest trunk enhancements into the jsonb branch.
FossilOrigin-Name: e6406a9865b75dea2f26d3ee4f4c206958400059c7f92ced88edc8507dd3c82f
2023-10-03 11:36:47 +00:00
drh ee0f6777f9 Fix a problem in the JSONB parser that comes up following an OOM.
FossilOrigin-Name: 355acfb18897254f6b6444a21d781b5e10e930b81952850dd2a40d88bbf2f3db
2023-10-03 10:59:01 +00:00
drh 8405e1140a Fix an issue with the use of jsonb_group_array() and jsonb_group_object()
when used by window functions.

FossilOrigin-Name: 808bd349ba587fbcdc4aea3f9c08e0df01ba08dec181c5af5ea157e89d86ff7b
2023-10-03 10:43:30 +00:00
drh 313336f90b New test cases for JSONB.
FossilOrigin-Name: 6d4aeff5751722c83ebb0e1f21072b72be418c64dcf28ee032d3a548e1b3a951
2023-10-02 23:56:46 +00:00
drh e5c384e9a5 Fix jsonb_insert() so that it does not behave like jsonb_set().
New test cases added.

FossilOrigin-Name: 54197149b811d30b6c4487eedf5692b164ed0f90cfcc541aa3157094f5f17f6a
2023-10-02 20:16:06 +00:00
stephan 4f65a3cec9 JNI: when fetching text-or-blob/length pairs, fetch the text-or-blob first, then the length, as the API docs suggest.
FossilOrigin-Name: e4f9d3946fa570bccea2de17c5536901bdabd46cfe25678fdd37ba0e2bfd99b1
2023-10-02 17:12:16 +00:00
drh 51e3f7a1c3 Fix the length of the string in the new concat() and concat_ws() functions.
FossilOrigin-Name: 325243226520376f4a142864ae3fb98866f25bff4dc50a6b648ef98428a7169f
2023-10-02 17:06:28 +00:00
drh 43d71eb892 Remove an ALWAYS() from a branch that can in fact be false following an OOM
fault.

FossilOrigin-Name: 3a1feb1c469d3dd50e2558581199b16bfd59b9d5bcb0de5f1a8f0ec30a17b4d1
2023-10-02 15:56:37 +00:00
drh dee29e8c7a New test cases for JSONB.
FossilOrigin-Name: d624c31e5c49e1ce63b4b72caa42a61c5167866f47d842fbcfe4e826fd079d7c
2023-10-02 14:51:28 +00:00
stephan 041fb16631 Pointer-casting-related fixes for JNI on 32-bit ARM (pi4 armv7l).
FossilOrigin-Name: 54441039ae3ebe39e42f008f77ce1be117cbda6daeed8b01739ad42a3c1f0754
2023-10-02 14:44:28 +00:00
drh c97407fdba Allow json_replace() to accept JSONB arguments as the new value.
FossilOrigin-Name: 95eb7b37fab29931924311f541d52173ef77a448efc8771b1a1783ccd786d23d
2023-10-02 13:35:06 +00:00
drh 8eeafb7544 Improvements to error handling for BLOB inputs on JSON.
FossilOrigin-Name: 14f20ecbfab44934e86f1ac7a3f745b989aa8190c6df119ff5aa8100fa248d93
2023-10-02 13:20:43 +00:00
drh 5624b0b4cd Gather forward references into a single place for the JSON code. Allow
JSONB arguments to json_array() and json_object() and similar.

FossilOrigin-Name: c352201b8c299c330d9abbff6dbcbcbcf00ada53183d3cd91020ec772e066357
2023-10-02 12:40:04 +00:00
drh 9b176f863b Fix problems following OOM in JSONB parsing.
FossilOrigin-Name: 0d8cd6b5fb592f88f593ceaad9cdfa95dcfdf14169c3a56a5becbd66d62fd02b
2023-10-01 18:59:34 +00:00
stephan dc13d74140 Add JNI binding for sqlite3_normalized_sql().
FossilOrigin-Name: 2532e5662a2bbb3e6b4c4be3fad035734b684bdf59b178cb64b444ca6d8b956b
2023-10-01 12:15:38 +00:00
stephan ec15e551f9 Make JNI binding of sqlite3_value_frombind() return boolean instead of int and add tests for it.
FossilOrigin-Name: 906e2ed3e9647f5c2355e9fea2beee141767cae7016da9fa1fe88283726b9369
2023-10-01 11:53:40 +00:00
stephan f046b82324 Fix JNI binding to compile without SQLITE_ENABLE_PREUPDATE_HOOK. Add build option to disable all optional ENABLE flags.
FossilOrigin-Name: c04022b7407f77eaf0175e831ebcd6bbdc0af1cef0d42c5c11102aa8484f24ca
2023-09-30 17:08:29 +00:00
drh d8f26350a0 Restore the trunk version of sqlite3_user_data(). Fix the xColumn for virtual
tables so that the sqlite3_context contains a valid but NULL user data pointer.

FossilOrigin-Name: 15ffd932fecc82a5791b2024f55d2ec80887e8eb7345de68d6f5cac4912cfbe8
2023-09-30 16:50:17 +00:00
stephan d1878bed86 Make all SQLite3Jni methods which take a void-pointer-as-long-int argument explicitly package-private, as they're always for internal use only even if SQLite3Jni is eventually made public.
FossilOrigin-Name: 5e387275f69ab2d3159b4b67b8cbfc6270410b61e5ac1f988616e8d051f6572e
2023-09-30 16:19:57 +00:00
stephan 41c6232e54 JNI: rename sqlite3_errmsg16() to sqlite3_errmsg(), as the 16 suffix is superfluous there.
FossilOrigin-Name: df25b21689de84b7b8c940df39461d3d88f592f0ba2dddcfcbb0bebcea6ee7da
2023-09-30 14:48:21 +00:00
stephan 0018ef15a3 Expose sqlite3_stmt_busy() to JNI.
FossilOrigin-Name: 5733aa5fe341d50b84baebd8803a86e381bb95959a8bc733b8fff6fd4e732867
2023-09-30 14:34:55 +00:00
drh 7e86d3fc69 Finish adding jsonb_ versions for all JSON routines that return JSON text.
FossilOrigin-Name: 6daa7b69695e1b68dba317abbcad4d0205c91963d4a9eb2d595a3ec10fa0fdf4
2023-09-30 14:34:39 +00:00
stephan 8f43696c8a Correct an undefined-value deref in the JS bindings, triggered by passing invalid args and then tripping over an undefined value while trying to report that. Reported in [forum:dd42df144651d779|forum post dd42df144651d779].
FossilOrigin-Name: fd4632540717c4771ac9ae1fe4400091a2cd0553639bf809abaf4ecbc54de4f0
2023-09-30 13:13:23 +00:00
stephan 9ac7e2d0fc Make (almost) all SQLite3Jni.sqlite3_... members public but make the SQLite3Jni class package-private, which has the same foot-protection effect but makes it much simpler to patch SQLite3Jni to publicize the interface later. Remove some of the extraneous foot-shooting protection from indidivual functions, in lieu moving it to the still-pending higher-level interfaces.
FossilOrigin-Name: 082b86683e77b2d4a30e1515552f52511324ef97c43b622b4dab7975ae9fbd7d
2023-09-30 10:59:18 +00:00
stephan 008b62f1ae Merge in and close jni-ptr-passing branch. The remaining pointer-passing adaptation can be done incrementally without affecting the interface.
FossilOrigin-Name: 7927c42410ba0d5fc8e6e08caaa5cbaaeb359869f17d1afaa082f1aa47fdb2fc
2023-09-30 10:37:36 +00:00
stephan d1ce2755f0 Make all native JNI sqlite3_...() bindings package-private as a foot-shooting protective measure (higher-level pre-native-call argument validation is largely pending). Move SQLTester.java and TesterFts5.java into the org.sqlite.jni package so that they can access the being-tested methods.
FossilOrigin-Name: ec82f7251acab7df40755ef5f456f36fe49b59e63a20be59bd610fc4280ba8cd
2023-09-30 10:31:56 +00:00
stephan ebce46f02f More work towards the new pointer-passing mechanism in JNI, and code-adjacent cleanups.
FossilOrigin-Name: 6c63987e893357dc8b10decaa96c30fb37b75481640a303e77a0d8224354491e
2023-09-30 09:41:58 +00:00
drh 5736ba5ac3 Update json_each() and json_tree() so that they work with JSONB inputs.
FossilOrigin-Name: bb5e50ff56dff95d954aacdd4c5461790f953cef8d7b89da000d8d587fcdf9b8
2023-09-29 22:37:18 +00:00
drh ee50aad865 The u.zJContent field of JsonNode for a string or label should NOT include
the quotation mark delimiters.  Ever.  This is an inefficiency that really
ought to be fixed on trunk, but that can wait until this branch lands.

FossilOrigin-Name: 96f545f6f839dab4829861361ee3d7a56840217c5f954f334616e77d23c5fe29
2023-09-29 19:47:25 +00:00
drh a99e2fb576 Additional refactoring and cleanup.
FossilOrigin-Name: 45dd1760875b1ad899a10189c6f5a0a9a0677903794fb5a06ffacd39952a7882
2023-09-29 16:37:22 +00:00
dan bee5657042 Add test case to ensure the assert() removed by [f090af77] no longer fails.
FossilOrigin-Name: 615ab71761754b072439f92e73fdb98c916b820937bcca8904f4be34936d5f9c
2023-09-29 15:56:40 +00:00
drh 8be48440b6 Remove an incorrect assert().
FossilOrigin-Name: f090af775f6d2ce8b50850f5c9e33546bcb9cff5467d99ca210530cb7ca83d5b
2023-09-29 15:53:55 +00:00
drh ae5f55e227 Improvements to comments and procedure names for clarity in the JSON
implementation.

FossilOrigin-Name: 9b620d813ef483f1277c1683c5e926a882f07f3b90804dea0c91b325ff8e45a4
2023-09-29 12:45:14 +00:00
drh ecce6022d5 Describe the JSONB encoding in a header comment to the json.c source file.
FossilOrigin-Name: 1c0cba3461d6111b3aeb77726880221f1240355f0b57e060febbdeb12fb688c0
2023-09-29 11:17:43 +00:00
stephan 83f62816f8 JNI: correct a NullPointerException triggered via SQLTester.
FossilOrigin-Name: 0a873de76c0cbcd8e2eda3f0508e427f1dcb32b01798687c0545acfe10102179
2023-09-28 20:34:28 +00:00
drh 8cdab14286 Fix some minor memory issues so that all legacy tests now pass.
FossilOrigin-Name: 1744bfc669346ff221f28d45fd978863e876a2d2f0b82bcf0e5ee6f0326900cc
2023-09-28 19:11:36 +00:00
drh 0a19bb078d Allow the sqlite3_user_data() function to be invoked with a NULL argument
or with an sqlite3_context pointer from a virtual table.  It returns NULL
in both cases.

FossilOrigin-Name: 2f49687371ada65fef374336c28b352c48ab98dc31282ac82397035efe04ba11
2023-09-28 18:23:52 +00:00
drh edbc44c5a1 Merge trunk changes into the jsonb branch for the compiler warning fixes
from two days ago.

FossilOrigin-Name: cee113cc315b04fd75ccc172cf4529bf15b2050bf274433496c31a282e281ab8
2023-09-28 17:41:45 +00:00
drh 77e00f24d7 The json_extract() function now works on BLOBs without the need to translate
into an array of JsonNode objects.  This is much faster.

FossilOrigin-Name: 5853065b7ee326c04ddfcde64c178f487af04cd3adc3cc99f559907484ec169d
2023-09-28 17:27:24 +00:00
drh 6b1db92228 Extract directly from BLOB is now complete and appears to work.
FossilOrigin-Name: 3de58ec99444b16dfcda1e226420e2343450b77abd3faf33a88b6d18339ef17c
2023-09-28 17:23:46 +00:00
drh 59862e6d22 Miscellaneous bugs fixed.
FossilOrigin-Name: 5c0815fa2e422d81198a43a2c04a022e319fcbcadfd4be4437f2e663892ca26b
2023-09-28 17:07:43 +00:00
drh 2dc60ec57f Improvements to json_extract() to better support JSONB. Still not 100% working.
FossilOrigin-Name: 8c82576176539c4d132b14d46adbf31366c4bcaa59a61dd639dc9cc308fe8825
2023-09-28 15:56:35 +00:00
larrybr d93cfa2831 Clarify serialization API doc w.r.t. buffer validity and usage, instigated by [forum:/forumpost/8922e2ad2ad41205|forum post 8922e2ad2ad41205]. No code changes.
FossilOrigin-Name: b488b9fb71652eca90d2bf73d32f3d748badf517859dc833c58e021b0e017194
2023-09-28 15:00:25 +00:00
stephan 0dabcd139d Adapted the JNI bindings for the sqlite3_value_...() and (pre)update hook APIs to the new pointer-passing mechanism.
FossilOrigin-Name: 4182f0275d5d65e04a130eeef4d44642a5ffeeb4b84430d240ea2605345f1404
2023-09-28 11:19:37 +00:00
stephan 5d1448d08a Convert the remaining macro-generated JNI bindings to the new pointer-passing mechanism.
FossilOrigin-Name: 250fd6ae806cf705c0f29ad30ad8fb885b12590848e7adae63bc21d874c6d3bd
2023-09-28 10:50:26 +00:00
stephan 50b2a41330 Convert the macro-generated JNI bindings which take a db pointer to the new pointer-passing mechanism.
FossilOrigin-Name: 5f47fb77db4ee77afc541e680559ad88e66ba7fd04b830e70f8be92cf8d0a60c
2023-09-28 10:27:01 +00:00
drh 1854837b5a Work toward getting json_extract() to operate directly on the BLOB, omitting
the translation into a JsonNode array.

FossilOrigin-Name: c1feba70f55a8e5f4696d48e4706855415d173ac8ac3c2656787c242a883b4f5
2023-09-28 10:20:56 +00:00
drh 733da8d31a Performance tests for JSONB added.
FossilOrigin-Name: 7c1be8e361db87458ac9d8fcee080c2b558936539c852bb80f0f7941d61bf15d
2023-09-27 16:55:13 +00:00
stephan 1d74432fef Adapted JNI sqlite3_blob_...() and sqlite3_close...() to the new pointer-passing mechanism.
FossilOrigin-Name: 0b22c8ef93e5ccd45316099fb8575e27620158b1992c0c70fe0348cfc10147f8
2023-09-27 14:41:49 +00:00
stephan a917ee7056 Adapt JNI sqlite3_bind_...() bindings to the new pointer-passing method and correct the mapping of sqlite3_bind_zeroblob64() to use zeroblob64() instead of zeroblob(). Related internal API renaming.
FossilOrigin-Name: 980d559fff6e55d1e2ef39f242a8a09313a936cfb141807db475bcceff924869
2023-09-27 11:01:32 +00:00
stephan 9afd67cce7 Add missing license headers to JNI annotation classes and extend the definition of @NotNull to include 0L as the equivalent of null to account for the current rework of how pointers are passed from Java to C.
FossilOrigin-Name: bccdfeb9efde20410bde545893fb98ce5c0c4d04a8e7797b868ba8994120e1db
2023-09-27 10:29:07 +00:00
stephan 4632f91403 Rework the JNI sqlite3_backup APIs to the new pointer-passing approach.
FossilOrigin-Name: 48aeb1e2cdeb4aec85c4f63a1f809215cd4b473791169e72b5ddf6d6bdc2f7b7
2023-09-27 09:58:36 +00:00
stephan 36f60e6c1a Start reworking JNI methods such that they pass void pointers from Java to C instead of passing their strongly-typed wrappers, as that is reportedly significantly faster than passing the wrapper objects to C and extracting the pointers there. There are still many, many functions left to rework for this.
FossilOrigin-Name: 66c814dd473731703ee00e1ff610acfdccf09d1b87dd2355795ed697d4ed5d3e
2023-09-26 21:37:52 +00:00
drh 61df55d0be Fix harmless compiler warnings in the testfixture testing utility.
FossilOrigin-Name: 5a39a02d2dfd9ed6171cd0bd434b2bc268d0ed8ead6e1a396d1603266d9493ef
2023-09-26 21:22:46 +00:00
stephan e81d229fb7 JNI: implement AutoCloseable for the sqlite3 and sqlite3_stmt classes, and adjust a few tests to use it. Override Object.finalize() for sqlite3 class but not sqlite3_stmt (where it triggers a JVM crash for as-yet-unknown reasons).
FossilOrigin-Name: b633d8bbdbd43464ee03058c273e5188117b9538ed5d5148a1a3851e4adf6410
2023-09-26 19:49:35 +00:00
drh 676bcf7457 Merge partial-index constant value fixes from trunk into the jsonb branch.
FossilOrigin-Name: 700bdbd7383f66a0da675c197204da4e7b6ed757155145ee98d572de32a5d0ae
2023-09-26 19:46:38 +00:00
drh 0e224d93ea Fix yet another assert() in the partial-index constant value optimization.
FossilOrigin-Name: f459d0806cf044fd07743e4c91d0a5a6ddf45b3b41004bde4278f190d99a4cf5
2023-09-26 19:38:24 +00:00
drh 42156fd90c Add in many jsonb_xxxx() interfaces. Still uses the internal JsonNode
representation for transformations and search, but it does at least conform
to the desired API design.  Largely untested.

FossilOrigin-Name: e6045b4e1bf3a8e33926fc12b3c039f5e1002eaecbe277ffa82b0ec271a29d17
2023-09-26 19:30:46 +00:00
dan 1c67749166 Remove a NEVER() from a condition in the partial-index constant value optimization that can sometimes be true.
FossilOrigin-Name: 63aea8c3e0605edcf49ad1e5ce1a56d2690dcb8c4857043997545daab46ca035
2023-09-26 19:26:47 +00:00
drh 219e7f7986 Merge all the latest trunk enhancements into jsonb branch.
FossilOrigin-Name: ac242c4d47ec36aab1c2fa5e65e7b595e686f49473b75bd63708d05c59ce3f0f
2023-09-26 15:13:04 +00:00
drh a6e7202677 Remove unreachable code from the partial-index constant value optimization
([8d4160910d651246]).

FossilOrigin-Name: 7ad38254c37153efa72291d09800693ca60894359548eda877d59defa8c70d49
2023-09-26 14:57:30 +00:00
stephan 6f737371c0 Remove some outdated JNI docs.
FossilOrigin-Name: 2ca5cbbb2a8ee94fe9f4888dea695a4429aefa17611fd9ebe59bf4866290ab0c
2023-09-26 09:46:04 +00:00
stephan efa8369214 Minor JNI doc updates. No code changes.
FossilOrigin-Name: 63687cc659f06a1f6aa9d4496ef01d9f0adbee6184b8e265fb8feec1ad30449b
2023-09-26 07:43:51 +00:00
dan 47b427423d In partial index scans, if the WHERE clause implies a constant value for a table column, replace occurences of that table column with the constant. This increases the likelihood of the partial index being a covering index.
FossilOrigin-Name: 8d4160910d6512469cb0a060b89c2509dde1c8e33d9358034d7fb39cd494eabe
2023-09-25 17:46:02 +00:00
dan 1b96f9167f Another very minor tweak to code on this branch to save cycles.
FossilOrigin-Name: 2c805dc68112db7f06aea56a731a33e046e876851bdda3e725ba96f6da1bf2ee
2023-09-25 17:21:53 +00:00
drh 09f23d2a36 When parsing JSON text into the BLOB format, only use node type JSONB_TEXTJ
for an unquoted object label if the object label contains escape sequences.

FossilOrigin-Name: a82ebbac3c542ec7f86d1e8414d7fd166db48450115ee3b26d12b5bb445f5896
2023-09-25 17:14:17 +00:00
dan cf0906575d Change the order of an if..else... in new code to make it slightly faster.
FossilOrigin-Name: f80dc64483a54a488e507da140836261ae1b864577cc95d8e869919d5a25e73c
2023-09-25 15:25:49 +00:00
drh cc11526e22 Enable -Wall for the compiler on --enable-debug builds.
FossilOrigin-Name: 63a7b521390001939909d43d908af78f7df7cb92272d2b74f038da918c2bef05
2023-09-25 14:20:07 +00:00
drh 5933581cf0 As a temporary measure, try to translate the BLOB JSON format into the
legacy node format for processing.

FossilOrigin-Name: 14f2e95a9e531ef0d3fa7f1249f23c073a50c31b2109eefc2f258cada635ac2f
2023-09-25 13:23:29 +00:00
dan 873849b81e Fix a bad interaction between LEFT JOIN, bloom filters and partial indexes.
FossilOrigin-Name: 5775f92b31f0539c66d37e50689bec17a82c0e857a781331c03e34bc18157087
2023-09-25 11:12:39 +00:00
larrybr 1da3215d65 Cause sqlite3_exper_new() to replicate UDFs and custom collations early enough to appear in virtual column expressions during schema copy. [forum:/forumpost/e030aa4b3a|forum post e030aa4b3a]
FossilOrigin-Name: 8fc2c45558a826a84e6f7e98564573da125a5e2d5c635a85bdeba2abe29fbfa1
2023-09-25 00:39:43 +00:00
larrybr 16096e6bd1 Make expert changes acceptable to C89.
FossilOrigin-Name: a4f03c22ec13d2f9769ac655ad77ccd906ab8c0ee32df37139bd998c234a7a0f
2023-09-24 22:51:56 +00:00
larrybr e5adfb1a5f Merge expert changes to accept UDFs and custom collations.
FossilOrigin-Name: 8ead1b5ed6693c0e3c30428033049c95a1fe636d1d1eff1734d0a896b5ff1f5a
2023-09-24 19:47:37 +00:00
larrybr 5918adb374 Add minimal exercise of expert acceptance of UDFs and custom collations.
FossilOrigin-Name: 0f6bbb79cbb52f5ddf38840898ab6789a4e108353918296df20b332117e6a148
2023-09-24 19:38:22 +00:00
larrybr 7ecef617ee Sync w/trunk prior to merge back.
FossilOrigin-Name: 2d290854cd9b6cef70b53c28745e1ccee5b5443378407bfc726e452e7e7a0a95
2023-09-24 19:28:07 +00:00
stephan a2ead23868 Resolve the JNI FTS5 test5() failure and remove some nearby dead code.
FossilOrigin-Name: 5655c51f103c6dfade799ffb5ca2eb21b4360ada598741f373c952ae732535aa
2023-09-24 19:13:31 +00:00
larrybr a25d478ea8 Give expert ability to deal with custom collations.
FossilOrigin-Name: ac1dc1b6ded0e43acb5b2b707110f767f7bdb5cecdb6b28a982db174ae233d37
2023-09-24 19:00:00 +00:00
dan af7cce98c5 Add extra tests for changes on this branch.
FossilOrigin-Name: cb45f5b070cfdbb4358e8e909b0b34d658b5925db06e282b38046298a159c545
2023-09-23 21:31:40 +00:00
dan 7a9bbfe1fa Avoid an error when parsing a schema that contains indexes with WHERE clauses containing unknown collation sequences.
FossilOrigin-Name: d5dd39e37ab994ce5d75688a40c6494c17807431529e85eb2b11d58f367fe232
2023-09-23 19:25:03 +00:00
dan c18c7523a4 Fix a failing assert() caused by changes on this branch.
FossilOrigin-Name: d0e21f20bd8643d21c4b88fb75b83939d68e9b7118f7289a164656f11d2ee555
2023-09-23 18:52:22 +00:00
drh 9f811a35b8 Fix a harmless compiler warning.
FossilOrigin-Name: 7c4210253b660b385d14de3ae7ab30f038036308e1164ec17b40e2805b9b2235
2023-09-23 18:49:30 +00:00
dan 2d3c5385bf Add extra tests for the fts5 xPhraseQuery API.
FossilOrigin-Name: b5edea16317ccc896394011f45442af44d5ea1041b6abfe152c90412c78bfc5b
2023-09-23 15:34:02 +00:00
stephan 6c248201b1 Resolve the JNI FTS5 test5() crash, caused by two unrelated typos in JNI callback signature strings. That test now fails with an expected-vs-got mismatch but no longer dies in the native code.
FossilOrigin-Name: 0233f310bf6bbb84f76b25205af160951e0bf6d8b2f1bca4537384af4fe47e58
2023-09-23 06:50:19 +00:00
stephan 8a67a46340 Remove an invalid optimization from JNI code and add a missing check check for Java-side exception.
FossilOrigin-Name: b8151f879718f7e30520f7e5ae04bd6724ae98af067be097b1a00df2175a038a
2023-09-23 06:37:18 +00:00
dan bd42642431 In partial index scans, if the WHERE clause implies a constant value for a table column, replace occurences of that table column with the constant. This increases the likelihood of the partial index being a covering index.
FossilOrigin-Name: 66ed7abdfa228abde2052e3988589371f0e49b11582b1b4a83255d2df3a0aefa
2023-09-22 20:21:27 +00:00
drh e367e453e1 Fix minor parse-to-BLOB bugs.
FossilOrigin-Name: 8b53b2e6600c324ff7864840d98a3f03896b9792fcb60b70cc1f6227b3bd4ca1
2023-09-22 16:20:48 +00:00
drh c1c1d4d4a6 Add the ability to render a binary BLOB back into valid canonical JSON.
FossilOrigin-Name: 0b70cb77a4c8e3f17932f1ecca3942e0b0b03de637fb9656a130fe045f7ef826
2023-09-22 14:33:39 +00:00
larrybr 163af02aca Give expert ability to deal with UDFs.
FossilOrigin-Name: 3406b05b4f57901f64f9b5fc83fe0250b827ae7f342c2026818dab0840aafd23
2023-09-22 14:20:45 +00:00
drh 90189be7ce Begin adding code to render binary JSON back into text. Very incomplete.
This is an incremental check-in.

FossilOrigin-Name: b817dd865ed60fc4da0b662a9edec0fceb8921b02ce98133bdd565988939fd0f
2023-09-22 12:16:56 +00:00
drh 8c55945220 Improvements to the JSON binary BLOB format design.
FossilOrigin-Name: 2c89ae5d02f6a40ef869e2a162e2c72871df60572b27959fd1d7171f495ce881
2023-09-22 11:20:35 +00:00
drh 769a8dea81 Reorganize the code to put the new JSONB routines together, for easier editing.
FossilOrigin-Name: dc23e783d4147d363856abe109586fc79a5b535b492beee0cf7a0234c0210667
2023-09-21 18:16:35 +00:00
drh 6831cad03b Initial development code for an experimental binary BLOB encoding for JSON.
FossilOrigin-Name: 8131b3c272f47db2618886046a9713285ce120cb87d721484ee7444273290681
2023-09-21 17:51:39 +00:00
dan 8aaf63c6ac Add extra tests for java Fts5ExtensionApi API.
FossilOrigin-Name: f9d62b853ce8bfbfdc9f137e984e7a1b51d70e88c38b136b4fad1e8ae6ee8913
2023-09-18 20:42:06 +00:00
dan 876f09b560 Extra tests for java Fts5ExtensionApi interface.
FossilOrigin-Name: c923893f3604b278277de1bb919ef713bf7a4296b7ff71451cfe19bc2ff03190
2023-09-18 18:12:03 +00:00
drh 38119058aa Remove a NEVER() in btree.c that could actually be reached, with the approriate
database corruption.

FossilOrigin-Name: d1fd00e0875f881cacdc581a75215ec1872a81969ac07b8ebe3aaa707a9a3640
2023-09-17 16:36:22 +00:00
dan f99a27fab9 Fix resolution of "rowid" and similar identifiers in queries that use nested joins.
FossilOrigin-Name: 37ec43d92bde13efc71fa57ff5e59c4a95b9054c298f844aefeb06d4a91ad0d4
2023-09-16 18:18:57 +00:00
dan 2599705105 Enable "OR IGNORE" support via SQLITE_VTAB_CONSTRAINT_SUPPORT for internal-content (not contentless or external-content) fts5 tables.
FossilOrigin-Name: c362bde4f4b8489947f080154d7fddcfd6e8e21d742a483c496fb7fbe59969d2
2023-09-16 17:11:44 +00:00
drh a91fe45339 Add a NEVER() to an unreachable branch.
FossilOrigin-Name: 6b6eb38979d68c06e382620c8813d6b67a3de02c4a7a029c84f924b9a2e380c6
2023-09-16 16:39:27 +00:00
drh 5bd96b2446 Fix minor code indentation inconsistencies. No function changes.
FossilOrigin-Name: 05107a0ae1456b746d3119af68d39532fad23a7eef88c09a9ef46ab7f8da4b9d
2023-09-16 16:23:20 +00:00
drh 5ddf0cbe25 Fix a harmless compiler warning.
FossilOrigin-Name: 9e9ff4ca3a205d0b599634782a5ad389fbe4823663550f48c54ca35dbc660dc7
2023-09-16 16:12:10 +00:00
dan 2e8edc1d45 Further tests and assert() statements for the change on this branch.
FossilOrigin-Name: 1c202d540ac362bfc747a9f8472e83c9d7614e38467f8b48787a669fb34664ba
2023-09-16 14:42:18 +00:00
dan 63702bccaa Fix resolution of unqualified "rowid" identifiers in queries with nested joins.
FossilOrigin-Name: bbcbd3d537d6790373d97f59386b8ce7fa2177db572af0f1babe058a76e25cc3
2023-09-15 20:57:05 +00:00
drh 124a6aa7d5 Simplifications and performance optimizations for the RTree extension.
FossilOrigin-Name: 04a333f5faf6b90592f1f69889ac6c28949955e186f39037cd639480b06feae8
2023-09-15 20:28:27 +00:00
drh ef2056e533 Drop support for the view-scan optimization (check-in [609fbb94b8f01d67]) as
it was causing multiple performance regressions.  In its place, reduce the
estimated row count for DISTINCT subsqueries by a factor of 8.

FossilOrigin-Name: f911f1c4977fbcae041243955cf2b98d8cc8baa337885a69be0f2b9bd2efa6f3
2023-09-15 19:51:18 +00:00
drh 58c10d9469 Minor simplification to the DISTINCT output row count change.
FossilOrigin-Name: 0738386d7c505a98768ddaff7ff1fc76f115882d92d6f84bf136b4aa2caad8b8
2023-09-15 19:27:36 +00:00
drh c09701db9b Do not reduce subquery output row count estimates due to DISTINCT until
after the decision of whether or not to use an index for ORDER BY has been
made.

FossilOrigin-Name: 27390051e86ad86fb35219329d359be9e83073f59782631af7fc519225e10565
2023-09-15 19:00:47 +00:00
dan 81b70d97eb Allow expressions like "<tbl>.rowid" to refer to implicit rowid columns of tables in nested FROM clauses.
FossilOrigin-Name: 59a1bbc69f5dbb33418fa4b383393fb13a46bc1e531577da8ad54ae2fad5a10e
2023-09-15 18:36:51 +00:00
drh 20220d4682 Experimental: Assume that a DISTINCT clause on a subquery reduces the number
of rows returned by that subquery by a factor of 8.

FossilOrigin-Name: 5a940e47d501f23347eaf084f4820e064e93665d19c8938c80715b71006ede8b
2023-09-15 16:15:33 +00:00
drh 658096e124 The view-scan optimization was added to enhance the performance of one specific
query, but it causes performance regressions on a host of others.  Disable it
(at least temporarily) in order to try to find an alternative way of fixing the
one specific query that it was created to fix - an alternative way that does
not cause performance problems for other unrelated queries.

FossilOrigin-Name: d9625a9eafe88859c3255849ee750cc6322d5a13ddad45107f9b2085c20eff50
2023-09-15 14:06:44 +00:00
drh 581b229362 Do not try to convert a double into an unsigned 64-bit integer, as that does
not work on all platforms.  A double can only be converted into a signed
64-bit integer.

FossilOrigin-Name: ce339046416e9ddeffe77f71a8bfb8c5b302160c3207f919aebd62902618ed59
2023-09-15 10:24:29 +00:00
dan c9823e3960 Tests for java versions of Fts5ExtensionApi.xSetAuxdata() and xGetAuxdata().
FossilOrigin-Name: 858fa236f8b673eb03cf2242e8202cd33ae1f13a43b59bdbb2eecdb21d42ef55
2023-09-14 20:56:42 +00:00
stephan a0f878959f JNI: move XTokenizeCallback interface out of the Fts5 object - that level of indirection is a holdover from when those classes were in the core package.
FossilOrigin-Name: 385ec08570451c0be212677d1b681c13d225dc690965effa2293e64cb3d56023
2023-09-14 20:02:49 +00:00
dan 93717c2306 Add a few more tests for Fts5ExtensionApi methods.
FossilOrigin-Name: 4ce2a7e8ed0f2859efc1038fd0812bc2b667be1479f45c081deccc2cf188c807
2023-09-14 20:01:18 +00:00
dan 9ba2d13ed3 Add a test for the java version of Fts5ExtensionApi.xRowid().
FossilOrigin-Name: 227facf879d2ade348cdb51f5b50ba2f95b8621acc0cf7a5fed4a310b4c76baa
2023-09-14 17:42:34 +00:00
drh e744f66008 According to [forum:/forumpost/9f6db917e1|forum post 9f6db917e1], older
MSVC compilers are unable to convert a double directly into an unsigned
long long int, but must first go through a signed long long int.  Work
around this by restricting the range of doubles that are converted into
long long integers so that only the lower 63 bits are used.

FossilOrigin-Name: 0ab05871ba36e2a0eddfbc700c1d39891c54c453727a4effcdac3327efec163b
2023-09-14 16:45:46 +00:00
drh e68decd54c Document the use of OPFLAG_ISNOOP as the P2 argument on OP_Delete. No
functional code changes.

FossilOrigin-Name: d88f41b2cf2b721170d2428a50b717976091389d9a07d0ffa15c3323b0eaad37
2023-09-14 16:02:56 +00:00
dan 82cdf4987d Avoid a segfault in test program showdb.c that could occur when analyzing a corrupt db.
FossilOrigin-Name: 6bf3e90687d48243544cc07baa01cb1b25ee0b77c59437f96c8da8e5d8b8095e
2023-09-14 14:05:51 +00:00
drh 60e53c57cf Improve the sqlite3IntFloatCompare() routine so that it is testable.
FossilOrigin-Name: bb221381fa38db5be1222d89c567a9bca0e6c441e26a8c797bd326b263556afb
2023-09-14 13:10:17 +00:00
drh dc03af67a4 Do not set the global bUseLongDouble flag based purely on sizeof(long double)
being more than 8.  That is a necessary but not sufficient condition to use
the long double routines.  Instead, do a run-time test to verify that long
double really does provide extra precision.

FossilOrigin-Name: aa999d490b743f455f2fb6c41009098330380f0bc2e9b1c28fbc82141a9138ce
2023-09-14 10:46:50 +00:00
drh 647e3ed2ba Reduce the number of calls to sqlite3_mprintf() made by RTree.
FossilOrigin-Name: f158b7d4917e0951fbb86a6f438abcb618d8602566fa54bf04c05a37f3a73513
2023-09-14 01:46:57 +00:00
stephan 595c3ecc94 JNI: enable the fts5-customization bits by default but elide them from the docs until it's clearer where they're headed. Unrelated minor doc tweaks.
FossilOrigin-Name: 21adb1aef24398fba918a5e1479ce2b6fd7e86d87eb9f2bea86a0eb43e8809c2
2023-09-13 21:32:20 +00:00
drh 23164c984e Improved comments on the hasHighPrecisionDouble() routine. No changes to
the underlying code.

FossilOrigin-Name: 810c635ce063d873e969bf83339c654f6008e84ce8a61f0ffc61806e98d13dde
2023-09-13 20:35:04 +00:00
drh 5b5d4492f2 Determine at start time whether or not the underlying hardware supports
high-precision long double computations.

FossilOrigin-Name: 9a854b919667e0e679a259542b2ee444ee416dbd73ecd9458f6ced35d9d3f264
2023-09-13 20:06:46 +00:00
drh 7de8ae22f7 Omit the Reinsert algorithm from RTree. This causes most benchmarks to run
faster, at the expense of having a slightly less dense and hence larger index
(example: 33 entries/node versus 34 entries/node).

FossilOrigin-Name: b3049a1d3dbdd63c471499c2f6b417655defe9ad90228e7cc722f5be877aae01
2023-09-13 17:30:12 +00:00
stephan 1eca330a08 Doc corrections for the previous checkin.
FossilOrigin-Name: 9ea0a9f39d03707d251af5af08d1ba94958704ba74019d7e8823ccd21936023c
2023-09-13 17:16:25 +00:00
stephan 181063d477 Add JNI sqlite3_prepare_multi(), based on feedback.
FossilOrigin-Name: fa1c1534724b03debc83ae35c2fadab83faf4b4e62b91981fed103888de41396
2023-09-13 17:11:32 +00:00
drh f9967cb302 Performance optimization to cellContains() in RTREE.
FossilOrigin-Name: 43cde22bf3f36687df231eddf642581d1d4f4102ad8568d31f5b2ff0302ca800
2023-09-13 14:07:07 +00:00
drh cdae00b3e8 In the ChooseLeaf algorithm of RTREE, do an initial pass through the cells
of a node looking for solutions that involve no cell enlargement.  Only look
for the minimum cell enlargement if the enlargement is non-zero for all cells.
This results in a performance improvement by reducing the number of calls
to cellUnion().

FossilOrigin-Name: 59f0e239d19393190911ea3bd89b5a24be0d85d64ccf51906efc9537ad5d7298
2023-09-13 13:12:08 +00:00
drh 386e359044 Reduce the number of calls to cellArea() in the ChooseLeaf() algorithm of
rtree.

FossilOrigin-Name: c633fe6be0edc07a0aa1ac819d58b6c629b7c95b47b63516ae1b0a937f6a2b89
2023-09-13 12:06:02 +00:00
drh f4ae37ca44 Remove unused variable from the ChooseLeaf algorithm in r-tree.
FossilOrigin-Name: fb2e4a8067edc29ba64973820f265b21dfee0d32f4d675d32100d5eef93c2830
2023-09-13 11:36:44 +00:00
dan fef1c11f92 Replace an if() condition in fts5 that is always true with an assert().
FossilOrigin-Name: 2170312c8d7f076cbb8319227de3fac981432dae186bc1928cd217e41119b580
2023-09-13 11:24:58 +00:00
larrybr 111815609c CLI .import to accept EOF in lieu of record terminator on last field of CSV (with multiple field records), per RFC 4180. [forum:5b21c25bdfa|forum post 5b21c25bdfa]
FossilOrigin-Name: 6f7842f577a28df1f809cd4bae9e8eafa26f2b54a25a1362ebbdebf5026be57c
2023-09-12 23:21:39 +00:00
dan 12439f9c16 Fix a use-after-free error in fts5 that could occur when querying the "rank" column immediately after another connection changes its definition. [forum:a2dd636330|forum post a2dd636330].
FossilOrigin-Name: cb54c2da52d3175844127ead41cfed18793e91ea52e02af2d57c1d17b4312437
2023-09-12 18:36:46 +00:00
stephan 0148013416 Clarify some metrics output in Tester1.java.
FossilOrigin-Name: 4ae7199a82f0b6e0f5575e39d8bd454e527ab949f0c59b773a44a078c65c5676
2023-09-12 17:49:25 +00:00
stephan 18e143d1d5 Re-add the fts5 decls to sqlite3-jni.h, removed in a rebuild without fts5 enabled. Need a better solution for this problem.
FossilOrigin-Name: 6f269790a09350f49103bb0925e2a78a1f9b7a8aad84408eabbb8d64ef79d551
2023-09-12 17:06:51 +00:00
stephan 09d72e4a6e Rename JNI sqlite3_errmsg() to sqlite3_errmsg16() for overall constency with the text()/text16() family of functions.
FossilOrigin-Name: c4a8d8a5711bd2abeaf1c31046784d1bccff4b98e96522250866d1ee8ec6a9fe
2023-09-12 17:03:40 +00:00
stephan 1110f8ffcd JNI readme fix.
FossilOrigin-Name: 7dde000efcf170c33701947d1aab71bb45b2b0ae8646ee41bb2b2adbee56d1e6
2023-09-12 16:33:57 +00:00
drh 50925d23ab 64-bit builds on Windows default to using UTF-8 output (as if the -utf8
command-line option had been specified.)  32-bit builds continue to use
Windows code pages.  There is also a new -no-utf8 command-line option to
force the use of legacy code pages.

FossilOrigin-Name: d8c7560e8822f336e8c4b300eb1f2e55203fdf9e756d2e8af719563235f79ebc
2023-09-12 14:36:07 +00:00
drh 09e96ea731 Show the "(utf8)" tag on the CLI banner in Windows if in UTF8 mode.
FossilOrigin-Name: 822699a4d9756ec68f17d6e06da7f994730bcf5c7d69ca15e9131721c4ff3b19
2023-09-12 14:09:45 +00:00
drh ceeb9eeaac Add the -no-utf8 option to the Windows CLI to cause UTF-8 mode to be omitted.
FossilOrigin-Name: 4cb799c690986b8bd38d07461998824fa53418f7fe31f59d0bf38cae328d9b89
2023-09-11 20:09:05 +00:00
drh 17ca53da26 Make the -utf8 option the default behavior in the CLI on Windows for 64-bit
builds.

FossilOrigin-Name: 8b3c306def81546c5987f0f245d2bf3960218254c2833e9c27cd242e9118db62
2023-09-11 20:02:38 +00:00
drh 833ed7ee64 Refuse to build amalgamation-tarball and snapshot-tarball if the TEA version
number disagrees with ./VERSION.

FossilOrigin-Name: 3308fdda4b81c110ba4a66d0b325e7653c2f8155e7864aeb78991ed1da061836
2023-09-11 15:27:27 +00:00
stephan 5c3a25b450 Merge in JNI's significant divergence since it adopted sqlite3_client_data().
FossilOrigin-Name: e8e07dbbeaa7799eb0b90726e18e994aaf8c8d316ab4bdb06da732bc51cf0919
2023-09-11 15:20:15 +00:00
drh a9d788f08f Add support for the sqlite3_get_clientdata() and sqlite3_set_clientdata()
interfaces, to better support JNI.

FossilOrigin-Name: 9806c0dd2802d68b67c25c4f3347ed532f9a98b051e775d34e9182dd2f099891
2023-09-11 14:55:05 +00:00
stephan 7fa8d65539 Roll back [84e38341aeab] because a direct pointer comparison is ill-advised when the client-data string is dynamically allocated.
FossilOrigin-Name: 7b884832b71c23e62ba3c0d53f7c89199734c351f909d84ac19ac18c7ddccbd8
2023-09-10 10:56:28 +00:00
dan ef2e43304e Remove out-of-date comment regarding use of Parse.pConstExpr.
FossilOrigin-Name: 71548f72ad578db3d6ee409956386ace23ab2258d37cdc29ef2fb126e48f1ee3
2023-09-09 17:53:55 +00:00
stephan 8f825a7a1b In the JNI build, emit a reminder to not check in the javac-generated sqlite3-jni.h when FTS5 is disabled because changes in that feature flag result in unnecessary and large diffs in checked-in generated code.
FossilOrigin-Name: caceb52fb58f924e7cbbebac05e7fd7822b7f6f56a3f834392d2be801bf04633
2023-09-09 11:20:35 +00:00
stephan 8ea10ea520 Disable FTS5 in the JNI build by default until it can be thoroughly tested. This is mainly so that the public-facing javadoc does not include it.
FossilOrigin-Name: 6f25f02b70f561864264a71a85cb5efa05550743371fd655804033ea08d00f33
2023-09-09 07:52:05 +00:00
dan 1b2d170a8f Add extra tests for [4b60a1c3].
FossilOrigin-Name: 6357491bd1b97f7abddf1a83860f3763cd870033bddc74bd9994a6440fe8c762
2023-09-08 19:18:02 +00:00
dan 410acf6944 Fix a problem with fts5 secure-delete mode causing fts5 to corrupt its records.
FossilOrigin-Name: 4b60a1c358c966f161601e749ccb0a7c5f4126e20350f4d6c83b6f35841fbe0a
2023-09-08 18:34:42 +00:00
dan fb77b3ed3f Add -DSQLITE_OMIT_LOOKASIDE=1 to the "Sanitize" configuration in testrunner_data.tcl.
FossilOrigin-Name: a917ebf17b27951d1c6906e9068e816ef29861701f2e8f4f1a13acce5737f680
2023-09-07 17:54:12 +00:00
drh 8525e2d931 Enhance the ./configure script and its associated Makefile.in so that the
--with-linenoise=DIR argument cause the linenoise command-line editing
library located in directory DIR to be linked with the sqlite3 CLI.

FossilOrigin-Name: 6c7822b554c01f01e3e02a385b3370313022b16ba8d932c2ba5d3c3be61361fe
2023-09-07 16:53:30 +00:00
dan 09f1652f36 Fix a buffer overread in the sessions extension that could occur when processing a corrupt changeset.
FossilOrigin-Name: 0e4e7a05c4204b47a324d67e18e76d2a98e26b2723d19d5c655ec9fd2e41f4b7
2023-09-07 13:53:09 +00:00
drh 0a4c31d87b Fix a harmless compiler warning in the sqldiff.c utility.
FossilOrigin-Name: e6390a656713b855258277b65066d2a701cedd05981f1672c9acc2d6dc37d032
2023-09-07 13:48:42 +00:00
drh 6e2b63893c Add the "tool-zip" makefile target with the intention of using it to build
the "sqlite-tool-*.zip" deliverables.
deliverables.

FossilOrigin-Name: 06e4af492d669bb45c00b1b876cd82ec8813ac9ed580eb1dddd18147f7fbe7f6
2023-09-07 13:43:46 +00:00
drh dd69129f2a Fix a problem with the new xIntegrity method for virtual tables, and also fix
a bad assert() in FTS3 that was found by the new xIntegrity method.

FossilOrigin-Name: 52bbf44f2d9addc2b5f68b0fe33542470852310ce3a283e2c7ff4c52831d0ed1
2023-09-07 02:13:01 +00:00
drh 93b461ff00 Add documentation to sqlite3_get/set_clientdata() to make it clear that these
are security-sensitive interfaces that should not be exposed to potential
attackers.

FossilOrigin-Name: 2837061d66672a0a73c3fd2ff35ce0e1a66ee6d5365be25d4cc0ccba170c3665
2023-09-06 17:07:53 +00:00
drh d4bb0896d4 Add the xIntegrity method to the sqlite3_module object, thus enabling
PRAGMA integrity_check to operate on virtual tables.  Make use of this
new method in the FTS3/4, FTS5, and RTREE virtual tables.

FossilOrigin-Name: 2f08e7d233b6ddc28b61f0d17ae9b815a91c3c6e080862ce009dc0ad73c64cf5
2023-09-06 16:51:13 +00:00
drh 9e5d21092d Fix unreachable branches.
FossilOrigin-Name: f00fc16394427f25249dd58c25a9115fef1ae389e232d574b4e91db2e7f8c04d
2023-09-06 16:22:48 +00:00
drh d5ab4dd9e4 Change the xIntegrity virtual table method signature so that it returns
an integer error code and writes the error message into a parameter.

FossilOrigin-Name: f1d4024a8ca06cf954aaf1f612684d1a5d28492bde757695db3f22c50c649709
2023-09-06 14:00:01 +00:00
drh 961c2a9f36 Add the xIntegrity method to the sqlite3_module object. Implement this
method in RTREE, FTS3/4, and FTS5 so that "PRAGMA integrity_check" also
verifies the correctness of shadow tables associated with those virtual
tables.

FossilOrigin-Name: 17bede8cdefd968210dd8a5a2617acbe12ba2c99fdd5e88c5def8665e7bec2d7
2023-09-06 12:52:00 +00:00
stephan 7c1588b3ad Move Java-side FTS5 bits into the fts5 subpackage to (A) make it easy to optionally bundle it (or not) and (B) set a precedent for puting extension APIs in their own package.
FossilOrigin-Name: 4c773e1f331cd5854d4c6e8825197933c0bb6827b7437e56ac0ad650d1ddf55e
2023-09-06 07:39:25 +00:00
stephan f31b9fcf86 Java FTS5 API renaming, for consistency.
FossilOrigin-Name: 5e4cc9f386364858c6457c40f13f01e1826a59086aa7f0fa10244ea846f5acfd
2023-09-06 06:54:25 +00:00
drh 5a05a68315 Fix a few SQLITE_MISUSE returns so that they call sqlite3MisuseError().
FossilOrigin-Name: 93f74490faf8cc07e107afdab6737c6e5141ae1f01a05142bfcede2dd1b2ba4e
2023-09-05 15:03:23 +00:00
dan dd24b1c8a1 Add tests for, and source code comments to, fts5. No changes to code.
FossilOrigin-Name: b12afff4efe4de84388d102060e8c312abd2f9eca8270b3c0f01ac6d1f2e329a
2023-09-04 16:48:31 +00:00
drh 25a6e6ec8b Do not make assumptions about the byteorder of PowerPC processors.
FossilOrigin-Name: 4a2498fed4c5436fbcd4179db85e2741fdab37d42b0eebf12f41ec4573ce2c61
2023-09-04 12:50:17 +00:00
stephan 0c072b698b In JNI routines which set an OutputPointer.value, always do so, even on error, for consistency.
FossilOrigin-Name: d0df62f0af19a69f7ceb3ad1dceda0d8cc2ca5016a6e2a3a1e36356424c3ca49
2023-09-04 06:15:37 +00:00
stephan 3aeb58c90c Remove several unnececessary functions.
FossilOrigin-Name: 84433c72b0b08e7906ef87758caffe98f95df812945a7aea675bbc20791e33c0
2023-09-04 05:58:13 +00:00
stephan 4eaf5883b3 Move an internal utility class out of the public Java API.
FossilOrigin-Name: 8e1ae3197a62796565b7260b230057bdb3f136edd00a83f8a63532d44d386243
2023-09-04 05:34:30 +00:00
stephan 411d9e7b2e Expose sqlite3_bind_parameter_name() to JNI. Extend the definition of null for the @NotNull annotation, to consider closed/finalized Java-side handles wrapping C-side resources as null for its purposes.
FossilOrigin-Name: fc34e2bfa4a2255f019a2514367e58596837289de49f7fab87279c8b40c7f119
2023-09-04 04:23:31 +00:00
stephan 14412e9526 Add the missing sqlite3_blob_read() to [9a9945c405cf]. Elaborate on the definition of the @NotNull annotation.
FossilOrigin-Name: 85952b4a52a09e9da35754c1a1ec70b1bf277296011a2d37ac536b33437223d7
2023-09-03 14:41:13 +00:00
stephan 805e8c96b9 Expose the sqlite3_blob_...() APIs to JNI.
FossilOrigin-Name: 9a9945c405cfe9b6bf399655f498457da66b06c8b92014510ab9fb83fbb1c7d4
2023-09-03 14:13:29 +00:00
stephan 7dfacc9f96 Expose sqlite3_system_errno() to JNI.
FossilOrigin-Name: bac54da29c19ef5e0a76599b5d10b377ecd7b391a651af99526529df16a83c7d
2023-09-03 12:46:09 +00:00
stephan 286f30f85c Expose sqlite3_stmt_status() to JNI.
FossilOrigin-Name: d266acc23ecb7e76c8c68c6e89a76e6f3054f33f0262e60e06b258db5a5e2ccd
2023-09-03 12:32:09 +00:00
stephan 1943356268 Expose sqlite3_randomness() to JNI.
FossilOrigin-Name: de9692242132b8f2c92ef4acb08dd3063327b18666cbb17c4f1153dee9146eaf
2023-09-03 12:17:34 +00:00
stephan 83a8b6dd82 Expose sqlite3_get/set_auxdata() to JNI.
FossilOrigin-Name: 0de3271717e0298070097d7ea0ecb996d2e95cf65384e494515a554d0a1dffed
2023-09-03 11:58:33 +00:00
stephan e30436b605 Expose sqlite3_get_autocommit() to JNI.
FossilOrigin-Name: 67391c6588c71af411767d1e4b63c5041a8a9e669102da318e3482ccfc9d9bb9
2023-09-03 11:16:48 +00:00
stephan ce9902f8c2 Remove some unnecessary code.
FossilOrigin-Name: c7f9bbbbf0252c9749450e3ec994bf5ac945bd62428657ddff469291a0fd4b0d
2023-09-03 11:08:31 +00:00
stephan 54021c3bd0 Add a couple more sqlite3_backup tests.
FossilOrigin-Name: 5c2cf89d2c3ae5fc31b5e311bc2f2c1fef0037bdb0dff48c0a2a6bb6610e0215
2023-09-03 10:36:21 +00:00
stephan 24a689cf01 Export the sqlite3_backup_...() APIs to JNI.
FossilOrigin-Name: 865a3df5d550f6f4edbf95854750706e4e747d8f1825ae73d300b8e31934959b
2023-09-03 10:23:03 +00:00
stephan 2966b85df3 Export the sqlite3_keyword_...() family of functions to JNI.
FossilOrigin-Name: bd554db39159d8a538ce913d55285e3c417db8823c717a3e34bfa0678de42be7
2023-09-03 09:28:45 +00:00
drh d3c5471a9a Performance optimization to the OP_MakeRecord opcode.
FossilOrigin-Name: 2aef9af3dd493a82ba090ccb8ab1b8974349392ec3f1c35db7a3dbbec41b5c51
2023-09-02 19:35:15 +00:00
stephan de4d1c357f Add sqlite3_config(SQLITE_CONFIG_LOG) support to Java and correct clearing of SQLITE_CONFIG_SQLLOG. Correct missing names in propagated UDF exception messages.
FossilOrigin-Name: 652e2e82100ac3d8238d15ef6d43de4162fd02d695db4936ad5706fe4f2d7dc3
2023-09-02 14:36:41 +00:00
stephan 7d31de600b Expose sqlite3_complete() to JNI. Slightly widen the definition of "canonical" for the purposes of the @Canonical annotation. Add a few new tests.
FossilOrigin-Name: a59fe0cd9552128ca4186f4e66f0d188f2a57c7647f43828e42cb95801f350ce
2023-09-02 12:53:25 +00:00
stephan 216433636d Duplicate code consolidation.
FossilOrigin-Name: a1872ae959cca11d5d8bc224aaeb0b53e772d999df9924b2e9ea84ceedb99148
2023-09-02 11:26:36 +00:00
stephan 70b58fc119 Remove a superfluous level of indirection in the JNI internals.
FossilOrigin-Name: 8dca6f7660c15eacbda20da1c66c9ef1de36864f78750658226b1a7baf22b726
2023-09-02 10:18:10 +00:00
stephan 63f4a9c4db Further JNI cleanups.
FossilOrigin-Name: 30e38173c3ece0c9f8e7a9710f46cb5e8e8ef101c04531318a7adb070242f5dd
2023-09-02 08:51:14 +00:00
stephan 21d4f0cc7b Optimize sqlite3_get/set_clientdata() for the presumably common case of a static string pointer.
FossilOrigin-Name: 84e38341aeab4fd51690e8536d0101d00e3fb11cc50ebcff05ed219c98328872
2023-09-02 08:04:56 +00:00
stephan 49a40ef6d2 Add a timer to the JS SQLTester app to warn if it appears that loading the module has failed, which is commonly caused by the test scripts not being compiled.
FossilOrigin-Name: f9c1f9cad1ad22f689a4afa13d59bf9969ddaed6624cbc16cb1bf5d1fd0c8a5b
2023-09-01 17:51:09 +00:00
stephan a5f425cc6e Correct a Tester1.java test which inadvertently created a spurious db file in the current dir. Unrelated minor cleanups.
FossilOrigin-Name: 016bbdadbf9f95cef185d2606c23e3b3c724586fcca470e11e6fbdbaefd75c79
2023-09-01 16:28:46 +00:00
stephan 4ea63e5609 Make the JNI-internal metrics opt-in rather than opt-out so client builds won't have them by default. Unrelated doc tweaks.
FossilOrigin-Name: aa72d25a22c80a7a376127266b8923f0661147c8bc483baa327230364851a255
2023-09-01 14:20:39 +00:00
stephan d086065644 Make JNI interfaces of sqlite3_column/value_text() match the C ones better. Internal JNI cleanups and simplifications.
FossilOrigin-Name: eb24f97253d9c8e7c728c2ac67a0824b10ca67362ac4f8abb94d5d3c54c58c5b
2023-09-01 13:27:59 +00:00
stephan dba0ceea60 Second half of [1c532e80].
FossilOrigin-Name: 8ca1d815a57be1fade59fb8ea5705c27b10294e4959c8a9c624f1623df6a5f63
2023-09-01 11:10:09 +00:00
stephan f551aa4e80 In the OPFS VFS' importDb() methods, overwrite the header bytes 18 and 19 with 1 instead of 0. Both seem to work, but 1 is correct.
FossilOrigin-Name: 1c532e807bf8466b67d9600ab9630a6736de77259e9a71ac435641715ee0e04b
2023-09-01 11:05:22 +00:00
stephan 7862e19468 Remove some newly-dead code.
FossilOrigin-Name: dc0fa76b395c5e352116dc33cc1b455e5e9c9f01c285af52ccba4d37e4453b87
2023-09-01 10:37:15 +00:00
stephan 03ca59afb9 Expose sqlite3_limit() to JNI and remove the all SQLITE_ macros from the Java interface which are not directly exposed to client-level C code (SQLITE_MAX_... and SQLITE_THREADSAFE_...).
FossilOrigin-Name: 5fe5b0a585dbfa06cfca1fea40aea5ea3ccfe8295f41306ad2d5b851ace27203
2023-09-01 10:32:31 +00:00
stephan 55171a75ef JNI build improvements. Rename a Java class. Get the jar target working again.
FossilOrigin-Name: d086b7844cace5c997261c97565aeef62aaeeef727ccc7e83f17c54d6217b779
2023-09-01 09:43:56 +00:00
stephan a4b47b034c Expose sqlite3_stmt_explain(), sqlite3_stmt_isexplain(), and sqlite3_stmt_readonly() to JNI.
FossilOrigin-Name: 208b786afe16eafaf0ce791f319a5e05f733a7b71ce1c542e1b83481b013ec38
2023-09-01 06:50:17 +00:00
stephan 0d3f0a9c11 Export sqlite3_txn_state() to JNI.
FossilOrigin-Name: d732f71d0a292dbb493f79f7c6ecd8a4effbfbc91453b1c54bdd9becf2d75bdb
2023-08-31 19:35:59 +00:00
stephan 63e7ed3b44 Minor JNI cleanups.
FossilOrigin-Name: 2e2bb841ef8b53266ec48d8c3408d6396f29f17922e858eac1b4f12f4adeb05e
2023-08-31 18:34:43 +00:00
dan 8628c10e27 Do not disable unused columns in a UNION ALL sub-query if any component of the sub-query is DISTINCT.
FossilOrigin-Name: c84d5602ac9bfb4f12c3cf62033af616e51913c26877d1a0761363a625295f53
2023-08-31 18:00:10 +00:00
stephan 062bafb851 Add convenience overloads of JNI sqlite3_table_column_metadata() to simplify usage.
FossilOrigin-Name: faf4e6d398f444d970be8b1364c984c67ccbc5f8a27f3aa93f05b8a8de7bec9b
2023-08-31 15:24:46 +00:00
stephan 0c2ba994d2 Export sqlite3_(db_)free_memory() and sqlite3_table_column_metadata() to JNI. Further internals renaming for consistency and legibility.
FossilOrigin-Name: 7c86aa3400ed591d38c1828f366f4b5de97954c2b301919d3f06d9c2d3d7d1f2
2023-08-31 14:57:01 +00:00
drh 3823208d5b Use mutexes to make sqlite3_set_clientdata() and sqlite3_get_clientdata()
threadsafe.

FossilOrigin-Name: 443ea20ddb0f3bf5d77ef59cd4678f0e32d7da328002bb44d6fc080a53a37e29
2023-08-30 18:51:26 +00:00
stephan 7fc4223411 Replace JNI's per-db-instance state tracking with the new sqlite3_set/get_clientdata().
FossilOrigin-Name: e7c11d34ee2eebdca4d9db1496bbb4152e4c62745c083ad5e0337733e8d1254e
2023-08-30 18:45:42 +00:00
drh a7ec1f9a1b Fix a bug in json_array_length() introduced in version 3.43.0 by
check-in [df099ad713011b67].  If the JSON input comes from json_remove(),
the removed array entries are still counted as part of the array length.

FossilOrigin-Name: 69a635958a4a14187e88dd297df8978a4805b1b0c7bff3ec29d5632c0f68d889
2023-08-30 18:19:40 +00:00
drh d25d938877 Remove another unreachable branch in the sqlite3_set_clientdata() logic.
FossilOrigin-Name: 57e31e6b10b2aa68b6039914d191819bae0aa60999a99d5ef23ca18128b72e0e
2023-08-30 17:41:55 +00:00
drh a5af4a6614 Remove an unreachable branch, and improve documentation of
sqlite3_set_clientdata().

FossilOrigin-Name: 565c68adee64b6c838f2fcb162485b6cfe9a976adb79de4370be98fcd4af2baf
2023-08-30 17:14:12 +00:00
drh 7bb5a6db40 Fix the AVG() and TOTAL() functions (after the SUM() fix of [77d3dcd283595c52])
so that they work with infinitites.  Fixes a bug introduced by check-in.
[c63e26e705f5e967].  Bug reported by
[forum:/forumpost/8960fb40cc|forum post 8960fb40cc].

FossilOrigin-Name: 6df6f17ccb404c648076ccff4200d0eb5437f0e3e82424bf3da5ea682b107bb4
2023-08-30 16:03:27 +00:00
stephan 85a05d895a Further tweaking in both SQLTesters.
FossilOrigin-Name: 46b0eb98e7a56cc188836c46f4289de6281f80e6bc6eb33e0aa9133a0f86f00c
2023-08-30 15:51:42 +00:00
drh adf54de51e Fix to the sqlite3_get_clientdata() interface.
FossilOrigin-Name: 937cd1762b36d48d9e7bafe42e99e524c9b5528d39f64521bce2f81d9c887db8
2023-08-30 15:42:41 +00:00
drh 10deb35995 New experimental API for attaching client data to a database connection.
FossilOrigin-Name: d542837fdb42ebe810fc99225860d2cc7e6dd829a635bde820a09beff6bcb481
2023-08-30 15:20:15 +00:00
stephan 8dd07389ac When a JS SQLTester script throws, report the exception details back to the UI regardless of whether it's fatal.
FossilOrigin-Name: 273d3b05f630d399d42914e95c416b107b4746bbef129cfba9d00fd921666261
2023-08-30 14:20:02 +00:00
stephan 24c32c2e39 Add a JS implementation of Java's SQLTester.
FossilOrigin-Name: b530792a514d95c4e8f93cf2170d9fc4de367055fa1704fc171551c946024fa9
2023-08-30 13:07:35 +00:00
stephan e621556724 Add a UI, of sorts, to the JS SQLTester.
FossilOrigin-Name: 249e82b9917ea47c56ee1cbd3345a977d335fd3fc0d67a1ef157813ef4571c7c
2023-08-30 11:54:43 +00:00
stephan ac5e1f82ce Add a mechanism with which to import external SQLTester scripts into the JS testing tool.
FossilOrigin-Name: bb08ba020ce1d86ca6aa92f43d5ae915f67d08fa73120e1f603d150e76166624
2023-08-30 00:22:54 +00:00
stephan 4f1387e9ab Add directives support to JS SQLTester comparable to the Java impl. This brings the two to feature parity.
FossilOrigin-Name: 7cef4a8300826adbdcb3b205e134a4272b12b4aa7dbee97731ac12282a4a9f06
2023-08-29 21:30:37 +00:00
stephan 267c44771f More fleshing out of JS SQLTester.
FossilOrigin-Name: 8c503dfb9fa15389613a819fcc1792e23d3c05f99a9f450f82eac5125298726f
2023-08-29 20:44:40 +00:00
stephan aa15047796 JS SQLTestRunner can now run the Java impl's core-most sanity tests, missing only support for directives.
FossilOrigin-Name: 5e798369375ce1b0c9cdf831f835d931fbd562ff7b4db09a06d1bdca2ac1b975
2023-08-29 20:01:01 +00:00
drh a910f3f1c5 New conversion letters added to strftime(): %e, %k, %I, %l, %R, %P, %p, %T, %u.
FossilOrigin-Name: 058722b2d0b995195a8ce3effe9722ae1c18cb1e7a520b481030da0bd579fe41
2023-08-29 18:28:08 +00:00
stephan 69a55ca17d Get the JS SQLTester command handlers in place sans those which have to run SQL.
FossilOrigin-Name: d21b1217964a53f33b7ba3958b34aa8560dff8ede33e66f54aa0afbab7099ec3
2023-08-29 15:39:57 +00:00
drh 9d6acd960f Change the version number to 3.44.0 to begin the next development
cycle.  (Should have done this a few check-ins ago.)

FossilOrigin-Name: 253fe4a45decdcc4180c5aeccaf4cd0d1325a0f6df0913b0e6721ea3e26b4fe2
2023-08-29 15:38:54 +00:00
drh e1e67abc5c Add support for the CONCAT() and CONCAT_WS() SQL functions, modeled after
the PostgreSQL behavior.

FossilOrigin-Name: 0b434ca7aa19eff4ad134a8c6f88f6a7ccab88864faa55e93579a2462d8ac3bc
2023-08-29 15:24:41 +00:00
stephan 0fc20a32c0 Get the basic parsing pieces and command dispatching in place in the JS SQLTester.
FossilOrigin-Name: 8fcc2a553c1e26734902bbdee0c38183ee22b7b5c75f07405529bb79db34145a
2023-08-29 13:28:36 +00:00
stephan 524ddc940d Init bits of a port of Java's SQLTester to JS. Far from complete.
FossilOrigin-Name: 60eec5ceda80c64870713df8e9aeabeef933c007f2010792225a07d5ef36baef
2023-08-29 11:22:45 +00:00
dan b44b802378 Change a variable from "int" to "i64" to make it easier to prove that it cannot overflow.
FossilOrigin-Name: 00a8b3a263f3537588063ce42fad6e21fa343dad850b086d0929ed1617eb44fc
2023-08-29 10:50:11 +00:00
stephan d10ed826eb Remove some dead code. Add a skeleton SQLTester script for fts5.
FossilOrigin-Name: 6c83e31fa96f65b61377c0c801cc32b3c8ca27a0c8442f860364bec258c003cb
2023-08-29 00:10:31 +00:00
stephan 4e379078e7 Factor out a superfluous struct member.
FossilOrigin-Name: 76d3911c370b3dc02d119045003f87ad20a6efd9f7a44d4feb59b7c801ac8981
2023-08-28 23:18:19 +00:00
stephan 924c4545d3 Further minor internal JNI simplifications.
FossilOrigin-Name: 1808d12ee0d1f1e5ee49d48c699ca10c4f822989ac9b4ac08f2b861513ee5997
2023-08-28 22:52:04 +00:00
stephan fa23b4fc61 Improve threadability of the JNI collation-related bindings and add infrastructure for similar cases.
FossilOrigin-Name: f02dad66b965b9e3c504001e9603af8f74977f151bede9db369f88e86a4aeb00
2023-08-28 21:27:32 +00:00
stephan a08f737503 Correct JNI mapping of collations to be 1-db-to-many-collations.
FossilOrigin-Name: b927b0f5a68684b0a9799396d153bf1e2306351e8039c2bacb3d5b2056a0634f
2023-08-28 20:21:56 +00:00
dan 35c7f538e9 Updates to testrunner.tcl so that it runs fuzztest using multiple jobs.
FossilOrigin-Name: ceeabe9f8b31a30c65147fd270b92d43c7842247548cee9de165113991f6c2cf
2023-08-28 20:14:19 +00:00
dan ecdc5b3e6f Fix the "sdevtest" testrunner.tcl command so that it correctly enables asan on windows.
FossilOrigin-Name: 6463192369ff22eb15d4e34590ef55cd8a6f2501227835664816277806bf961b
2023-08-28 20:02:59 +00:00
dan 9b0e9f9089 Fixes for testrunner.tcl on windows.
FossilOrigin-Name: 26ae090889640bfcf02a6dd5aa58044edd4d0708a11d8e005149f1a3c8495bba
2023-08-28 18:15:14 +00:00
dan d7def73c97 Another fix for valgrind permutation. Remove the "copydir" table column - just add a [cp] or [copy] command to the test script where this functionality is required.
FossilOrigin-Name: 448b9a575e66960a0885864f88b99b6ea15655c77a42eef956c09c8dc70e539a
2023-08-28 16:28:00 +00:00
stephan ece2d7f2be Remove a pair of what are arguably unnecessary mutex locks (and often hit). More JNI-internal cleanups.
FossilOrigin-Name: ecf07a0144dc6402b1e0924b1775d99dc465b27aa86a2718cac60a9b4c974312
2023-08-28 16:22:31 +00:00
stephan 8d64c12a10 Update tool/mkctimec.tcl to account for [fe7365254d343e].
FossilOrigin-Name: 349aac7e8d513bc420e8948b84cf715e454443439fdcd5aff0f2b13815a9cbb3
2023-08-28 16:05:19 +00:00
drh 57b1800773 Add support for the -DSQLITE_EXTRA_AUTOEXT=name compile-time option.
[forum:/forumpost/00829394c74a670f|forum thread 00829394c74a670f].

FossilOrigin-Name: fe7365254d343ed03e11a4e9cad7f0e5d5182b9220c6fde6d30e434ebdaba2af
2023-08-28 15:58:00 +00:00
stephan 25e7d8af75 Duplicate code consolidation.
FossilOrigin-Name: 71f239747c7934310dedf9fc0cbf84fbeeed53808234067147335c12396849a1
2023-08-28 14:59:27 +00:00
stephan bdb149d063 Move a metrics counter so that it can be used to indirectly witness the race condition fix from [f5274e00f17d58e0] by recording varying final values for that metric across consecutive multi-threaded test runs.
FossilOrigin-Name: 54d2209d24547dbb05e7c12daa27211593c34de8005e8adba0989b8d219f5f3b
2023-08-28 13:18:46 +00:00
stephan 4e9b48cfc4 Resolve a JNI-side race condition. Removed a now-extraneous struct member. Internal API renaming for consistency.
FossilOrigin-Name: f5274e00f17d58e075f90ae5c1d4b38933da315e51592171fa35bcbd67b40b2a
2023-08-28 13:06:26 +00:00
drh bd953dfcad Fix an issue with infinity handling by the SUM() function that goes back
to the extended-precision SUM() enhancement of
[check-in c63e26e705f5e967].  Problem reported by
[forum:/forumpost/1c06ddcacc86032a|forum post 1c06ddcacc86032a].

FossilOrigin-Name: 77d3dcd283595c52f24c07fc59ba60c9133b71c440cf3f799cf48c907c6fae3e
2023-08-28 12:20:18 +00:00
stephan 0d066bc8a6 Resolve a condition which could cause an is-interrupted db to call its progress callback.
FossilOrigin-Name: a0d0f1aafc6086726131dff5e6628f2771c20db3122a53bdbb82945ab5d326d1
2023-08-28 12:06:38 +00:00
dan f98612b1c5 Fix the valgrind permutation on this branch.
FossilOrigin-Name: ccbf06760e8377f9209ba3fd4895a25fe1a5ea688c384bf357e765b6a2b4c2f7
2023-08-28 11:22:33 +00:00
stephan f0e9e59d8d Further JNI cleanups and javadoc additions.
FossilOrigin-Name: c393b1c9f5639c259406e587b3e08efdc1962203001823747681379ffc54fc10
2023-08-28 11:10:13 +00:00
stephan 2597ec6385 Lots of javadoc-related tweaks.
FossilOrigin-Name: cfe06f90e2c0231efded98767ef3cc646d3a7daa34d77b62b7c04b5aae9448fd
2023-08-28 07:28:36 +00:00
stephan 23dfa67c1e Correct a package renaming bug in the previous check-in and start applying the @Canonical annotation to functions to distinguish canonical API functions and Java-specific APIs.
FossilOrigin-Name: 8a016006805b08b72bfc4093c795d8cd8d7fe72f8ae234c175a8b7be3a841cbf
2023-08-28 05:48:34 +00:00
stephan b041f8ffdd Move the JNI annotations into the annotation subpackage.
FossilOrigin-Name: e4dedf90a92a069daef967dfe975469bf8ec7883c44c95e73345d4eded48e996
2023-08-28 05:39:05 +00:00
stephan 56d3b21317 Resolve a makefile bug which causes sqlite3-worker1-bundler-friendly.mjs to be built incorrectly. Reported in [forum:a874e435cf4690c1|forum post a874e435cf4690c1].
FossilOrigin-Name: 9d68c7ef103b9b51ed35fb91e62351eb6c62ad63ab03ac456ff9787cc83f10ab
2023-08-28 04:59:16 +00:00
stephan 5020ddc243 Reimplement JNI's sqlite3_value_text() as a native instead of a Java-side proxy. Unrelated minor simplifications.
FossilOrigin-Name: 401dd6ee283a4e7da2e50175a1b58e6a83afb0ea2d3b6cf80c7c621e17bcce98
2023-08-27 15:15:46 +00:00
stephan 0639c58836 Generic cleanups in Tester1.java.
FossilOrigin-Name: fbc1b6310b01f4f79439bb6ce59b2aaabd126da7201f36d4fc5d7ef95118a494
2023-08-27 14:47:45 +00:00
stephan 5575d6421b Remove JNI sqlite3_column_java_object(), as the protection rules of sqlite3_values makes it impossible to implement safely. Add JNI sqlite3_bind_java_object().
FossilOrigin-Name: 29bd4a23a4afd96b2cc06d2b91a4f30c0bbf2347af0b0d18f8d4cf8aafa63160
2023-08-27 13:43:45 +00:00
stephan 88bd53dfd0 Correct the signature mismatch between JNI sqlite3_column/value_text16() and add related tests.
FossilOrigin-Name: 77f6e70f17c0cb6d031f983c458c9ec2e88d92b4716397533a029af39da2d128
2023-08-27 11:28:57 +00:00
stephan 95f5d85d4a Make JNI sqlite3_trace_v2() thread-safe. Re-add a piece removed in [bae4d022aad9b] to work around a JVM crash which is unpredictably triggered by its substitute. Fix the THREADMODE=0 JNI build. Further internal API simplifications.
FossilOrigin-Name: 3f9f7a9cb08b0687ad206605a5109306762df9ae8bdeab2d8d60bf9373c9ad32
2023-08-27 10:40:00 +00:00
stephan 32a79760b5 Factor out an unnecessary struct member. JNI sqlite3_shutdown() now frees up the various object-recycling bins. Doc touchups.
FossilOrigin-Name: bae4d022aad9bbeb78cb027ecad799af87afe331e697add44ec22297c873141d
2023-08-27 09:12:50 +00:00
stephan 60aca33a8b Factor out a superfluous JNI class. Doc and code style cleanups.
FossilOrigin-Name: 0f37f27148dfa93ecc42381ad3455a9059285d1af2df027429044942dc4d861b
2023-08-27 08:10:59 +00:00
stephan 0f4bf3435a JNI code reorgs and simplify the failing-alloc interface a bit.
FossilOrigin-Name: deed5797de65a25896e991a441f0d05eb92662536296485920fb081e84ad5d32
2023-08-27 07:26:33 +00:00
stephan c7e7c88873 Apply the JNI OOM checks to memory returned by JDK APIs, as distinct from our APIs.
FossilOrigin-Name: 1ff78582bfd934e0c76464b5f23ed9bf09a3491b145e0ca34acb6e59c4f53995
2023-08-26 22:34:26 +00:00
stephan 0deaea218d Eliminate a superfluous JNI-internal middle-man class.
FossilOrigin-Name: daede0f801f59d6501a863c4688e4635b34171e98b56b8ab4432c779113f1997
2023-08-26 21:13:57 +00:00
dan 1f1169ad09 Updates to testrunner.tcl so that it runs "make fuzztest" using multiple jobs.
FossilOrigin-Name: 7596ea7074e0ac73312586ed3d28cdacf97f54b8af73f804cbc8066c94d4b4ef
2023-08-26 21:04:54 +00:00
stephan 506a8b52f3 Whether or not OOM is always fatal in JNI is now a compile-time option.
FossilOrigin-Name: 320a34c080d8bc1feae1578697923dfa7c4144b78de36f704c24cc4a4ce9d535
2023-08-26 21:02:50 +00:00
stephan 6428cd18d6 Correct a string length misuse in JNI sqlite3_result_error() in an OOM case. Unrelated minor JNI cleanups.
FossilOrigin-Name: 4252f56f3d8574b7b43306440726daf3b5f5500d5d9105784b2f82753e7c71dd
2023-08-26 19:34:49 +00:00
stephan 6a1ebbdb90 Remove a bunch of commented-out debug output.
FossilOrigin-Name: b49488481e2952294960bb0ee971f6eca126c19d68ef92152894aa28393e6865
2023-08-26 18:15:33 +00:00
stephan 220b6145f5 Re-order some out-of-order JNI APIs. Correct JNI sqlite3_open_v2() behavior in a particular OOM condition.
FossilOrigin-Name: 49d3be002ce5e594027f47a3ba448f0c21ec68b416b8df997497753f53e3ca52
2023-08-26 18:01:36 +00:00
stephan 2a6835fe0c Two significant typo fixes for [1a30919bfbb686].
FossilOrigin-Name: 6a6ee6dd54ec1a1dc7fb7d2f331951525d394a8362aef911f62878ac3961898f
2023-08-26 17:36:15 +00:00
stephan 3cb610c010 Bind sqlite3_db_handle() to JNI.
FossilOrigin-Name: 1a30919bfbb686464e8020e684120d22dd2375b910a17446b75452fda96a7449
2023-08-26 17:30:14 +00:00
stephan 2a0dc4895b Correct a potential duplicate xDestroy() being triggered for Java-side collations.
FossilOrigin-Name: 25331172f78544f7f23ad7821cbf065589f6d02706184d4c83fe3988452dac5d
2023-08-26 16:55:27 +00:00
stephan fec793dd8a Recycle per-UDF JNI state.
FossilOrigin-Name: cf406528eb86d8d0d55a468b2c4ec32a11a4947f45c4bbabdde8742ae199ce1f
2023-08-26 16:29:48 +00:00
stephan a7da592bd7 Code consolidation cleanups.
FossilOrigin-Name: d6b5ecd28740c2c5d21797fce9fe137c8a83f702f22901720cc6e8b1b42af001
2023-08-26 14:55:44 +00:00
stephan acfbe5284e Add -sqllog flag to the JNI 'tests' target because it hits APIs which are otherwise not tested.
FossilOrigin-Name: cc3153ed341f59262485c3541a8879c4e86520c8a10f4ce819344a88099e7d0e
2023-08-26 12:19:51 +00:00
stephan 3ff458d61e JNI internal cleanups and docs.
FossilOrigin-Name: b7b26bfb4f86e0b8aaabab258ccb0713737ffd4ecd3156d6a83a4f9a1d25edf6
2023-08-26 11:57:34 +00:00
stephan d9cf47e377 Remove the Java BusyHandler.xDestroy() method - it should not have had one. Eliminate the last of the potentially-significant MUTF-8 cases.
FossilOrigin-Name: c852f1ebbde273c3d28fe5aff0bf73cfc06b41dd371a94d7520536dc7a1dbcc1
2023-08-26 10:51:19 +00:00
stephan 6f8f587d2f Improve threading support for all types of JNI-side callback hooks, making them safe to invoke if another thread is busy replacing them.
FossilOrigin-Name: f2af7bbf493fe28d92fc9c77425f8bb9d48c02af9a5eabceb0365c705651e114
2023-08-26 10:20:38 +00:00
stephan 25033b6b3b Move the 3 Java SQLFunction subclasses from inner classes to package scope.
FossilOrigin-Name: 21fd47a68db9df1828f4cc4131d326a193b5751d56a40ae77ed0a78dc0621af1
2023-08-25 16:43:51 +00:00
stephan 9fed4aab77 Missed a straggler when renaming Java classes in [8ca528006533ac1]. Document threading limitations.
FossilOrigin-Name: 5786b95f5d09b4462aff0fdeac37992a2b64c47b004d18960f51e4e6a5796106
2023-08-25 16:26:16 +00:00
stephan a5f09eb29d Hopefully the last major API rename in the JNI bits: rename all lower_camel_case classes which do not have direct counterparts in the C API to UpperCamelCase, as that's more idiomatic. Java types which reflect C-style types are unaffected.
FossilOrigin-Name: 8ca528006533ac152e9b322e421cf5d339414c77332d9836a9ff429b793d0d2d
2023-08-25 14:17:36 +00:00
stephan ad818380cc More Java API renaming for consistency.
FossilOrigin-Name: 4e457778de189d8393e88f399dc521c8a5ad4ddd80a5fd69295bcb104fac5cbf
2023-08-25 13:23:45 +00:00
stephan a899186c86 Re-enable SQLITE_THREADSAFE=0 in the JNI build and add multitest target which builds and runs the tests in all 3 threading modes.
FossilOrigin-Name: 6b1c816f6aae9face07ed7ead969257cf915c21ff9cdad9a21b7cf0e7496e09e
2023-08-25 12:29:59 +00:00
stephan 0199669fa1 Code restructuring. Force SQLITE_THREADSAFE in JNI builds for the time being, as threadsafe==0 leads to as-yet-mysterious JNI-level reference errors.
FossilOrigin-Name: 5a099caa2c21bec647f0a521e7f5d0d1cc2f96d388d3d6c53d5ec80947f33e8d
2023-08-25 11:32:56 +00:00
dan 35f9b1719b Remove incorrect (but harmless) dependency on source code file "sessionfuzz-data1.db" from main.mk and Makefile.in.
FossilOrigin-Name: 17d56c0207f63614b34ef3594d06602ab7a6e85604f3589b30aa79316f1744ee
2023-08-25 11:06:26 +00:00
stephan 5ecb51c260 Only build in the JNI-side metrics tracking in SQLITE_DEBUG builds.
FossilOrigin-Name: 646e7fc3b5ba81c207f013c9a06781986138379f20e787320a811ba3ed5489dc
2023-08-25 04:27:17 +00:00
stephan aec1361a2b Javadoc and internal API cleanups.
FossilOrigin-Name: 10a43cba9bb2c23b5f31b31fcac81be34acec1940cca47d0b8e5a5ccbf1aa8eb
2023-08-25 04:02:33 +00:00
stephan 5276552083 Replace all of the JNI XyzHook/Handler classes with snake_cased ones which follow unified naming conventions to make them easier to apply.
FossilOrigin-Name: 76e62a381249b3b4262b22bdffe7fc2816c820115c9df266956ab8817b127aca
2023-08-25 02:57:30 +00:00
stephan 44b4b8260f javadoc additions.
FossilOrigin-Name: bedf33d403677d243a1505ce549166850ab55671700b143278db5feb84883ab3
2023-08-25 00:27:28 +00:00
stephan 3401736694 Re-frame the incongruous SQLite3Jni.uncacheThread() API as sqlite3_java_uncache_thread().
FossilOrigin-Name: 7232b033954fae40df3db43e489e0e5a703c03308f500a1ae36fd9d707632d7f
2023-08-24 22:28:44 +00:00
stephan 70dcc2822d Update some outdated JNI docs and account for a function renamed earlier this evening.
FossilOrigin-Name: 3f684ef5018116f4be46a07779451c8983ac87a5db182477f71ee7bf28287a04
2023-08-24 21:45:30 +00:00
stephan 0f0bf3ff9e Do not pre-allocate sqlite3_aggregate_context() for Java UDFs, as it unduly complicates UDF initialization.
FossilOrigin-Name: e8308f0c6ec2d8999c8a2502fb130cb3501ba326f23f71f2cd8d452debae79b5
2023-08-24 21:31:56 +00:00
dan 36018803d6 Add doc/testrunner.md, for documenting the testrunner.tcl script.
FossilOrigin-Name: 9c69a28401c7273823f2c2b291fd417febeb278afb9ce085a4b944505ca13d23
2023-08-24 19:08:50 +00:00
stephan eef599dbd4 More code legibility and style improvements in the JNI pieces. Start work on a javadoc build.
FossilOrigin-Name: 62b404d62fd62f4d220838b59c9f38a71afa2d4a8c3af0a5c9495fa7020972cf
2023-08-24 18:43:25 +00:00
stephan e336d92688 Doc, code style, and legibility cleanups.
FossilOrigin-Name: cf185bcd25629d882a030b8b87048179a120ab1f84aa1d68b279c499dbdf0dba
2023-08-24 17:25:05 +00:00
stephan cf8a93f804 Doc style fixes.
FossilOrigin-Name: 862f9828e078ae138c3533658c45e4c45155975794e752b9b3a71a693842f37a
2023-08-24 15:42:08 +00:00
stephan e6c299a5c8 Add a new non-goal to the JNI README.md.
FossilOrigin-Name: 53000cdae9b265752f8d8616679da621a571a6590c6eaeca23cf87a0e13aa4c4
2023-08-24 15:11:28 +00:00
stephan 8e52281659 Merge the jni-threading branch into trunk, eliminating the JNI API's prior threading limitations.
FossilOrigin-Name: 3739c8aa7080d8e1044ca51ab7b699b50da4d29620b35acfcea2745059b65bac
2023-08-24 14:49:29 +00:00
stephan 49ca4d20fa Merge into trunk improvements made to the wasm APIs which were too late for 3.43.
FossilOrigin-Name: ac9da5c79a4f56d25202d50974e16e2a463d77c99b1907aee2605d5a3e50a565
2023-08-24 14:43:30 +00:00
stephan e7a4685837 Correct JNI layer's misuse of an sqlite3-internal error-reporting API (no mutex held). Style cleanups. Eliminate lookups of per-thread state by approximately 85% across the test suite.
FossilOrigin-Name: 1f46ba8d3bc61af771c1e33d09ad25f0da4fc4f915f7a9f6223ebfd99526d81d
2023-08-24 14:31:36 +00:00
stephan 6043947eee Ensure that db mutex is held when using sqlite3ErrorWithMessage() to avoid a potential assertion in debug builds.
FossilOrigin-Name: b8f6a50a4bf9478324f0272d79f2fe6992a49b753e79e39a268c6afb261bb01e
2023-08-24 14:17:30 +00:00
drh 995f96f8e1 Version 3.43.0
FossilOrigin-Name: 0f80b798b3f4b81a7bb4233c58294edd0f1156f36b6ecf5ab8e83631d468778c
2023-08-24 12:36:59 +00:00
dan e174dba739 Change a variable name in fts5FlushOneHash() to avoid shadowing another.
FossilOrigin-Name: 3f5bec9a684f6b3d88383617831862fc5595a51ad99ec12430ac6650b704ccbb
2023-08-24 12:01:30 +00:00
stephan bfdc7ab5a7 Add more JNI docs, tests, and a handful of Java-side overloads.
FossilOrigin-Name: d19a431facbde6a6b960664674753ee85d2c051a76109ce7db0b079c65fbdea0
2023-08-24 11:57:51 +00:00
drh af885d9fc4 Fix typo in the Lemon documentation.
FossilOrigin-Name: 5daeb318c0472e8c004109caacd8bc977145a2c1bfa6f5bdcf87ab89ea51c30b
2023-08-23 18:42:04 +00:00
drh a4709f27d4 More changes to documents for GitHub-style markdown.
FossilOrigin-Name: f20f84ef717cea11f92d6a1ab3b4e0fec5752b7ec98ad5b59e06745e7e25b6ee
2023-08-23 18:29:41 +00:00
drh 2bab8774ba Fix the Markdown in the compile-for-windows.md document so that it works
on GitHub.  Improved header text in README.md.

FossilOrigin-Name: a25360c020275064711d0aa3eb3cec3d764b5266b86f9728243e501d082306ec
2023-08-23 18:26:00 +00:00
stephan 8cafdfa916 JNI cleanups regarding building with certain features disabled.
FossilOrigin-Name: a9e6d5158b8a4a6b8554a5f8f0a35785ee450d42ea877275dc27085e89716c18
2023-08-23 17:52:51 +00:00
stephan 1c3bf8a3e1 Remove unnecessary jclass-type struct members.
FossilOrigin-Name: d67255f7251cc5d1d27d77d4c84ff216e2da71202db989718189a6b4beff1cd0
2023-08-23 17:15:48 +00:00
stephan 495046ef88 Add a note to the JNI README explaining why the Java API has callback names like xFunc() and xPreUpdate().
FossilOrigin-Name: 415447a310f6a7d06b4aa9ef51f110cf8e2ef9545c69cb5983c367c50fe641d2
2023-08-23 13:36:27 +00:00
stephan bea9ed0f1f Bind sqlite3_preupdate_hook() and friends to JNI.
FossilOrigin-Name: d0c425b5c1d3aac5ead18a501a3760b4506d68d373cb3be484247042cf2fa8d4
2023-08-23 13:17:37 +00:00
drh 44a430b33c Minor adjustments to the wording about pull requests in the README.md file.
FossilOrigin-Name: 6b2d68afd8ecdfe8eb04e3f6c13bea4ba42f7c2f16e6fc4263ffefa29c55c527
2023-08-23 12:53:35 +00:00
stephan 4e97ab4296 Bind a subset of sqlite3_config() to JNI: threading modes and sqllog.
FossilOrigin-Name: fce8ecaf7f2e69a168978e6993e58c452c45f76c39da33f2869c9d947c16cab1
2023-08-23 10:36:12 +00:00
drh a2a4c3f14a The pageOnDirtyList() assertion is too slow even for debugging builds, for
some corner cases.  It makes the query appear to hang.  So make it an
EXPENSIVE_ASSERT instead.

FossilOrigin-Name: bb9dcdaf3244c4fc6872850a82b80c469203911e2f6e3a3211e508be39b59fa3
2023-08-23 10:20:39 +00:00
stephan 4c8ef3894e Numerous minor cleanups and code style conformance improvements.
FossilOrigin-Name: 6c92d884920e4ace54913fc60ceef6e43a4351f45a4cb3c4a0ed3d29d544a31b
2023-08-23 09:05:16 +00:00
stephan 336bc8a281 Improve C-side exception handling from Java-side UDF callbacks.
FossilOrigin-Name: aebbc24afb339ed07b7cd767fbc0d25f3e9c3d9bb5ef3d1c10b04b605c7261bc
2023-08-23 00:17:28 +00:00
stephan d53565b4f8 Minor Tester1.java cleanups.
FossilOrigin-Name: 70d936953ba55cfed32efe7e3a9d4b71da9a7ffc8818b6540471e0bf311bc688
2023-08-22 23:00:44 +00:00
stephan 3600976bf1 Fix Tester1 so that exceptions triggered via threads are not silently ignored. Disable auto-extension tests in multi-thread mode because concurrent threads rightfully interfere with that.
FossilOrigin-Name: 56b2a077ace6e6ad5834e1a597b710f212a5b7d5db5b9a27a41f2aa0f6952c55
2023-08-22 22:13:08 +00:00
stephan d1c7216b2f More work on the JNI multi-threaded test runner.
FossilOrigin-Name: 9a74ad716bded1e14333bf7c72392916f800d58a96240eabe4988ca5fc9e8752
2023-08-22 20:10:28 +00:00
stephan 87bb103038 Disassociate JNI db handles from the thread that created them, as it's no longer relevant.
FossilOrigin-Name: 8b78b737e66a399b04e555a8197f63a73198a4105cb2f37ffd5b0e6014302caf
2023-08-22 18:36:30 +00:00
stephan c675add616 Correct JNI binding of sqlite3_shutdown() to clean up all cached JNIEnv objects.
FossilOrigin-Name: 02e868690f97ca728b0f2dd018aa79a9d13c85dd85b164caa895d319ae8f3ff5
2023-08-22 17:51:57 +00:00
stephan a7e3a1c09b JNI internal cleanups and correct two leaked db handles in test code.
FossilOrigin-Name: f927a30b5bba35991f472084ebaf02779e84c343a4e84f0efb3df7679ff212f8
2023-08-22 17:36:59 +00:00
drh 675342a5a8 Fix a harmless memory leak in the sqldiff utility.
FossilOrigin-Name: 724bc15701f3f647c741b614d4ac4fd98e1cdfe49a85f48c64fb1df5b83811b0
2023-08-22 16:44:33 +00:00
stephan 9828aa223a Move the JNI per-thread cache of NativePointerHolder refs into global space. This allows better-targeted mutex locks and incidentally eliminates the lagginess and post-run hangs in Tester1's multi-thread mode (presumably caused by deadlocks).
FossilOrigin-Name: e209f56a9745695aadc04418c7bebe62b79e38e5aee26c3248a30f73bfa460c2
2023-08-22 15:30:35 +00:00
dan 44e24e9640 Add makefile target "sdevtest" - like "mdevtest" except asan and usan enabled for the non-debug test.
FossilOrigin-Name: a449d650cd3dcd1baaeb3a3de2aaaac45594397e04f95fe637b0fe4ddb273404
2023-08-22 15:19:50 +00:00
dan 310700228b Update version number in autoconf/tea/configure.ac.
FossilOrigin-Name: 055ae724ff34bd41a1d97180b90fce081861856ac67572a633d31f518c20aa1e
2023-08-22 13:41:18 +00:00
stephan 0a102087d8 Move most of the per-JNIEnv global Java class refs into the global state, saving a bit of per-thread overhead.
FossilOrigin-Name: 7342bf578790e1a87c128a7c1c7745fe2e7c442890370feb160d406597d4d8ec
2023-08-22 11:34:34 +00:00
dan 216524f858 Update autoconf/Makefile.msc (by running mkmsvcmin.tcl) to support the "ASAN=1" option.
FossilOrigin-Name: 17028ba7a9bb1c5e26cb8c0d051d709df95d9f779fd3b1ec42adc924246f8f19
2023-08-22 11:20:38 +00:00
stephan 484f9bed4e Minor JNI cleanups.
FossilOrigin-Name: b88910aaaaaaa0936974379bb3eb8a5a3a634395b14e67cc9030f8a520f471f1
2023-08-21 23:45:19 +00:00
dan da7662a4b0 Fix an undefined integer overflow in fts5.
FossilOrigin-Name: f5c50349b1e4da233b554f371be49297b8b40ca5fe0188899ed906f682bf70d6
2023-08-21 16:14:58 +00:00
drh ba481c3756 ifdef adjustments so that SQLITE_OMIT_WAL works with SQLITE_USE_SEH.
FossilOrigin-Name: 0462a2612d1fc1d077acf60ef415f358aa2c44174c4ef28f4bbfdbcbf03d3b7c
2023-08-21 15:29:21 +00:00
drh 3404b452a4 Fix another UBSAN inaccuracy in fuzzcheck.
FossilOrigin-Name: a6fb9de8f87261194e6034376f62b1cac30b764f106dd3adcf728e020f8a4fd5
2023-08-21 14:40:19 +00:00
drh 114ad2f86f Address minor inaccuracies in fuzzcheck and RTREE found by UBSAN.
FossilOrigin-Name: a64e6c2fd8ef1d2d7f14aa9265629853298bbc480c2683c79dfd24847a61b68b
2023-08-21 14:28:47 +00:00
stephan 1089277105 JNI cleanups.
FossilOrigin-Name: 0a84131008a2e7886dac64a3545dea634811f6eac2b90885ec9c61ed1e6544c3
2023-08-19 14:49:08 +00:00
stephan 6f92f35449 Merge trunk into jni-threading branch.
FossilOrigin-Name: 8254479c6ff1ea3cc9e56de1698db8405c03da90b9bf4c401182e47e0842baf8
2023-08-19 12:34:23 +00:00
stephan bfa486d5fc JNI doc additions.
FossilOrigin-Name: 0c7ac34f30e1f7e35a2ac4e5e55e5f24857b24afa81a7abecba60f1c9c68b9ff
2023-08-19 12:32:00 +00:00
stephan 187da43379 JNI test code cleanups.
FossilOrigin-Name: e202b6e69da8cced114d027cf2e91a04dfdd50b601b3274214783f7d750c558c
2023-08-19 11:52:36 +00:00
stephan 8d9179bd07 Minor JNI cleanups.
FossilOrigin-Name: 1cecb9e0383aa78c491f9ba88c831a88b4b2d40ceef1b87be494b6ddc0789e41
2023-08-19 11:26:52 +00:00
stephan 3cf6c0f276 Add multi-thread run mode to JNI Tester1. It works but hangs on exit sometimes for Java reasons as yet not understood.
FossilOrigin-Name: bdbaf7a4534f40e550b646979e67e7b7731566bb5a2631ed376ac85a9bec40a7
2023-08-19 10:43:05 +00:00
stephan 46d677e713 Replace JNI::NewStringUTF() for the remaining cases where output may be incompatible with MUTF-8. It is now only used when we know the output to be plain ASCII.
FossilOrigin-Name: 2d955eef25ab116c487ebc34c6f2d2836d310af239ef1993f5aeee5a3f68d590
2023-08-19 08:22:34 +00:00
stephan 78ad745ab9 shell.c.in help text typo fix. No functional changes.
FossilOrigin-Name: c319033276c3565d0f1f2cae1c91791940d322fe79696bc26d74fddeb2664373
2023-08-19 08:12:10 +00:00
drh 4d8eb16f07 Rename the decimal_sci() function to decimal_exp().
[forum:/forumpost/fa027bb0ab|Forum post fa027bb0ab].

FossilOrigin-Name: c48f1be8cc505a7e2902c79e26c1d9a121ff5c55785ac812d2e09232b2414695
2023-08-18 15:39:38 +00:00
dan d5a9d522fb Avoid running test case sort2.2.3 under address-sanitizer. It runs out of memory on some 32-bit platforms.
FossilOrigin-Name: 925be7539947078547ee129b3d328ab0c59b19c6e2e5a391840f2d58f6d59acd
2023-08-18 15:17:52 +00:00
stephan 656f6c0c80 Improve exception handling in OpfsDb.importDb().
FossilOrigin-Name: a4eedd63b58f5c273b671f687cb68a32259963dbb053a710770383c735fb7f26
2023-08-18 14:41:21 +00:00
stephan ccbfe97cd5 Extend the importDb() method of both OPFS VFSes to (A) support reading in an async streaming fashion via a callback and (B) automatically disable WAL mode in the imported db.
FossilOrigin-Name: 9b1398c96a4fd0b59e65faa8d5c98de4129f0f0357732f12cb2f5c53a08acdc2
2023-08-18 14:16:26 +00:00
drh f0eabde9a4 Enhance the CLI and the fuzzcheck utility programs to report whether they
are compiled as 32-bit or 64-bit for things like the --version option.

FossilOrigin-Name: ec7eeb055bfb0e5a4467d8a45fa53d84bb8ae80ca0474b687e2783e971648008
2023-08-18 12:15:44 +00:00
drh 2cffc5be2a Enhance the notes on compiling for Windows11 to explain how to do
32-bit builds on a 64-bit system.

FossilOrigin-Name: dd749e4042bdffb20fe71dd2dba91eab5edc32e4b2773c2fadba43cb54027a17
2023-08-18 11:36:33 +00:00
stephan 383df02b16 Remove some obsolete JNI-internal docs.
FossilOrigin-Name: 00a2a3736a6dcde81d920815520040f3c47f965165e7128ca1f4062e6ec7c17c
2023-08-17 22:04:07 +00:00
drh 2e0ce58d2c Do not allow bound parameters in the ON CONFLICT clause of an UPSERT.
dbsqlfuzz 9983e2c77634a8ccf33b5c91fa9982599de5f9e9

FossilOrigin-Name: 3c06709335eb4b98e3a684e3ebbae69eeb6a21b452bce29159c82bb632d6a042
2023-08-17 17:48:20 +00:00
dan e15b35d5a6 Fix an error in documentation comments in fts5.h.
FossilOrigin-Name: 24d96bd65d8ccdd50a6f4fe217d196728a2f8658d128f9352c287f64e034104a
2023-08-17 16:59:35 +00:00
dan fab6882fd6 Add extra test to fts5synonym2.test.
FossilOrigin-Name: 39ea11be3b35c5ecde9bc0fe66af673b8f440dc523422887d05a29feef036b01
2023-08-17 16:49:06 +00:00
dan 699803e114 Add "Windows-Sanitize" configuration to "testrunner.tcl release" tests on windows.
FossilOrigin-Name: e68b46dc6aff48c81a2a5e7662d0f694615b64aa50bcfee9a0b8b25fa4e5f33d
2023-08-17 16:06:17 +00:00
dan 58c44132d1 Have all test names in json101.test start with "json101".
FossilOrigin-Name: af56d7b6b9c50f2541a51fcb7d94b89aafba4f343a9103b21a15aad3f283c63e
2023-08-17 15:14:43 +00:00
drh 81ef0f8e89 Enable address sanitizer in Makefile.msc by adding the ASAN=1 argument on
the nmake command line.

FossilOrigin-Name: 3eaadaff85111103c9441a9d6cd31291b95db9f3a6ec7003e0ff4ce5db7490da
2023-08-17 15:11:59 +00:00
drh 36a9f5c24c Fix a possible UAF in SEH if an exception occurs at an inopportune moment
during WAL processing.

FossilOrigin-Name: 91b91037e348fa10cf6a9d8a0ffbfdebb4a40e0e336b65b112ed1c828e75cda7
2023-08-17 14:19:44 +00:00
stephan 6c5f96fc4d Tighten up the JNI auto-ext handling.
FossilOrigin-Name: c09c8d05a20d916a9d9304eeea723ef7666a862a9e53f5feeeb1b03f9153d4b2
2023-08-17 13:13:22 +00:00
stephan 1dcb246988 Minor internal JNI cleanups and fixes.
FossilOrigin-Name: 0e9437de026cbfb333b90bb3400f1c015f85d49d73a25ad1000623216b88bfa0
2023-08-17 12:44:52 +00:00
stephan 0fa2545e7f Remove the FIXME markers related to threading. Code style cleanups.
FossilOrigin-Name: 154ab26dc6ba2d1fd976e8fe6dc1b1a06c734f7e9a276a3edc5c2f30b0d6d36a
2023-08-17 10:49:06 +00:00
stephan 95484726f6 Add SQLITE_EXTRA_AUTOEXT, similar to SQLITE_EXTRA_INIT but adds a builtin auto-extension provided by the client. Suggestion from [forum:00829394c74a670f| forum post 00829394c74a670f].
FossilOrigin-Name: 423e77277a61d7febf4c3fc737981fa22a82b5c774a8ada5375a01a0611535b2
2023-08-17 09:49:53 +00:00
drh 936f7deee4 Refinements to instructions on compiling for Windows.
FossilOrigin-Name: d543c36c35f71c5f0a7ebf6f496feca40d16566d0c5b2c2ba205ff43437ffcd1
2023-08-16 17:23:42 +00:00
drh 7335f75daf Improved instructions on how to build SQLite on Windows 11 and similar.
FossilOrigin-Name: 0f6b2b33736ee07f17f3a4e5f077bb4d0e2481c8f81251b8ce6b78510f372237
2023-08-16 17:12:26 +00:00
drh 0d83c71b34 Make sqlite3_stmt_explain() accessible to loadable extensions.
FossilOrigin-Name: a237a31ff4a7cd3dc744af1e1ebb9466e42eebafa31c1b5c0027cbaee9babc9b
2023-08-16 15:27:49 +00:00
drh 7309b50fbc Mix the current process ID into the randomness used for generating
temporary filenames on Windows.

FossilOrigin-Name: 775a36ee093df4b5f7529a43eeaee9d5a9a943ad5ed8ae03bc74e459e87ba438
2023-08-16 15:10:07 +00:00
dan af8980bdce Update testrunner.tcl to use environment variable %NUMBER_OF_PROCESSES% when running under tclsh on windows. Also modify the internal database schema used by testrunner.tcl to be compatible with old versions of SQLite.
FossilOrigin-Name: 6542ed3b9e028c44aca504eadca843ee9b2ba08f5f650523238dd1253f7e221b
2023-08-16 14:18:53 +00:00
dan fe18355267 Fix test script problem preventing veryquick.test from running.
FossilOrigin-Name: e73886574042108eb31641d0820c273c10b83fbf08ac6cb52d3e4c27830e2c23
2023-08-16 13:55:38 +00:00
stephan ff9ff54801 Cherrypick [00ac653562a6] to remove stray JNI debug output. No functional changes.
FossilOrigin-Name: e12d70f38c0ed5c0abf9a1e30ce4db59a4426e1e8cb1f8f07834fc1bedfcc65a
2023-08-15 21:44:22 +00:00
stephan c131675401 Cherrypick [653ed92dc391] (went to wrong branch). Doc change only, no code changes.
FossilOrigin-Name: 79a14457f0067814e7c33de709e2f9e19ab880d3eebbcb6f9fcce52a4720ecac
2023-08-15 20:57:42 +00:00
dan f05630fb1c Update testrunner.tcl to make it easier to add things like mdevtest.
FossilOrigin-Name: a531b71c4fd64fff6008876af825107ebc403f80dcb95273a05936ff6587b7aa
2023-08-15 18:52:25 +00:00
drh baa574ef32 Improve the portability of the src-verify.c tool on Windows.
FossilOrigin-Name: 4068d94acd042a0a9887fba8ba38fa94c6f41a6d3c7a61d5514a57f613b3d278
2023-08-15 18:08:14 +00:00
dan 8c5611a169 Fix problems in fts5 test code.
FossilOrigin-Name: c16ed3b1cc7b4c21bfa96062bf9028ae071c89e63df3b444b59f1753594e4e94
2023-08-15 14:53:49 +00:00
stephan abfe646c12 Add note about the current threading limitation to ext/jni/README.md. No code changes.
FossilOrigin-Name: 653ed92dc39185cdedfab3ea518bc7ec2d2826120e5fa4cbdee3343301396184
2023-08-15 13:01:20 +00:00
drh b93760e1e7 Use a dodgy substitute if the INFINITY macro is not available. See
[forum:/forumpost/8e66e19bb9|forumpost 8e66e19bb9].

FossilOrigin-Name: 4ae72c896761df5cd37c98ca3eeee7d4c6af3e4ccf74edd88ff902ef38f0765b
2023-08-15 12:27:27 +00:00
drh 00eee07911 Fix the PRAGMA case_sensitive_like command so that the LIKE function continues
to be innocuous after the PRAGMA.
[forum:/forumpost/925dc9f67804c540|Forum post 925dc9f67804c540].

FossilOrigin-Name: 84c268c34cba7207a90dad2a8e972ce90c85304e91f4933c76963822ad1ae48b
2023-08-15 11:58:22 +00:00
dan f0c1ea9e86 Fix an off-by-one error causing a buffer overread in test2.c.
FossilOrigin-Name: e1edf95ecc8f6840fd62abe83121fddeab7413c316b2e20b60fd3b2c8fdd4dca
2023-08-15 10:57:08 +00:00
stephan 6b51e35a9b Minor reshaping of Tester1 moving towards making a multi-threaded run mode.
FossilOrigin-Name: f104c14c26c123ee78c09fc1bc59efb8668dc624da05c1d8dbeaf3c9dd02a393
2023-08-15 09:26:47 +00:00
stephan 39c3f5f363 Remove a piece of stray debug output. No functional changes.
FossilOrigin-Name: 00ac653562a66aad3112ea322d08be68e05e6bf7413c814dd3f81bf850fcf43b
2023-08-15 09:16:41 +00:00
drh db9a708ee7 Fix the autoconf makefile for the CLI such that it disables double-quoted
strings by default, as all other builds of the CLI do.
[forum:/forumpost/9794b6aaa5|forum post 9794b6aaa5].

FossilOrigin-Name: 85fd4f0ef41a883448f16d55b7f8bd5c7b1d9c89a3d7d9bcae9c2744ed843099
2023-08-14 19:10:48 +00:00
dan 9541fbc283 Fix a problem in sqlite3_snapshot_recover() introduced by the [8a6b0c24937e855b] merge.
FossilOrigin-Name: c2577eb0a4a627c65d675f1446514843486092be9cb271b682793a0107e496cd
2023-08-14 18:21:56 +00:00
stephan 9019e2e667 Bring handling of the Java auto-ext handler more in line with the core in terms of locking and mutability during traversal. This removes the explicit synchronous requirement from the Java open() and auto-ext bindings.
FossilOrigin-Name: 42994b952e092ae4fa319395208622e887387ca3ff8ac57961c824a6c272bf0e
2023-08-14 17:12:55 +00:00
stephan 24b4cdd8d3 Remove some obsolete, commented-out makefile code. No functional or build changes.
FossilOrigin-Name: ea574cd5de6dbc7db2f4cffe0b6c83c795c4ba634ae1984e5c95bb3529daa5bb
2023-08-14 14:21:28 +00:00
drh 1f72b53461 Change the name of the ".binary" command in the CLI to ".crnl". The sense of
the setting is inverted.  The original ".binary" still works for backwards
compatibility but is now undocumented.  ".crnl" is also undocumented on machines
where it is a no-op (all machines other than Windows).
[forum:/forumpost/8bd0b0fbdbc12477|forum thread 8bd0b0fbdbc12477]

FossilOrigin-Name: 544de2da09cd7b9bbb6cb4c52bb22325ace3391fca00c6a43847bfc158032b66
2023-08-14 13:33:19 +00:00
stephan 7f2dea75ad More work on the JNI-specific mutexes. Rework the NativePointerHolder cache lookup to be slightly simpler and O(1) instead of O(N).
FossilOrigin-Name: c84ded0e59aea4861d72b53b4b40cf580747c0f6ca58c334a996f1a825276cb5
2023-08-14 13:27:40 +00:00
drh 592ae2823c Fix harmless compiler warnings reported by MSVC.
FossilOrigin-Name: 391e21bb6e0b2f4632972b6617a3a18192f88deb29eee5bc060846468e624b21
2023-08-14 12:20:44 +00:00
drh 07ef60350d Enable SEH be default in Windows builds.
FossilOrigin-Name: faae792fd5db7afeb7e4345d96f14c8986ea9a3cd4764ddee71314bfca36ac71
2023-08-14 12:05:38 +00:00
stephan d518e94adb JNI-internal docs and removal of obsolete code.
FossilOrigin-Name: b62d93258b6a661f3a9b61468b3b641c14faf2d2196f78aca95fe14de43c9444
2023-08-14 08:28:46 +00:00
stephan 6a1ed4c811 Internal API renaming for clarity's sake.
FossilOrigin-Name: 911e4fc5aaf9478214095a65f74af3ebca883922c36cf7a8d911116c42cf9de8
2023-08-13 20:58:12 +00:00
stephan 88381e53fc Add a mutex for auto-extensions, tied in to the open() process since that's the route into auto-extensions.
FossilOrigin-Name: 8da97e0db4eeacf91aa6fd909fd7cb73b050d194dfc7739a502b55f7eca6d7b1
2023-08-13 12:40:27 +00:00
stephan 4b4a911c5f Add some docs and metrics for the new mutex internals.
FossilOrigin-Name: 33d1780b43182d2574adbc1928707af825c485c99762738e58bc6d7c6c52ac6a
2023-08-13 10:28:35 +00:00
stephan 8a6888c19e Remove SQLITE_DEBUG from JNI build.
FossilOrigin-Name: 82f004cf7408c503c5787b46c7b13ceb0a20a3ad1d4f69d56127772c40e214a1
2023-08-13 10:22:15 +00:00
stephan 71e5694cd5 An initial attempt at protecting the JNI global state via mutexes at the C level instead of relying on Java's synchronized keyword. It seems to work but increases the run time of the single-threaded batch tester by roughly 3 times.
FossilOrigin-Name: c64e6a52ac79164be37fe643a4a39bd187af198a379410def8b8419f7c2224d4
2023-08-13 09:53:27 +00:00
stephan 0c07549fd6 Bind sqlite3_interrupt() and sqlite3_is_interrupted() to JNI but with caveats regarding mutexing of the JNIEnv cache. Add a loud warning to the JNI 'dist' target that it should be built with JDK8 (a.k.a. Java 1.8) for compatibility reasons.
FossilOrigin-Name: fbf99a2423dd20e4544bdeea85f714e9368ce3b92fefe97efb39a0fb4a557abe
2023-08-12 23:47:58 +00:00
stephan 202651fe86 Merge the Java Native Interface (JNI) binding into trunk.
FossilOrigin-Name: 48b13edcec6935bf125b265b41a3e6f7b2407afff89d5b4daa2939e3c5679ca0
2023-08-12 21:39:18 +00:00
stephan 54402964b0 Update wasm's push-testing rule to exclude files which were recently removed from the build.
FossilOrigin-Name: 0a6930a7ff8f8c6ca244d1d654532f3d2a02d77ef67c6cae0c53092743d59ea6
2023-08-12 21:08:41 +00:00
stephan d8a94dbb36 Correct JNI .jar rules to only include *.java/class files, not *.* (*~ files).
FossilOrigin-Name: 1ba7754045a009d9c94b23ac76b9bb8d9c9cb24d42dcdf1203ee75ac85765d3e
2023-08-12 20:50:29 +00:00
dan 46451fa8b5 Fix the mdevtest target so that it works with msvc.
FossilOrigin-Name: 01f49448cd0cfe3af499aedfe887b7b0be4f1ab09cd5a16119ddafb32b107708
2023-08-12 19:58:47 +00:00
dan 890a9ede3b Fix testrunner.tcl so that it restarts itself using [testfixture] if its current interpreter does not support [package require sqlite3].
FossilOrigin-Name: 408be98dddf500f2d6c9049703997836d80d156d86cf297f664175a107f7331e
2023-08-12 19:25:26 +00:00
dan 7c036fabe5 Have the mdevtest target support the OPTS variable on unix.
FossilOrigin-Name: 940f58d9b944d3344b40a4bc020842d9b6703685c9c51f4010e265afd98608ff
2023-08-12 18:23:38 +00:00
dan a6f09a0193 Add "mdevtest" target to makefiles. Equivalent to running "devtest" on both debug and non-debug builds.
FossilOrigin-Name: af354fbc0be06d5792a1e712ec4e9cec4c6cb4521335aa31f8f51ca5c94499ae
2023-08-12 17:36:57 +00:00
stephan 68522e1627 More JNI docs.
FossilOrigin-Name: 290028d3ca5638f3bb18a0b243c7ba3c31c8a2b1a837c36ec29e0fc4ed6533f0
2023-08-12 15:37:53 +00:00
stephan 5b3a754360 Further simplifications in the interface of the OutputPointer family of Java classes.
FossilOrigin-Name: 962c3e0de2d64ab8a2bcf1a19f9c4224df3d15a41ac9f9b29da685be95c4ef7a
2023-08-12 15:09:09 +00:00
stephan c216df75c6 Tweaks and docs for the OutputPointer family of Java classes.
FossilOrigin-Name: 265c8fd0d4d425054f6bf7e9cb607ad2e0e46189f16c3014f7fdf9b650085497
2023-08-12 10:39:26 +00:00
stephan 613390680d Bind sqlite3_db_status() to JNI.
FossilOrigin-Name: b79477a0af94127b0638a8822de01156bef855a7e167f678809e1c978e1a0c3e
2023-08-12 10:27:08 +00:00
stephan 238bea2ae6 Bind sqlite3_status(64)() to JNI.
FossilOrigin-Name: cefb6614e65ca1764ec72702f92f801382e63aa9b221fc9c68719d497e7499fd
2023-08-12 10:06:59 +00:00
stephan 6c6badd2ef Modernize the sqlite3 shell man page a bit, per [forum:d9aa201dee553bb2|forum request].
FossilOrigin-Name: 3bc0693c4633f545f09dbee702e25354504b20836373a068447e6c61cb2ebd79
2023-08-11 22:40:06 +00:00
stephan 48dcdd3b74 Add timing info to SQLTester.
FossilOrigin-Name: b69b5facbf94e03e74d4a739ab85c5baac1c9ecbea8c330b2135d77e525b5d8a
2023-08-11 21:25:33 +00:00
stephan a6746456d9 Merge trunk into jni branch.
FossilOrigin-Name: 4f0aeeba0287e846908180eab6f7080ebe1323ebe49340771864d110e1ca5b2b
2023-08-11 21:24:08 +00:00
stephan 783f5b5aa2 Fix a makefile deps problem which caused ext/jni build to fail if sqlite3.c/h were not created beforehand.
FossilOrigin-Name: 101de670774f63757180282763730aa53e70198bd7a674c27e6044632d39d22a
2023-08-11 20:32:40 +00:00
dan 2a28cff2e7 If SQLITE_USE_SEH is defined, handle structured-exceptions thrown by MSVC builds if the *-shm file mapping is accessed after it becomes invalid for some reason.
FossilOrigin-Name: 8a6b0c24937e855b710f97b4aea973eff53e6d43e1182842731547aa4b37db2a
2023-08-11 19:31:51 +00:00
stephan 16c8c96745 Minor SQLTester test tweaks and have 'make tester' include the out-of-tree tests if they are found.
FossilOrigin-Name: 6c8538d83495ce65dbd7417263b3b06dbbb2a649e9a61a743911944599d75ffc
2023-08-11 18:59:57 +00:00
stephan 7009c43eaf Bind sqlite3_sql() and sqlite3_expanded_sql() to JNI. Start marking C-side functions which would need explicit mutex support if we remove 'synchronized' from their Java entry points (but there are many more left to mark).
FossilOrigin-Name: c7fb32d1ef30d34449c3289c384ce33317c770927534af20d4b96fa385da40bc
2023-08-11 18:04:53 +00:00
stephan c3b7e1c4e8 Add a doc link for the new sqlite3_js_posix_create_file().
FossilOrigin-Name: 0d7aac45b8e7078cc80757e12d6f0b2584f2b0b184dacc2348ad3519978e5bf9
2023-08-11 17:45:23 +00:00
stephan b949244ea1 Add sqlite3.capi.sqlite3_js_posix_create_file() and oo1.OpfsDb.importDb() as alternatives for the newly-deprecated sqlite3_js_vfs_create_file().
FossilOrigin-Name: da6eaf8d8258f3e2c8633fd7faf4e90c3307b5c60bd8b69c626b3c82b19dbdef
2023-08-11 17:38:17 +00:00
stephan 7e13152952 Deprecate sqlite3_js_vfs_create_file() because, it was discovered today, its out-of-scope use of the sqlite3_vfs, sqlite3_file, and sqlite3_io_methods APIs triggers unresolvable assertions in the core when built with SQLITE_DEBUG.
FossilOrigin-Name: f3647a3ac8eca8c821b0b1e403da7bfb0feabd0eb5ee83709cd4956dfc56a492
2023-08-11 14:31:20 +00:00
drh 59d01de837 Provide the -DSQLITE_LEGACY_JSON_VALID compile-time option to restore the
(incorrect) legacy behavior of json_valid(NULL).

FossilOrigin-Name: 00bc9f1b573d683829bf5eb301606c38d6a60fba957d8edaf59116c02cc650bf
2023-08-11 11:30:43 +00:00
drh b8a0fc52a2 Fix bug in the test case for the previous check-in.
FossilOrigin-Name: c5daae88612607aa7a8b13b021cf586fc66e8bcd1fa9d948a860c881b7247761
2023-08-11 11:24:44 +00:00
drh 91c0092917 Up until version 3.42.0, there was a bug in json_valid() such that it would
return False (0) for a NULL input.  That bug is fixed in 3.42.0.  This
check-in adds a compile-time option -DSQLITE_LEGACY_JSON_VALID that restores
the old buggy behavior for applications that depend on it.

FossilOrigin-Name: 15c2eadbff8e732cca45d6c3771d1fcea5aab2127e87f2a611b41ccfef4d1a0d
2023-08-11 11:12:46 +00:00
stephan a0423e0fc1 Mark _all_ JNI binding funcs as synchronized so that Java can lock them and protect our global-state access. The alternative is writing a mountain of C-side code to do the same thing.
FossilOrigin-Name: afe190a940441de9bef8835c2dc6d278f861a772c3b7c7a2d399b2eabd4872e3
2023-08-10 21:50:52 +00:00
stephan 5c9f5e4022 Work around jdk8 and jdk19 mangling the C name of sqlite3_db_config() differently. Correct the variadic arg handling of the JNI-side subset of sqlite3_db_config() options.
FossilOrigin-Name: 746894c3c043c47f8b4c231de8921df81c5d0634260d299359bea73132dc7867
2023-08-10 21:29:59 +00:00
stephan 0c08c8c208 Add a working dist zip file for the JNI bits.
FossilOrigin-Name: d6a4d212ceba662470d8957b6a8d7075d18a84bd0d3e13ce7adcab03604fc3b7
2023-08-10 20:52:14 +00:00
stephan a52b8c8845 Initial pass at 'make dist' rules for the JNI bundle, but they still need a bare-bones, posix-make-compatible makefile to include in the bundle.
FossilOrigin-Name: ff54e66a4d43f2f0f8b25ded970779e6760865a05346e09b39607bb035b02bd7
2023-08-10 18:57:37 +00:00
drh 89e1caf294 New testcase() macro in the tokenizer, to better document its behavior.
FossilOrigin-Name: b2fdac0b151864eb2aa79f0b0ee60f9c6d9f3eb8c7626605eac17a02a8cf59bc
2023-08-10 18:50:00 +00:00
stephan 2ee01b0c9a Merge trunk into jni branch for the newly-relocated version-info tool.
FossilOrigin-Name: cc8e8cba67c0dcfb9b416041a19456cf5248d909f3efb6fee707a5950be4f374
2023-08-10 17:39:26 +00:00
stephan bb51af68f4 Move ext/wasm/version-info.c to tool/ for re-use in build other dist bundles.
FossilOrigin-Name: 4b0871fd367b6d9706e892aa13f64604967f5e3ba92381960f73aeabd3d23f84
2023-08-10 17:32:37 +00:00
dan 6de6121e36 Merge latest trunk changes into this branch.
FossilOrigin-Name: 3ed89c344fcb3b7ee8b764d95144643e42e053e1116150d6eda8355fbd6669df
2023-08-10 17:07:34 +00:00
stephan a22a2d266f Add SQLTester --keep-going flag to allow it to continue to the next script after an error.
FossilOrigin-Name: 4d635f781b55ed9011bdf07ee6bed2d004b1c2ebba76aa110e26d8fe3152a733
2023-08-10 16:42:22 +00:00
stephan 92f699d384 Document that auto-extensions registered via JNI will fail if they open a db (to avoid triggering an endless loop in the auto-extensions). Discover the hard way that JDKv19 creates different mangled JNI names for some functions than JDKv8. Start reformatting much of the JNI API decls in prep for making them even longer.
FossilOrigin-Name: 99c0941f1c006622932a9cca12661f354f363a6c8a2b5675ea66149e0a9eb927
2023-08-10 13:10:25 +00:00
stephan 1528014703 Resolve two assertions in the auto-extension JNI which were triggered via new SQLTester infrastructure. Move SQLTester's db-init SQL injection into an auto-extension.
FossilOrigin-Name: 2952906c30bc2b7987f2c39837d56bd121f5817dc094e6ccdb6d4eea5e9b8d17
2023-08-10 12:36:40 +00:00
stephan 61186d1243 Make test completion status more visible at a glance on modern terminals.
FossilOrigin-Name: a4e96c306c4c270f417243e7923d7e6c4f860528dd67990dfd8d9768a6c4873f
2023-08-10 11:15:20 +00:00
stephan e51fae4134 Disable REQUIRED_PROPERTIES handling in SQLTester, per /chat. Scripts with that directive are now skipped.
FossilOrigin-Name: ddc534cb25b59faf18a860a51f2dd41a1a73963aeb541b9553301fe784608393
2023-08-10 11:04:46 +00:00
stephan 0a8cc1f634 Add a visual indicator (emoji) to TestScript verbose messages, dependent on the message's level of verbosity.
FossilOrigin-Name: dc323d3894f2d53470cd8be261632267fa3d2af73500acfa1e9adbfa53b771fd
2023-08-10 10:58:55 +00:00
stephan 34fac74362 Add the current --testcase name to SQLTester --verbose output.
FossilOrigin-Name: f87367402b25adf30f35ab75aa5efc495230d4a83f2fc10b99734c3f3f593840
2023-08-10 10:44:53 +00:00
stephan ec8298fbd1 More SQLTester docs. Add --verbosity command to help zoom in on script areas while debugging. Spice up test-start/end output with some emoji.
FossilOrigin-Name: 8dd08021496f504c23945ecc2bbe1e4a13109fdd03457ca6269b4cb1cc4cd04c
2023-08-10 10:34:50 +00:00
stephan 63fb588d30 Give DbException the option of closing the db to simplify error handling in one case.
FossilOrigin-Name: 908c9a44505422a3a15bef3a174d8b931863bc9c74485311a0e62cfec30087bd
2023-08-10 05:25:13 +00:00
stephan 6a61a19f9a Initial version of REQUIRED_PROPERTIES support for SQLTester, with TEMPSTORE_(FILE/MEM) and RECURSIVE_TRIGGERS options.
FossilOrigin-Name: 48d16c9d2fe5f54b09004b4f09759c4e2ad247ae84130feb557951e32f48976a
2023-08-10 05:14:22 +00:00
stephan f703dfa381 Generic cleanups and fixes in SQLTester.
FossilOrigin-Name: fc5d3cc30d2b96da42ea10dfb39f1631ff93b8384514fffd641b343df51da2a6
2023-08-10 04:24:12 +00:00
stephan eda67031de Change the SQLite3Jni API annotations to use SOURCE retention (used only at compile-time).
FossilOrigin-Name: 3c3fea6bf284721ac376e2ab5a757cf30245dd39264aaf98a8d6cd5575484275
2023-08-10 02:09:12 +00:00
stephan 0c6df29cba Move all of the SQLTester code into a single file, since it's only got 1 public class. Remove 'public' from many methods which don't need it. Add more documentation to it.
FossilOrigin-Name: 2815d676951abdab674c374fd903486ea5796f8ee4cb338d41f19693419f8471
2023-08-10 01:44:48 +00:00
stephan 2a91065145 Defer static JNI-side init of SQLTester until main() is called so that its auto-extensions do not leak over to clients of the main library.
FossilOrigin-Name: e461fdd53bd3212bee24ec5f5d5c234011ab30f3f67e115de9f85fdb760e3848
2023-08-10 01:19:40 +00:00
stephan 58c2ca4483 Merge trunk into jni branch.
FossilOrigin-Name: 52fa6f78414c41073431c166550806bb8a835bd38cfc1236c9363784c78b81b9
2023-08-10 01:05:28 +00:00
stephan 50a17a1b3a Treat all args to --glob as a single glob.
FossilOrigin-Name: 4737bd345732e34cabbf90821e007d3027e68d6583c2e05e1e8bf5920373a6dc
2023-08-10 01:03:19 +00:00
stephan 3a0e45d0af Remove --print's ability to read ahead.
FossilOrigin-Name: 15cfee972bcbd857d18ea626c9a9da64b2a445516946f3dd83c75a4f6b8a92fa
2023-08-10 00:58:48 +00:00
stephan 84c1a7b49f Replace the SQLTester infrastructure with a line-oriented, non-regex-heavy parser. Add --column-names command.
FossilOrigin-Name: 88863908ee2059c2d18a095cbd91f41674c7b0d0a8864ec21715a5317054df4d
2023-08-10 00:34:38 +00:00
stephan e35a703b76 Port the SQLTester 'v1' commands to the 'v2' evaluation bits. Still TODO is swapping out v1 with these separate impls.
FossilOrigin-Name: 0cf57e5b0f90779e450e9db1ca009610df5e6f4487337d49017636bde3bb02d6
2023-08-09 23:47:14 +00:00
stephan 35af4c5ff1 Correct REQUIRED_PROPERTIES handling to not fail if there are no properties.
FossilOrigin-Name: 7a19bef4f572a90fb7896b9360f9c72b052955ca9b0549be870b2b245c1f1b2b
2023-08-09 22:30:10 +00:00
stephan bef0369ff4 More for the SQLTester rework. Can read input and dispatch commands, but only --print is currently implemented.
FossilOrigin-Name: 4fa2ad33edbcef393dd98dbf90586ad8f32ec0beab02f197c8038a44be86c314
2023-08-09 22:18:22 +00:00
stephan e41c09ac8c Initial sketches of a line-by-line parser for SQLTester to overcome its compatibility shortcomings. Far from complete.
FossilOrigin-Name: 43534cd042499c1bef44ca5c4a8305a710d99e70e8b0adce6df50c6a1f0402b9
2023-08-09 19:51:39 +00:00
stephan e0fb44090b Adapt TestScript to skip REQUIRED_PROPERTIES, per [7a07863e082664da], and improve a couple of adject reasons-for-skipping messages.
FossilOrigin-Name: f937097e9b22a6c78c242cbf00c71bdc57f04b1b9a15ae24058bc2813c99688c
2023-08-09 18:25:50 +00:00
drh 7f66a3eeb2 Add an extra constraint to the test-script-interpreter.md spec.
FossilOrigin-Name: 7a07863e082664da2efcf4ecd36785d2583abbda12526cdb643cf1aa0568292e
2023-08-09 17:47:34 +00:00
stephan 16e620e07f SQLTester --result command: do not double-{}-wrap error messages and do not throw on db error.
FossilOrigin-Name: f7be20f5a62f8970f957e5c3a1d1c6536995df5c078dbac34a44f09682e43945
2023-08-09 17:23:52 +00:00
stephan 5d2a618340 Move the test command body's trim() back into the TestScript class (to simplify Command-level code) but do it after double-verbose has had the chance to emit it as-is (so that debug output is not mangled by the trim).
FossilOrigin-Name: f15ecb68f7ca129478336b462508d2c40ea052b4040facefdbc67b13e6aea99d
2023-08-09 17:04:58 +00:00
stephan 583fc5b963 Tweak the SQLTester --verbose and double-verbose output a bit for legibility.
FossilOrigin-Name: 46b79afaafda40cb1f920cc96600adf11e8c688184c9559a08eb86776ccf3663
2023-08-09 16:56:42 +00:00
stephan 589857b965 Strip --oom commands from SQLTester input since (A) we can't currently do anything with them and (B) they can appear as body content of --testcase commands and the current parser cannot deal with that. If --verbose is provided once, emit the name and args of each command as it's run. If --verbose is used twice or more, also emit the command's body text, if any.
FossilOrigin-Name: 0770e8467d4bb9490d9ed6e8a20766ffee7049cc3667db6d036c13fccbb6f3ab
2023-08-09 16:29:07 +00:00
stephan bd87174bae Update the definition of "special characters" for the SQLTester and correct the code to match it.
FossilOrigin-Name: 217f6e0c9c09c576b09ea59fce085a53d1a133927046102b4d00fd58109efc93
2023-08-09 16:03:12 +00:00
stephan 65c7226e31 Extend TestScript to be able to report why it should be skipped. Expand the test-skipping rules to account for the current spec doc. Add the {} empty-string case to the spec doc.
FossilOrigin-Name: 4fcc8cb0cc2bbc0da71bdb99dacfdec54814af4c0e4c37619bad6a8e5fa62937
2023-08-09 15:46:55 +00:00
stephan 5b400552be Add a test for the --null command.
FossilOrigin-Name: 83ac815debcc75dac1fbbdc17736f5e33fb675fdab0bf649367592a0d18074e4
2023-08-09 14:47:01 +00:00
stephan aa6b35cc80 Extend SQLTester glob support with '#'.
FossilOrigin-Name: 756ef83f45b69d9f78965ef1171d36477a32f938fe179e59b95f32f07849c0e5
2023-08-09 14:43:54 +00:00
stephan 4340f27e17 Implement the SQLTester's result escaping rules.
FossilOrigin-Name: 61bb950873a1ec45a71b15a0ab5128a50417c4ecdd7d5bd9add0c18afcbadf34
2023-08-09 14:24:22 +00:00
stephan 5158a9a9a7 Add --json and --json-block SQLTester commands.
FossilOrigin-Name: 478129d901824e675d86494044f73c313532e9f80e7ee6f425474df8237a82f5
2023-08-09 13:51:50 +00:00
stephan ff6b15fbb2 Add SQLTester --tableresult command.
FossilOrigin-Name: 8c5b6d893df4a4e82c6d8e07507fc160b11412ede4bb903ff4e3f5ffa59a9cb9
2023-08-09 13:16:10 +00:00
stephan aec9aa9289 Clean up the SQLTester output a bit by using the module name, instead of filename, where appropriate.
FossilOrigin-Name: 5323e4fd254274cc527af7536c622b786394599c68eca2da6c7fc641727dbdb2
2023-08-09 12:05:17 +00:00
stephan 283e87146c Document SQLTester's --print command and add some argument validation to it.
FossilOrigin-Name: ab9c945bb0b4210b3f47e6341f150f8a7cc45f9e4e4c2247e91d2528ed4772a6
2023-08-09 11:10:48 +00:00
stephan e6c29da777 Rework how SQLTester's Command objects are dispatched and how TestScript stores its command entries.
FossilOrigin-Name: f929f1f7f70181813f74562614f3f2aa29e65590560e3fce1677b8b176e3c6de
2023-08-09 11:05:43 +00:00
stephan 23a35c2145 Some minor tweaks in SQLTester in prep for larger refactoring.
FossilOrigin-Name: 1d93f93ac9708839e62d2f1b489adc5d47ff290c2d5aef4dd56be4e1e46c81b2
2023-08-09 09:56:37 +00:00
stephan 41f9449016 Remove the current-statement tracking from the JNI internals because it will break down in the face of client-side mixed-mode native/java code, e.g. in cases like SQLTester. This makes tracing of sqlite3_stmt a micron slower but also reliably correct.
FossilOrigin-Name: 4c0ec89dca00a9199d1e36768c034aa5eff03b13b5e015cf580f160dc4f141ad
2023-08-08 22:10:27 +00:00
stephan 78fc4ae4d9 Adapt JNI build to be buildable with or without SQLTester.
FossilOrigin-Name: adae7d78692af73e770a9cc0a4264ab32ecc18a5c0deb64f3c1e790d959bab43
2023-08-08 21:22:56 +00:00
stephan c783fd36dc Add SQLTester dup() and dup_count() UDFs. Correct arg handling of the --run command.
FossilOrigin-Name: 0dba3073f44685a51a5db7ff8886295fe04dfd43f69cbf53ad3d5afce741076b
2023-08-08 21:05:39 +00:00
stephan 4d3aa08f67 Correct --result arg count check and add infrastructure to let us add custom C-side behavior to SQLTester via an auto extension.
FossilOrigin-Name: bb8321702eea52fa9d42987a4b053b32d8eba15580a39d7831cd8d6f1ceb62bf
2023-08-08 20:41:29 +00:00
stephan c15dce9f92 If SQLTester now treats a no-args --result as comparing against an empty string.
FossilOrigin-Name: 49005ca5cc191c52279bc7fdb45d95eeb6f8e344f78ce9dbd97aac814bc21202
2023-08-08 20:15:42 +00:00
stephan af825cd50b Add glob/notglob commands to SQLTester and complete the interrupted-midway impls of the strglob() and strlike() JNI bindings.
FossilOrigin-Name: 4ba98ec0bf24c31cce498031cb3727e09f928f54ec13c76fec50e439e0f2ba15
2023-08-08 20:02:10 +00:00
stephan 746bb422ea When SQLTester hits an unknown command, emit a warning and skip the rest of that script instead of aborting the whole run, per /chat discussion. Reduce verbosity a bit.
FossilOrigin-Name: 3e78d22d04e6ac2606bfc5ce250a4c3b39a2062e14011ca0a8a0a85491efbfde
2023-08-08 19:20:12 +00:00
drh 80c438613a Minor cleanups in sqlite3Int.h. By reordering some fields in the Parse
object, it packs more tightly and uses less memory and less CPU to
initialize.

FossilOrigin-Name: aa6de539c09faa320b68c63659e602107145c4263fa680d5b40fe4d7d7ac4534
2023-08-08 17:36:03 +00:00
drh 73cdbeb632 Fix an issue in the amalgamation generator in which it was not correctly
expanding the SQLite version in the header comment.

FossilOrigin-Name: 293f6191e9b328cb8a8d3fff0f7bd5a6f5390b5ff090497a170c791b6ea22917
2023-08-08 16:53:12 +00:00
stephan bff14ecfcc Correct the spacing output of multi-select SQL blocks for SQLTester --result.
FossilOrigin-Name: 8d98645a9e524b30f7faa1cffd8f09e7aab3c25ac7b08dd6884141dfe9cdb0d3
2023-08-08 14:58:00 +00:00
drh 358c531e9b Updates to the test script interpreter spec: Rather than failing immediately
upon encountering an incompatibility, simply abandon the rest of that particular
input file.

FossilOrigin-Name: d2c99b96f4b61561c3fa34947ca7bfd2cd214b1913aff7ba64b7b897a574fea3
2023-08-08 14:49:16 +00:00
stephan 671757f4d4 A number of baby steps for SQLTester.java. It can now handle basic --result cases.
FossilOrigin-Name: 0404f688f6a22b6bbe009de1bee3341ca00e19e2cc32081265cf151876dc032f
2023-08-08 14:40:47 +00:00
drh aa29140637 Add a few words of clarification on how the SQLite Test Script Interpreter
should be initialized to start each test script.

FossilOrigin-Name: 3aa2b5a5cadb214dc64a3db412b7dfdd805abd8681b61da857b886cba3b937b5
2023-08-08 14:25:47 +00:00
stephan 5f4e94722b Correct JNI sqlite3_prepare() to emit a null stmt handle when the C counterpart succeeds but results in a NULL pointer.
FossilOrigin-Name: 94628f88b5cc82832f0ca2b00fd5346bfe99323097c6e659c5ac818c4e31d3e9
2023-08-08 13:05:12 +00:00
stephan 405dffd591 Implement the new/open/close SQLTester commands.
FossilOrigin-Name: dc823bf00f78e7cd626329220c42c46da12d565e3273a08eda5fb512c1d807c6
2023-08-08 11:46:46 +00:00
stephan ec9b37b33e Add a JNI-layer sqlite3.toString() for debugging.
FossilOrigin-Name: 456691649aa2a7672d5d110acdde92426a9d34552863db3e0c86b73d9c5d9aac
2023-08-08 11:46:26 +00:00
stephan c693bd59f7 Bind sqlite3_strlike/strglob() to JNI.
FossilOrigin-Name: eb5440f71be32812f6310756b8e30958002e8e8e41a7eb16f081058ff733b47c
2023-08-08 10:58:16 +00:00
stephan aeae7904a5 SQLTester: add print command and improve argument error reporting infrastructure.
FossilOrigin-Name: 1b6e84f6aa5c7626a308b5e8efe5c3d83ec8e7eaa803f047576b7c65333c2d44
2023-08-08 09:45:33 +00:00
stephan 58eebdaa25 Add missing license header. Minor cleanups in SQLTester.
FossilOrigin-Name: 5be50fd5887e5378f7d66405bff3a44117a826a17e5f1a18c5129d1109ecdae2
2023-08-08 00:59:40 +00:00
stephan ec8edf3295 Rework SQLTester dispatching and add stub impls for several commmands.
FossilOrigin-Name: 9e61af75ac83e74487a6ae681ee3ff891d8cf1f1d23bf895e9e3963ddf6eaf28
2023-08-08 00:37:31 +00:00
stephan e9ed375876 Add command dispatcher to SQLTester.
FossilOrigin-Name: e0a06931e91459ea43fed2954568bfafa7ad6b794fcff66e0d3bf0ed181db386
2023-08-07 23:59:08 +00:00
stephan b7f75b7bb2 SQLTester now ignores tests which contain constructs specified in the spec doc.
FossilOrigin-Name: ecaeee652aa2cc6893ded9231d7e9b2783465516016740b307b74e4e81598ae3
2023-08-07 23:04:17 +00:00
stephan fdeaee5f57 SQLTester can now split a test script into a series of individual commands.
FossilOrigin-Name: d3d1accc8b4ba0cd396ee3a58d9710a54b8e1d1b171d67595d4ef1fc7faea8cb
2023-08-07 22:32:22 +00:00
stephan 70679d135d SQLTester can now read a script and strip it of all noise content.
FossilOrigin-Name: 59bd392817ac69ffdf60ab7a2094b0d616bf593da060b6acf1b4ce9837847fcb
2023-08-07 22:02:43 +00:00
stephan 3897a882f7 Initial skeleton for adding an SQL-driven test script interpreter for the JNI bindings.
FossilOrigin-Name: 2aa8f0edecd3fc30eec28987cdbf1003ace154ddc1447b6f8715ecf38d3b06fb
2023-08-07 21:04:13 +00:00
dan 786a9dcf5d Enhance sqlite3_vtab_nochange() so that it works with "UPDATE ... FROM ..." statements. Use this to allow some updates on fts5 contentless-delete tables.
FossilOrigin-Name: 27ff86e4d8d251dbbcc9f0682d3d7b040518cbeee891cfe253661d1fdbec4e4f
2023-08-07 17:09:25 +00:00
dan be3ab76bfb Changes so that sqlite3_vtab_nochange() works with "UPDATE ... FROM...". Use this to allow UPDATE on a contentless fts5 table if new values are supplied for all indexed columns.
FossilOrigin-Name: 16cd2161e312cf97129011fc829079db8f762b822b2f4fabf7ff6742c071302f
2023-08-07 16:15:56 +00:00
stephan c7f7b45a15 When converting a Java exception to a db error message, use Throwable.toString() instead of getMessage() so that the exception type's name is included. More internal API renaming for consistency.
FossilOrigin-Name: 2d44720d06d9e50cb037e92981d2473a3ad0b7560f2f5923d428f59de6fd6aaa
2023-08-07 11:18:44 +00:00
stephan be7aef1f85 Lots of JNI internal API renaming, for consistency, and moving-around of utility functions. Make it safe for more callback types to throw.
FossilOrigin-Name: 9a494394b9eb28cf88dc5e7075a4b8c682c8e14fdd6837b595bec8011d7e9e72
2023-08-07 10:59:27 +00:00
stephan becf29a36c JNI API renaming to better match the C API.
FossilOrigin-Name: 6e0bd03d0ba9ee8422853241ba1c4e963d158d1f042855c0cb0026701907896e
2023-08-07 10:01:59 +00:00
stephan 7c26811c2f Rename fts5_api pContext parameters to pUserData, per /chat discussion. This is a cosmetic change made to reduce confusion between those parameters and the two other context-type parameters in that API.
FossilOrigin-Name: 2ca064d8eb37252e16b0fec9924e9ba9289d96a737346431c6ba9cb1c161e5de
2023-08-07 09:44:00 +00:00
stephan 5d48fb1784 Make sqlite3_stmt() Java ctor private - it's only constructed from JNI code.
FossilOrigin-Name: ce82c42f151e38b23945e6f5dd99cb6a77b3c6440508f41abc35e9f6c29cd440
2023-08-07 01:06:27 +00:00
stephan 005baf67d5 Minor internal cleanups and additional test metrics.
FossilOrigin-Name: fa0a6b6e8e6c711585bca30357e465f7a2f08a1c7159ecf23031af1e5158b89d
2023-08-07 00:29:38 +00:00
stephan 8af781fc4d Bind the auto-extension APIs to JNI.
FossilOrigin-Name: 746a5fa079ad80b3c59411202ee601e0b5c50e79e5994d5e464fa06d3c276324
2023-08-07 00:06:31 +00:00
stephan 60745265e1 Rework the sqlite3_open(_v2)() order of operations so that pending auto-extension support can get ahold of the open-time Java state despite the Java/C (sqlite3*) binding not having yet been established.
FossilOrigin-Name: 34da294ab558880e81eebd7d261bc590551d5a7d2855e844695cef6394647ea7
2023-08-06 22:09:09 +00:00
stephan d85f9bf6d5 Completely rework how the JNI sqlite3_open(_v2) and sqlite3_prepare(_vN)() bindings deal with output pointers to give the JNI side full control over the origin of db and stmt handles (necessary for solving chicken/egg situations in auto-extensions and prepare-time trace). Lots of adjacent internal API renaming.
FossilOrigin-Name: 644999caff9db79562d45520d94aaa24ee88c65e397b6fb9c20a4f0e7f84e1a5
2023-08-06 21:29:13 +00:00
stephan af90dcf324 Start adding JNI-side support for auto extensions, but hit a brick wall which requires slightly awkward semantics changes in the JNI bindings for sqlite3_open(_v2)() to resolve, so stash this #if'd out for the time being.
FossilOrigin-Name: 77a32d238e80fe1d237768d88780043a7bd2b3543e6672536254782cbea0039c
2023-08-06 20:01:30 +00:00
stephan 1a2563a950 Doc cleanups.
FossilOrigin-Name: 5f56b007704f2aad4cbc6f0ccd1e1f1c974865971f99451352714ee7e077c284
2023-08-06 15:01:38 +00:00
stephan 5558624b43 Bind sqlite3_config() to JNI for call variants taking (int,int*) and (int,const char *) variadic arguments.
FossilOrigin-Name: 6119289da85ac0c83e2a7236d24bbfff22334d6cf1d852756dc658ad6a75dfec
2023-08-06 14:19:09 +00:00
stephan 9c4c16c80d JNI-internal cleanups and API renaming. Add a C-side java-string-to-utf8 conversion.
FossilOrigin-Name: 672d85795d04131135b1dc6a02d64eceb8b5084217c17766afeca4af23c07ec4
2023-08-06 13:02:43 +00:00
stephan 6a9364a518 Remove doc outdated warning about sqlite3_trace_v2() JNI binding being incompatible with MUTF-8. Use new to-string capability to simplify Fts5ExtensionApi::xColumnText() JNI binding.
FossilOrigin-Name: ebcfc2379be12f76a96f3605b734f406b3354d4c985062cdbfca0cf7e3f31379
2023-08-06 11:05:17 +00:00
stephan 57645b67fa Add a way to convert from standard UTF-8 to a Java string (JNI lacks this capability).
FossilOrigin-Name: 07dd082c9e371829a18aeb574f842891e545e1fc125760238ede7e7e2b6a4262
2023-08-06 10:49:47 +00:00
stephan 1bce6b468e Bind sqlite3_db_filename() and (closely related) (A) add many more docs about the UTF-8/MUTF-8 discrepancy (B) start adding internals to enable us to perform the standard-UTF-8-to-Java conversion from C.
FossilOrigin-Name: 586720fa714ac74491cd85d0c6645242e55e5989ad312ef6e15e0b0acc6906ff
2023-08-06 10:14:53 +00:00
stephan 153288dc89 Bind sqlite3_set_authorizer() to JNI.
FossilOrigin-Name: e0fa03135942cd2fe732a74510d380ba78ab230c452168e638f32b4aee04b3f7
2023-08-05 22:41:36 +00:00
stephan 28e95830ad Refactor the per-JNIEnv cache from a fixed-size static array to a linked list of dynamically-allocated entries. Uncache all per-db state (which is necessarily JNIEnv-specific) when the corresponding JNIEnv is uncached.
FossilOrigin-Name: 9dd8b78419e19e88bc3fbff9bf200390b146b2461af2bb6b93d8467036619e33
2023-08-05 21:35:58 +00:00
stephan 67214a9594 Add SQLite3Jni.uncacheJniEnv(), a way for Java threads to clear their thread-specific cached state from the JNI bindings when they're about to terminate (or are otherwise done using the library).
FossilOrigin-Name: 7468f8761bece58f7ced3d112bbe2fb454432d9c54c9b96cedb5a15bc2926d0f
2023-08-05 20:19:45 +00:00
stephan 27cb7d6a1d In the JNI docs, note that the sizeof SQLITE_TRANSIENT and SQLITE_STATIC should ideally be the same as the platform's pointer size.
FossilOrigin-Name: 7d4ac44ec419ed0474bdb9d237b97660cf0d8faba8fe686f6a914d7bc04dfa3b
2023-08-05 19:20:15 +00:00
stephan 47c5db7f77 Tiny doc typo fixes from [forum:31eb707250|forum post 31eb707250]. No code changes.
FossilOrigin-Name: 8ede50082e7bcf0226a3b42a590f188b5a139bbf081c9eed7ff8f6f2a6a274a0
2023-08-05 19:15:19 +00:00
stephan fdeb2c8121 JNI internal refactoring and cleanups.
FossilOrigin-Name: 7397973a2f191d75b149cf73a6d7ee1798820c1cd37d83af14e565067ede1b04
2023-08-05 12:48:33 +00:00
stephan 695d0b698a Bind Fts5ExtensionApi::xUserData() to JNI and extend xCreateFunction() to accept that argument. In test code, use assert() instead of exceptions if assert() is enabled so that test failures (exceptions) thrown via callbacks do not get suppressed (which they otherwise necessarily are to avoid crashing the host app).
FossilOrigin-Name: e43837377696e468cd31cf71585fe235ffe67a9f4d3b036c5f9d0cb7141d0f57
2023-08-05 11:16:54 +00:00
stephan 4f07be39b7 Add missing Fts5PhraseIter.java to the build.
FossilOrigin-Name: 96281ad0d5b2f020622c4f85f8694886e6a29fb43e1fbeb2a346ed2e94f109fb
2023-08-05 04:30:31 +00:00
stephan ec71e555c4 Bind fts5_api::xCreateFunction() to JNI and demonstrate it with a test.
FossilOrigin-Name: c653bf16cbdccae05ab14059b140191afd5c17740fb78d756d8822986e54b17c
2023-08-05 04:23:27 +00:00
stephan 977b6919f2 Start adding fts5_api to JNI.
FossilOrigin-Name: 14d18fe983c83412d72fd2005a45a2b8c48d347b7bbf8ef9630ae460cff85c32
2023-08-05 01:28:30 +00:00
stephan 0c179dd5b4 Bind the remaining Fts5ExtensionApi methods to JNI, noting that all are completely untested because the higher-level bits needed to do so are still missing.
FossilOrigin-Name: 23383c1dfd240ce47f504dd5c3402c9a31f166fbde5bb72d91309a5655074b33
2023-08-05 00:40:28 +00:00
stephan 195611d8e6 In the opfs-sahpool VFS's importDb() and exportFile() methods, throw if the actually-wrote/read amounts differ from the expected-to-write/read amounts, per feedback in [forum:a4122e986f|forum post a4122e986f].
FossilOrigin-Name: a617ebf4e5d1af1b5b15e9782ad111399caaa3ea7b99bb0c8691c8b4283b6d6e
2023-08-04 16:01:55 +00:00
stephan cc8202b646 More work on the JNI binding of fts5 customization (still a long ways to go).
FossilOrigin-Name: 1a246fd21657f5bb13eeacc4059894ab787ea9a3c45bd9bdd3030a66643d2fef
2023-08-04 15:38:59 +00:00
stephan c7795cfd47 Eliminate code duplication in the two JNI tester classes.
FossilOrigin-Name: 63e7bbe3d5fcfb531f9d7fa88398c1191570e69b5d11adcb9c5e64b8345b4e6c
2023-08-04 13:27:45 +00:00
drh b87278f414 Fix a bug in the jsonLookup() routine used to search JSON. Bug was
introduced by the recent (unreleased) JSON caching enhancements
check-in [a4c1af616e672a0d] merged into trunk at [df099ad713011b67].

FossilOrigin-Name: ba7a66db13b28da0b41f03be825a593608474a5ebda89e12d58d89db709b8f65
2023-08-04 13:19:11 +00:00
stephan 5e208f712f FTS-related JNI refactoring. Move FTS-specific tests into their own class and dynamically load it, if possible, from the main test app.
FossilOrigin-Name: b7a8428fcd969e7a29a23c2dae61883f69501094f2de0f79bbee3c02c672cbf5
2023-08-04 13:03:31 +00:00
stephan b15223bce6 More work towards binding FTS5 customization to JNI. Add Fts*.java files missing from previous checkin.
FossilOrigin-Name: 91263178f463ca4623dd0203696eff6bcfd68abde5d2471be3f5a3edd791c52a
2023-08-04 12:44:06 +00:00
dan b383744835 Change comment on Fts5ExtensionApi.iVersion from "always set to 3" to "always set to 2".
FossilOrigin-Name: b7de399c277394a6a3bbf253d4177aa82753271e5087b98b2c2268f629a7dfdd
2023-08-04 11:10:10 +00:00
stephan c0952c11a7 Start including fts5 customization bits into JNI, but it's far from functional.
FossilOrigin-Name: abaf5edd0430e3301a11bd0acb9ce4b81b310237e1799701411db56ef7605e01
2023-08-04 11:08:25 +00:00
stephan e133a0ec05 Improve internal error handling in the JNI create_function() impl.
FossilOrigin-Name: 2c88390faa108a60c8fb1eb7aad05d90f3daf4cfef14ca73987597aaf7be83c9
2023-08-04 09:53:13 +00:00
stephan a24769454e Resolve the timing/ordering issue of a JS-to-WASM-converted xDestroy() function being uninstalled from WASM right before the underlying native call tries to call it. This has been a long-unnoticed bug which appears only when removing such functions or replacing them.
FossilOrigin-Name: 031c9a76b6ad1572e7a88f4d2d62f206b0d37bd1170e2c8a24248c5ec628f2f5
2023-08-04 08:45:25 +00:00
stephan 444424dab8 Part 2 of the fix from [a0f808363318c00fd1db78b].
FossilOrigin-Name: 5c3104228d75c8fafbe24e46623777f3a8647f1b50267af61d46f7fad0e0434e
2023-08-04 08:41:55 +00:00
drh 1050f1d2a1 Fix harmless scan-build warnings.
FossilOrigin-Name: 65a8716d8a1b7c5cffe9cdd25e4cbfa6528fcb146ff0fe67b0ce771b0537473d
2023-08-03 18:14:09 +00:00
stephan e98fa205f8 Fix a discrepancy between sqlite3_values_to_js() and its API docs/intended behavior.
FossilOrigin-Name: 7b85c68fe252c0aa69724c82a95a4432e33ebdfb879b4b1b2ee2f9b98e9b792a
2023-08-03 16:22:59 +00:00
drh 86d77fd95d Fix build problems associated with SQLITE_OMIT_WSD and SQLITE_OMIT_TRIGGER.
FossilOrigin-Name: de031a5ec863b11f0216d1a114740c8cf4572adb034a5d35ff1a5c07d81df800
2023-08-03 13:30:00 +00:00
drh 038ac625af Fix a test case whose value changes under the no_optimization permutation
due to the recent change to add the ability for SQLITE_TESTCTRL_OPTIMIZATION
to disable one-pass.

FossilOrigin-Name: 976e39c6322f9e51054546e2d9bc135b683d141054c645c9098002cdc310c7c7
2023-08-03 13:07:58 +00:00
drh 9b1334b7d3 Unix builds now assume the presence of nanosleep() in the standard library.
The -DHAVE_NANOSLEEP=0 compile-time option can be used to build on systems
(if any still exist) where this is not the case.

FossilOrigin-Name: 779d5dc8797ea246d0397f7e94b1be716b0baa735e8d9f5a6fc4cffd887a7420
2023-08-03 12:41:30 +00:00
stephan ed86e0a0a7 More JNI-internal docs. Correct handling of a Java exception if a trace callback throws.
FossilOrigin-Name: 306b269a01037bc5c98276276fdb17b37027d1ee0d603183f42a65966245bdff
2023-08-02 17:29:18 +00:00
stephan decc1425a3 Internal doc additions in the JNI bits.
FossilOrigin-Name: b56643644e065b03fe2c3545e02ed3b2f6fd9f77c39aa84a02a00d695e9a12dc
2023-08-02 17:20:52 +00:00
drh c874d60353 Performance optimization for JSON rendering logic.
FossilOrigin-Name: ea0b9aecbaca9a8e784fd2bcb50f78cbdcf4c5cfb45a7700bb222e4cc104c644
2023-08-02 16:06:02 +00:00
drh d3229ce23e Remove an unreachable branch in the ascii-to-floating-point conversion that
was added by [e989a37ff9d5b52e].

FossilOrigin-Name: c4347e4400e96f932ac12f8f22484a2ebce2a578d1b2181977954c432f117bfd
2023-08-02 13:45:16 +00:00
drh b1dcebfea3 Stricter enforcement of the idea that a MATERIALIZED common table expression
is an optimization fence.

FossilOrigin-Name: 354425f8d97437bd156265a6914f98cce91b42abc9e773453ef4d817e308dc9a
2023-08-02 13:38:02 +00:00
drh 26cf4af5d7 Remove a condition from query flattening that is now taken care of by
the caller.  Factor out the reverse_unordered_selects processing from the
main loop of sqlite3WhereBegin() for performance.

FossilOrigin-Name: f068f105fb158634321bf6401f0774c81059932d213a18b627ae98bcffc10912
2023-08-02 13:29:01 +00:00
drh aa250db66a Simplify the select.c logic that enforces the optimization fence around
a MATERIALIZED common table expression.  Do not allow the
reverse_unordered_selects setting to affect a fenced CTE that itself has
an ORDER BY clause.

FossilOrigin-Name: 165b9bda243777ae7fb6f51ee7ad58771c581b82ef9a70268eb9764e9e6eba6f
2023-08-02 12:39:56 +00:00
drh 5e82c06cf8 Never flatten a CTE that is labeled MATERIALIZED. This is really something
of a bug fix because the documentation says that a MATERIALIZED CTE is an
optimization barrier.

FossilOrigin-Name: b7ef9796f548ce43e06673f86bbec68157aae5e4ee8451e1d87ee5f70af1bb27
2023-08-02 11:06:27 +00:00
drh 4178849736 When a query is inside an AS MATERIALIZED CTE, do not attempt the
omit-ORDER-BY optimization.  If the developer specifies MATERIALIZED,
that means he wants the sort to actually happen.

FossilOrigin-Name: aa769ee7477171f26a091cadff98aa4f48da7752ade20b405a95c4525456f62e
2023-08-02 00:09:15 +00:00
drh cbaef88980 Avoid ASAN warnings when converting over-sized long double values into double.
FossilOrigin-Name: e989a37ff9d5b52e0090d59be077ad2260c8df5d4c2c2d8088b1160de64dffd4
2023-08-01 19:10:30 +00:00
stephan 2877c43e6c Filter the wasmfs-specific JS module result type check out of non-wasmfs builds.
FossilOrigin-Name: ef7c121049f77200ecc4da260a90776264af0e84122d0133b4986959aedc653a
2023-08-01 16:41:12 +00:00
stephan 887f6e5082 Fix part 2 of 2 for /fiddle: a make deps problem which caused fiddle to load but fail to run when sqlite3.c was missing.
FossilOrigin-Name: 6011bc26764760af750de68f1553f45382766dd2195633ae267bdb4f9ac75d0f
2023-08-01 16:38:08 +00:00
stephan a08d363f67 Fix part 1 of 2 for /fiddle not running: build flags which were renamed during refactoring.
FossilOrigin-Name: 214ffeeff77997c3352d30aaa6e51e2dea9706f0d08f73f4eacfad1864817fc6
2023-08-01 16:17:00 +00:00
dan c1a07d3bab Fix an assert() in fts5_index.c that might fail when dealing with corrupt records.
FossilOrigin-Name: aa55c83f35c2ab134e0842201e46e021079283f9c65595c86664060b3aa8d715
2023-08-01 11:03:06 +00:00
stephan ed77e13f05 More docs for the Java side of the JNI bindings.
FossilOrigin-Name: d8e9bcee96b90d56701f7907a8bd48853211caf757e1aa8decc7ed25eece6770
2023-08-01 10:19:05 +00:00
stephan e209d144c3 Internal exception-handling cleanups in the JNI bindings.
FossilOrigin-Name: 057b1d4f6ffff73c120566895f61ea3fd8118968464d67ec9262096a7aa03f39
2023-08-01 09:44:26 +00:00
stephan 0bc72d2b91 Minor JNI-related doc tweaks and code re-ordering.
FossilOrigin-Name: b663b27e425966f34fb62482a18048f0e2934380e5c411ae3627f1fe6a765c04
2023-08-01 08:48:13 +00:00
drh b9894f88b8 Remove redundant comparison operator.
FossilOrigin-Name: 018984329892c243b21adb10aa8efc7f49313ecf80477e6b61dc6365f62c489d
2023-08-01 01:07:45 +00:00
drh d192b4441d Restore part of the UPDATE one-pass optimization that was removed by
check-in [2c56b984a0bd3be5]: only disable one-pass if the WHERE clause
contains a subquery.  Allow subqueries in the SET expressions.
Fix for performance problem reported by
[forum:/forumpost/8ab195fd44e75ed0|forum  post 8ab195fd44e75ed0].

FossilOrigin-Name: 42916af9fc0f379a608a08db894400bd735a28e26ab1ffd604d1fddfbdb3ec0c
2023-08-01 00:34:03 +00:00
drh 00d635dda4 Restrict the new column cache to table-btree, which is the common case anyhow.
That way, writes to indexes do not need to clear the column cache.

FossilOrigin-Name: 659284ab0e22a4746c1337b3489e7b166d497fb7e5301e24dc115d2b0c4e097d
2023-07-31 22:03:24 +00:00
drh 98ab970bac The original column-cache implementation from check-in [ab1edcc7fedcf279]
(merged to trunk at [771fe35074b50b8d]) is unsound.  This check-in fixes
the issue.  Had to give back a little performance, the optimization is still
a overall win.

FossilOrigin-Name: ec95e970fb737adf0fab3cb4363040b036949e5eb966fc2d030a20f95e2bde60
2023-07-31 20:02:11 +00:00
drh 1f097a2f29 Add bit to sqlite3_test_control(SQLITE_TESTCTRL_OPTIMIZATIONS) that will
disable the one-pass optimizating in DELETE and UPDATE.

FossilOrigin-Name: 22f6dd5cd32ed9e7b932fdba98aa8652fde999df79f4995d3d8ac1acf430cf53
2023-07-31 17:39:36 +00:00
drh c8c6763782 Improved debug tracking of the Parse.nQueryLoop value.
FossilOrigin-Name: 4a8a17498ecaccca5e5aab65d4596cbca9799547f50683237d08c1fa3556f1cf
2023-07-31 17:20:12 +00:00
stephan db6f0bef91 Experimentally change the JNI sqlite3_trace_v2() callback type to have more convenient access to the current Java-side sqlite3_stmt at the cost of some uncomfortably fiddly current-statement tracking in the JNI layer. Subject to change.
FossilOrigin-Name: 459db332af6ea358b42bac096b9d26f1045b9ec32fad8463bca06807b2396b2c
2023-07-31 13:52:46 +00:00
stephan e3e9bdcd0b Add some JNI-internal metrics, accessible via passing -v when running Tester1.java. Document an OpenJDK bug which leads to incorrect -Xlint:jni warnings.
FossilOrigin-Name: a5d68a6b64abe3c2dfc3a32157f70fd8a4ad89feef2510b3bbb2d86b325d51ae
2023-07-31 12:10:32 +00:00
stephan 8f714e21a0 Correct a mis-cast in the JNI wrapper which just happened to accidentally work. Update JNI binding of sqlite3_context_db_handle() to return the bound-at-open() db instance instead of a new/temp proxy object.
FossilOrigin-Name: 9faca5d9ed4a749421e08bd1da8b7672c0fd31366124fdb613c46e19dece0fc1
2023-07-31 10:55:30 +00:00
stephan ede8900912 Add another test for JNI-bound scalar UDFs.
FossilOrigin-Name: 8b322d92e247be606f83977767dc361ee4f7bc819122a630bdaa1110177db9b8
2023-07-31 10:42:05 +00:00
stephan e2e696d11f Update some internal docs for the past two checkins. Add a way to dump out some debug info about the current JNI environment.
FossilOrigin-Name: ac9b8bb1e64450d980e2986084996549ae5c59e68c9f0c4c69539c239b64468b
2023-07-31 10:22:34 +00:00
stephan 452108b4fc Refactor the busy-handler-specific JNI hook type to use the generic hook type.
FossilOrigin-Name: d9efdc6dd20a34bfdaad5d4bf8e67cce7e35238299eb91e4459d59fda11978a6
2023-07-31 10:08:36 +00:00
stephan 46c46e400a Refactor the collation-specific JNI hook type to use the generic hook type.
FossilOrigin-Name: 02c1d3b6501fedf3d6e6d1ca60699df268522182c5ba3b49ae8f4691499ef0fc
2023-07-31 09:45:49 +00:00
stephan 41a7ac637a Internal JNI doc tweaks.
FossilOrigin-Name: f4aa2c82882cb6be1fd52977de19fd03c2e38abb857b520f951b32d610972ab6
2023-07-31 09:28:15 +00:00
stephan 9c1c6da930 Consolidate triplicated xDestroy()-calling code. Remove some unnecessary casts.
FossilOrigin-Name: 24c0763d0e025187c74002ffee11fd48d3cd7b40e01469d28484bb67f701884b
2023-07-31 07:15:25 +00:00
stephan 09947d0aad Internal JNI API renaming.
FossilOrigin-Name: fcfc070673cef2f657f4737f096678439ed7c011fb2e5391e0721f82f5d8af51
2023-07-30 18:41:25 +00:00
stephan 91710673e7 Internal JNI refactoring towards consolidating support for callbacks with and without finalizers.
FossilOrigin-Name: 120983a570d6de055cef9d916096de3410897ea9f46d23ea6eff1f9b549e423a
2023-07-30 17:24:01 +00:00
stephan 72bb29283f Mark the Java wrapper classes of the C-native types as 'final' (cannot be subclassed).
FossilOrigin-Name: 4fd3d93623d67c25fb8a490e0d4ea56d531d858067011ab1b28cce694098feff
2023-07-30 13:47:51 +00:00
stephan 13b059025f Minor tweaks and optimizations in the JNI bindings.
FossilOrigin-Name: 41fb5898f1a78d9fd85a020e28a6048a7359b54e35632e9072917cbdbcd8b07d
2023-07-30 13:30:52 +00:00
stephan 502a5c2e26 Bind sqlite3_collation_needed() to JNI. Related adjacent cleanups and fixes.
FossilOrigin-Name: 16ff167691733350907d2d995c774a885214acd0fe8ec491c16b786f00fe85d4
2023-07-30 11:36:41 +00:00
stephan 45fe10d02b Internal JNI refacoring to support the pending sqlite3_collation_needed() callback. Correct a bug in the linked-list handling of PerDbStateJni which triggered an assert().
FossilOrigin-Name: 7ac6614e69b03304d09745619ed83f12c7eb775aaf4a636a79289b01642ddd14
2023-07-30 10:47:38 +00:00
stephan 9c113744d1 Incremental checkin to minimize the diff while narrowing in on an assertion caused by refactoring.
FossilOrigin-Name: 2d7a91b1396d87852f1153ab7af7385514a9537cb64ba3bbd0faba2d28704214
2023-07-30 09:45:54 +00:00
stephan 70576de5e0 Consolidate some warnings for java callbacks which must not throw but do.
FossilOrigin-Name: 5e592ed2dfc89225fff3a1c76509adc799a238282413984e0c4b32af18525d18
2023-07-30 08:12:15 +00:00
stephan fecad503a9 Bind sqlite3_update_hook() to JNI.
FossilOrigin-Name: a5bbaa9017839f8d8b92bfb44472d4c60fa3037bfae7846dc8350262c1332cde
2023-07-30 07:44:03 +00:00
stephan b162c68d62 Bind sqlite3_rollback_hook() to JNI.
FossilOrigin-Name: 5f8ee44098d74ac2b98e4dd43ad80d3b919528358b3f992b425af7fa6262dcee
2023-07-30 06:44:21 +00:00
stephan 00c0c5883a Add commit hook failure to JNI tests.
FossilOrigin-Name: be4459b495cc2555e4d8ca24dd56c3da1036d25af7cc077bbc3d93c2a3e8c40b
2023-07-30 06:00:53 +00:00
stephan f52de0fdaf Bind sqlite3_commit_hook() to JNI.
FossilOrigin-Name: c687297fcae082cbd7f9258c43f3841fd34904d8b62b0adf3cd61fcddeee483d
2023-07-30 05:50:34 +00:00
stephan ca379859c9 Bind sqlite3_progress_handler() to JNI. Add some metrics to Tester1.java.
FossilOrigin-Name: 437ecfe8abf8d294d429d191d811da6148e0b2ebb74cf66998480bfc8ef58bdf
2023-07-30 04:31:56 +00:00
stephan f0c701c410 Add tests for SAHPoolUtil.exportFile() and importDb().
FossilOrigin-Name: 4aa88b8fb1c99137ed7531e5c82a241738a2b0554f7af82c0ed0ce8d16a0879a
2023-07-30 03:54:10 +00:00
drh b47e1c7866 Do not try to add MEM_Term to an MEM_Static string either.
FossilOrigin-Name: 8920dcef1f75f578e3f274ab2b980a83d6ea6b761d99fc48410b26886170597b
2023-07-29 22:58:47 +00:00
drh e809d84618 Do not try to apply MEM_Term to a MEM_Ephem strings.
FossilOrigin-Name: 2a378a66783ce4d749f7e1e7d08d8404a4aa78d47aeeefd6794b6753188438f6
2023-07-29 21:22:26 +00:00
dan 808cf29b55 Fix a buffer overread in the fts5_structure virtual table (test code).
FossilOrigin-Name: b837aff79cd159061b46af59eaf96a1a1920eeece27e9e27931cf3387068d96a
2023-07-29 20:13:19 +00:00
drh fb546c0bf4 Simplification to the sqlite3_stmt_explain() implementation.
FossilOrigin-Name: dd5eb4c7f71c658aff8c2f11ff163c5e79f0ebfd49fd13ca8e28381fe63cc82f
2023-07-29 17:05:35 +00:00
stephan 5d800cdb6e SAHPoolUtil.importDb() now accepts either a byte array or ArrayBuffer.
FossilOrigin-Name: 3848f04e542e2f0f8975e82632af598aac3b60312bd244b0472f7ddf1dda77df
2023-07-29 15:53:53 +00:00
drh 0c6b869263 Enhancements to the new sqlite3_stmt_explain() interface implementation and
documentation.

FossilOrigin-Name: dc98ac6b6de27fc096a715e650067154a545709cf1eecd6c2722ab30a91c2d1d
2023-07-29 15:31:48 +00:00
larrybr 104e225074 Fix doc error noted in [forum:/forumpost/11fc7308d5e607f2|the forum] and a misspelling.
FossilOrigin-Name: 6b79cc7dd07d41da6034386b39c0dff32f80a977db42a9602330ffd15706ab9b
2023-07-28 21:38:50 +00:00
drh 3e053c50b4 Do not allow the same JSON parse to be input to both arguments to the
json_patch() routine.

FossilOrigin-Name: 8e781c6742792b3899ca60efc49c7eaee1df9f6b03d27c254b0e148b82946e19
2023-07-28 20:59:25 +00:00
drh 9a897cb0de Merge the latest trunk enhancements into the wal-shm-exceptions branch.
FossilOrigin-Name: c44fb4d06591ef4872f015aa9e81ce8b93a38bd9143cf20db38cbe5c77a0d71f
2023-07-28 19:07:25 +00:00
stephan 33c8e9d233 Internal JNI binding docs.
FossilOrigin-Name: 991c66197e4dc7297fce3b20a1b4846873bcd4ce8add36aac71bd2e0e73c207b
2023-07-28 18:44:11 +00:00
drh 58b5921ca4 Add the sqlite3_stmt_explain() API.
FossilOrigin-Name: 0443c0ef85f8f6f3efdee68bf59126c9459f08a58458756f8cbeeaf043d43a86
2023-07-28 18:37:13 +00:00
stephan 888e14f551 Fix an incorrect file path in ext/jni/README.md
FossilOrigin-Name: bcefa2df563260933c7ab5df90872580f71010c11419f6b1de7b1e2747237ff8
2023-07-28 18:02:02 +00:00
drh 58bf53d573 Minor tweaks to the JSON performance measurement documentation.
FossilOrigin-Name: 0bed957e46aa3bf6a70292ae100de0459486c1469dd03de61207a708cc59a594
2023-07-28 16:12:39 +00:00
drh f69084aaf7 Fix minor typos in the JNI README.md file.
FossilOrigin-Name: e21cf2e8f13465dbff33f9c21580752c1d8c077d6a253f56f04dab0d47eb99b8
2023-07-28 15:58:09 +00:00
drh 8a056d486f Enhancements to the JSON performance testing scripts and instructions to
cover the recent optimizations involving updates to large JSON strings
that indexed.

FossilOrigin-Name: f9213289d09adfb0461d9644e80c7e889f4bae51563ae2e575a2c95336052bcb
2023-07-28 14:20:31 +00:00
drh 859dd4fc2f JSON performance improvements: (1) Add a cache for large string values
on OP_Column to avoid having to extract the string multiple times.
(2) Internal JSON caching improvements.

FossilOrigin-Name: 771fe35074b50b8d4a6583e61c53871b2445f7a58c82f3fc3bf6776e562e63af
2023-07-28 13:52:08 +00:00
stephan ab12b750de Correct a recently-introduced falsehood in ext/wasm/README.md regarding ssh port forwarding.
FossilOrigin-Name: 2a3f3a9cf28849c99d83c256f813405a2f47f84532e1192d67bb7aa655dd98a3
2023-07-28 11:52:58 +00:00
drh b1926192fa Mark an unreachable branch as ALWAYS().
FossilOrigin-Name: 8fd06d1151c521515d9f7181575a04e451b522335135a27248052b067d4005ae
2023-07-28 11:30:07 +00:00
stephan c7f602497b Minor java cleanups recommended by a modern IDE.
FossilOrigin-Name: f92a9bbabebc33a771835e259b04a57d07add1eacf3d9aec5fddf929ce8a3022
2023-07-28 10:37:17 +00:00
stephan 069029011e Add Java tests which ensure that xFinal() is called with no aggregate context when the result set is empty.
FossilOrigin-Name: 7ac8c66a6b62d6bb345e8b9957a26de463019ae7d30ff442f19482c3a6827fc7
2023-07-28 10:01:01 +00:00
stephan f326577619 Tweaks to the Java-side UDF docs.
FossilOrigin-Name: 48e0079d8db675a8761f27348bc6795aa6b6e49a3f76bd4adece83309dcf2aff
2023-07-28 09:54:39 +00:00
stephan 09c2640fe3 More Java docs about making use of the aggregate context. Change the JNI mapping to set the sqlite3_context::aggregateContext member directly, instead of via a superflous setter, because that way is faster.
FossilOrigin-Name: 7af0cb998f7161296d5e5e50a42e9db26ec13c145c61194a999a1a0104818d45
2023-07-28 09:25:05 +00:00
stephan 75d3b1b5a2 More docs and cleanups related to the aggregate UDF state. Correct the OOM check to behave properly if xFinal() is called without a matching xStep(), xValue(), or xInverse().
FossilOrigin-Name: ff53f1ccdc1780f2d9bd5f59804a76dbdf4f6b70696d3a7dbdbd96d1f8f6fa5c
2023-07-28 01:51:14 +00:00
stephan 7d207bf483 Add an OOM check to the previous check-in. Minor internal API renaming.
FossilOrigin-Name: 6b56e4d62b4945e52978d00aa8e2984faa731c92a7e002e81524fcfcf8ba0cce
2023-07-28 01:19:44 +00:00
stephan 48a8352a39 Add support making use of sqlite3_aggregate_context() (in a roundabout way) from Java to accumulate state within aggregate and window UDFs.
FossilOrigin-Name: 640574984741c7a9472d7f8be7bce87e736d7947ce673ae4a25008d74238ad90
2023-07-28 01:12:47 +00:00
drh 93269a9e41 Remove some unnecessary and incorrect code that was mistakenly added in the
previous check-in.

FossilOrigin-Name: d3f458dcc379dd7648262f52579ec55ba931852065ed278367a0629228d0ed45
2023-07-28 00:54:38 +00:00
drh caf7e26c4f Switch to using jsonParseCached() for json_patch().
FossilOrigin-Name: 2ed9c59e6a51037e63485d0a92dae25443116beddfca1ac73d2f5d5be38d4ad3
2023-07-27 23:51:36 +00:00
stephan 8ba5d79c35 Reformulate jni tests to not require the -ea jvm flag to enable assert().
FossilOrigin-Name: dc356667a8f4fa31a3fef1ae35873d834d27fd6a9f0818d6fb85e4751fde9fe5
2023-07-27 22:53:02 +00:00
stephan fd022a23a5 Add locking test for BusyHandler and clear up new -Xlint:jni warnings introduced by the busy-handler binding.
FossilOrigin-Name: 7dcde2bfce54b18f391776fa1cb93c0ff6153634bedcab0007b374c06c4d4079
2023-07-27 22:42:25 +00:00
stephan 1a95091b49 Bind sqlite3_busy_handler(). Correct mapping of pointers for, and cleanup of, JNI-level per-db state.
FossilOrigin-Name: 524747796a30a5c1c6c7567b49ffb1e35e2626c73e09c335c0ab74d4ddb5f005
2023-07-27 22:05:39 +00:00
stephan fd250c2cb5 Replace some www: interwiki references in the JNI readme with their full URLs to make the doc more portable.
FossilOrigin-Name: 63ce0c9bdde210cf2f8b6099ae5c73caac18e6debc13c2f77090b77f3de72beb
2023-07-27 20:32:16 +00:00
drh 6bc4baf313 If the input JSON to a json function that uses cache comes from an RCStr
value, then use that RCStr value in the parse rather than making a copy.

FossilOrigin-Name: 509ae9c1470dd79d320e84371e1e6662fb85fa0571df5ed8c4d946d10cdfe821
2023-07-27 20:28:29 +00:00
stephan eead0ffef8 Merge trunk into jni branch for compiler warning fixes.
FossilOrigin-Name: 0514fd340ae15a95760d50c747d6fb9eae5109cb5045eeabc2bc199be0a5ae35
2023-07-27 20:12:48 +00:00
dan ea6bccaa44 Fix a couple of compiler warnings in fts5_index.c.
FossilOrigin-Name: bf71faa2a1d29ea762c4d2485522d6f4f8a5a7166981a92d3ba9c96ccbbe1213
2023-07-27 20:08:44 +00:00
stephan dff3443939 Initial check-in of JNI (Java Native Interface) bindings for the core C API.
FossilOrigin-Name: b5374b9ef58fa0be80aefccde0721f5599fb820464b13940b6361b9aa09a59d5
2023-07-27 20:02:49 +00:00
drh 594f5e24c4 The OP_Column opcode caches large column values coming from overflow pages.
FossilOrigin-Name: ab1edcc7fedcf27922d5db4bc1bc673b1495ca9c66eb6debdda7b7776c068888
2023-07-27 19:39:53 +00:00
dan 9cf8961557 Add the contentless_delete=1 option to fts5. For creating contentless tables that support DELETE and REPLACE statements.
FossilOrigin-Name: d66b182d2bc6ce0772e69401b7affe1adbc1b128c4631cb3c17f98dde72af00a
2023-07-27 19:13:35 +00:00
drh 95daf355a7 Merge trunk enhancements into the json-opt branch.
FossilOrigin-Name: 5739a16ad270a5aadcbb46b28c34fa6ba975422788dcbccb1a8e0d1e6ed75144
2023-07-27 18:19:46 +00:00
stephan 9fcca3d27c Dynamically determine whether the wasm.xWrap() argc check can be applied, depending on how the wasm environment exposes its exports.
FossilOrigin-Name: fd59226b34fffb1479fb2d7bd7c0aff982aa4a1a73e6c0d81de6eaf9c075998c
2023-07-27 17:50:10 +00:00
stephan 4cfd54256d Accommodate a breaking change in emcc 3.1.44.
FossilOrigin-Name: 2c5dd34199f5bcf729be814b8b46d9997821fe3a39ab12779c93df1bb2fd108d
2023-07-27 01:38:19 +00:00
drh 93853a4846 Fix a performance regression in JSON associated with generating small
snippets of JSON from a larger JSON string.

FossilOrigin-Name: 837f2907e10b026f6db1ca2d44b4bf60a6f069bf534bf369ad9b5c513cb0c6e4
2023-07-27 00:21:59 +00:00
drh e94e132994 Reduce the number of memory allocations when parsing JSON.
FossilOrigin-Name: 9edd67162113df57dae21d4683f9495611e2cf4717c6d12f5b7b8e44156d5fe3
2023-07-26 23:22:32 +00:00
drh f7a164f345 Minor changes to make coverage testing easier.
FossilOrigin-Name: ec8b43382e5402e15d9f2dda3cf21ac8be8c1589ddbe6c9433c33eef0036f764
2023-07-26 21:53:09 +00:00
drh 3f3dc2f5d8 Change a switch() case to default for coverage.
FossilOrigin-Name: 04f497074b9210326030f36107a43d6490a2a59c8a574e2c5429cd9bde681bf7
2023-07-26 19:22:43 +00:00
drh beb3fb6904 Make sure jsonReplaceNode() always leaves the JsonParse in a consistent state even if an error is encountered.
FossilOrigin-Name: 01d52232dd6fbd253e77419a17df3df83d49434792d288ef96e14739a89cef3b
2023-07-26 19:11:47 +00:00
dan 5e22dfb6a5 Add a comment describing the contents of the ExceptionInformation[] array for win32 EXCEPTION_IN_PAGE_ERROR exceptions. No changes to code.
FossilOrigin-Name: 8e20354242d5f34a90b6c00d9034535ef6f6086ee4971230c7fd69985e2777d2
2023-07-26 18:34:34 +00:00
dan 5028dd89ab Add the ".scanstats vm" command to the shell tool. For profiling VM code in SQLITE_ENABLE_STMT_SCANSTATUS builds.
FossilOrigin-Name: 0cbec3990d4101142bfb831f8e6527b73baabebbd30fa7f59275b217dbce6a8d
2023-07-26 17:51:05 +00:00
dan a1016b4275 Improve the output when ".scanstats vm" is enabled.
FossilOrigin-Name: 7df08fd35e9d4bc471aa9fbc4c81d2ebcfd2be6c4c38143342b3d9d727c9df22
2023-07-26 16:41:23 +00:00
drh 9095063e14 Improved caching and reuse of JSON parses.
FossilOrigin-Name: df099ad713011b67b09cb1e5f0fa2e6b45f9cee0ce9d3c118c5dbca3563d20a5
2023-07-26 14:18:06 +00:00
drh 7286c59896 Change the debugging "json_parse(X)" function so that it shows a more
complete description of the JsonParse object on standard output and returns
the mimified JSON.  Former behavior was to return the text of a decode
of the aNode array.

FossilOrigin-Name: 1bf85d4e388714a88f8940dcdec353c3e0267456697eff6963d34637912aecc9
2023-07-26 13:17:43 +00:00
drh 2b57197066 More comment improvements in json.c. Do not run jsonLookup() following
an OOM error.

FossilOrigin-Name: cd5fda8c2e354da7458b7c1a82ff18c5946f8dab16095bb0293bec57f6804f17
2023-07-26 11:53:14 +00:00
drh 7725370ff4 Improved comments and other cleanup for the changes on this branch.
FossilOrigin-Name: bac953a80d1a541e7a12aef00c86c002133859237143ad670b39ea19799a8900
2023-07-26 11:43:39 +00:00
stephan 8c7ee455ff Remove the batch SQL runner from the JS build's 'all' target, as it's long-since unused and adds noticable build time. Add makefile comments about JSPI.
FossilOrigin-Name: b8f708e35d4fa027d12089ac7c5589c36da5f68b98cf9792bb11276bf233a860
2023-07-26 11:41:41 +00:00
stephan aef4fbf203 Add SAHPoolUtil.getFileNames() method, and tests for it, per [forum:a3da1e34d8|forum feedback]. Add a test to demonstrate that two SAH pools can coexist so long as they have different names.
FossilOrigin-Name: 72dc3f8c3255186ec412412b685b0b51ddcd08240f2353ac742fc7da8c23568e
2023-07-26 11:11:39 +00:00
drh 9a184daad3 Fix jsonForceRCStr() to also add the NULL terminator.
FossilOrigin-Name: 134b01f37f8f741d7f7b7eda81384695d1cbe4c39751d87f08832d5c9afdcef2
2023-07-26 11:00:47 +00:00
stephan 3ba6914121 Reformulate [907dfc4a7aa1] using awk instead of sed for better cross-platform portability.
FossilOrigin-Name: 82ff7cc6a4b0331677be87bc069da414a56fd531bae402d0f0808b5d2b0d45da
2023-07-26 07:57:55 +00:00
drh a757026c5e Merge the latest trunk fixes into the json-opt branch.
FossilOrigin-Name: ef4e1664d159b98854d9aa580b0bb942f1726f32a190e2ea659c171131ddce9a
2023-07-26 01:15:34 +00:00
drh 02e683e467 Add the concept of an RCStr string - a "Reference Counted String". Use
these to keep track of big JSON strings that might need to be used in
multiple places at the same time with different lifetimes.  Add two
different JSON string values to each parse - one with and one without
changes.  The net result is a large reduction in the number of parses
that occur when doing an UPDATE of a indexed JSON field using
json_replace() or similar.

FossilOrigin-Name: a4c1af616e672a0d4d04f2652e645617758231598fb7161b956883512303ae87
2023-07-26 01:12:08 +00:00
drh 44f53b9647 Extend the enhancement to json_set() and json_replace(). Clean up cruft.
FossilOrigin-Name: 2dbb22c75e86f2e3ced38ac14b4943570d5c2f86cd5e37e875bf0c863be28836
2023-07-26 01:05:08 +00:00
drh 0f200bc580 Add the JsonParse.zAlt field to old revised JSON text after a change.
Demonstrate that this elminates the need for reparsing after a change
by using it in the json_remove() function.  This is an incremental check-in
containing lots of cruft.

FossilOrigin-Name: f930b139d6db0ee799bc90397b225175103c4bf22923d1c9cbcd32509adc1738
2023-07-26 00:48:45 +00:00
drh 59b8e666f6 Clarify ownership of the various objects involved in parsing JSON.
FossilOrigin-Name: afe02a398a16d51bd7482b6fbe2fbd15d9ac4fd9cdbc9d2bf81f38b3391fc567
2023-07-25 20:26:47 +00:00
drh 440e696d5e Incremental improvements to JSON parsing - trying to fold in the RCStr object.
FossilOrigin-Name: 4cb15d934a85ebc290fe6dd8cd3bd47b159561ca75d72bbffef30b9ea4623b09
2023-07-25 18:28:03 +00:00
dan 0c39b13e46 Fix a harmless "set-but-not-used" compiler warning in sqlite3session.c.
FossilOrigin-Name: 54b3c43fdfdaca6b129a5f0ee93c34eb001663775d33c087066650f5e164d1c1
2023-07-25 17:54:25 +00:00
dan cbf4a0fa8e Merge trunk changes into this branch.
FossilOrigin-Name: 719973d7f5a47b110e9919fcb96d21feab1e41356dbb3ec674c1116c17bbb778
2023-07-25 17:11:29 +00:00
dan 5326953e57 Add extra test for 'deletemerge'.
FossilOrigin-Name: bc33cff4203cef12518e0f43d380a06d53d67c725fb96cfe6e934b7dc97a7efd
2023-07-25 16:48:54 +00:00
dan 3f874b58fb Change the name of the fts5 'delete-automerge' option to 'deletemerge'. And add tests for it.
FossilOrigin-Name: 1079300db2a7d1fbc86a01c215c234a3af64889c5396e6da63ff4f3c7efae4c5
2023-07-25 15:48:58 +00:00
drh 4bca124845 Fix a minor problem with error reporting in JSON.
FossilOrigin-Name: c456e4a8999066cd96246327101b3cca78294511a71a2ac07939bb702bfcb5f4
2023-07-25 15:43:01 +00:00
drh f02cc9a324 Create the new RCStr class of strings and try to use them for JSON storage.
FossilOrigin-Name: c1b8725089bb3d006ec69add28f4fcb3f4e79412c7f438b5b1067c2227e77b9c
2023-07-25 15:08:18 +00:00
dan 039d494d5f Further tests for 'delete-automerge'.
FossilOrigin-Name: ca26c7a37a7e680be633f43be28f8877bdf9917448ea51c3bedc9b2352a00601
2023-07-25 13:53:42 +00:00
stephan 322a325703 Reformulate a (sed -i) construct in ext/wasm/GNUmakefile to account for Mac's sed -i being different than GNU's.
FossilOrigin-Name: 907dfc4a7aa129cdcedeb3ba2d75e1b68a8f22c2545ee1c8cf7d705041644e5c
2023-07-25 12:26:05 +00:00
drh b715fe9d80 It is an error to try to insert a BLOB value into JSON.
FossilOrigin-Name: 28c6e964b1e2257527df02b352f02e135f7c4b764b8b41eda6461f9538f5d042
2023-07-24 23:27:05 +00:00
drh 3d2a559ecb Clean up some #defines in json.
FossilOrigin-Name: 327fff501e36f75d4901c520123c5ca45e0e0da1d9cc8fa8fa877ceb68c686d2
2023-07-24 22:45:59 +00:00
drh 27553579c0 Rework the JSON parse structure to facilitate better caching. Passes all
tests.

FossilOrigin-Name: ecdcb1ded76e9a0591bf7a2009679f49fc3aa639d3cc12406c6d29243ed8e1c5
2023-07-24 22:34:26 +00:00
dan 24730de8d1 Add the fts5 'delete-automerge' integer option. A level is eligible for auto-merging if it has a greater than or equal percentage of its entries deleted by tombstones than the 'delete-automerge' option. Default value is 10.
FossilOrigin-Name: b314be66b9ac0190b5373b3b6baec012382bc588c2d86c2edab796669a4303c3
2023-07-24 19:13:06 +00:00
drh 9125d5ab6c Incremental progress toward improved caching of parsed JSON.
FossilOrigin-Name: f2c063884685a79d5a787590447c292f51e898a98c9508159c788f505227ba85
2023-07-24 17:59:25 +00:00
stephan c76a8862a7 Resolve an ES6 module export construct which is incompatible with node.js, as reported in [forum:b9680fa9ad|forum post b9680fa9ad].
FossilOrigin-Name: 80927c3913561dddf75cf73be871d93ae06b16f83e8cc36fc360765014209615
2023-07-24 15:41:58 +00:00
drh 5202b7ca75 Update the latest trunk enhancements into the wal-shm-exceptions branch.
FossilOrigin-Name: 3187ee3f69fc28a259ba0e951ac10a65c07ef2c3866acbefaf9544333a930cc6
2023-07-24 12:59:53 +00:00
drh 7d70571142 Merge the latest trunk enhancements into the json-opt branch.
FossilOrigin-Name: 00bfc4918be49ac74a3e7851c88ab7ec226e6a37853f8ad4c77f758751960456
2023-07-24 12:37:23 +00:00
stephan 479cfd5af3 When writing the filename header in a opfs-sahpool file, ensure that all remaining bytes in that part of the header are zeroed out to avoid the downstream problems reported in [forum:d50ec48a293988a5|forum post d50ec48a293988a5].
FossilOrigin-Name: c1b080e39397c983c13a5e79303223827de7b4946c18a79396851ec1814782f3
2023-07-23 22:14:41 +00:00
stephan 2ecadd8869 Add the opfs-sahpool sqlite3_vfs implementation to JS, offering an alternative to the other OPFS VFS (with tradeoffs).
FossilOrigin-Name: d2e602cda44bf35e76167143262b4f91826d25780d0e095e680a31d5dedb2018
2023-07-22 19:57:42 +00:00
dan 2159292ce0 Integrate contentless delete with auto-merge.
FossilOrigin-Name: 85c1589ab1fc69d1eef4bbc1bdefa2b10af5f6b9c08e813130b93829b592f416
2023-07-22 19:47:46 +00:00
drh 5d03b1610f Do not read past the end of a text buffer looking for a zero terminator, as
that space might not be initialized.  If the buffer is owned, just set the
null terminator.  This is a better fix for the OSSFuzz-detected
use-of-initialized-value problem.

FossilOrigin-Name: 931bccb0cc290b8bf3027641e7a7fac30e3244d7dc84aa9e38b24b7e9544ca06
2023-07-22 16:37:28 +00:00
drh 61a5b6e3bc Fix a harmless use-of-initialized-value warning from OSSFuzz that results
from recent large string optmizations.

FossilOrigin-Name: 1a6b3dd1c40277a0d0f0bb562ddc4868aadd632fc2d29be1b17bb33fc22c46c8
2023-07-22 15:21:41 +00:00
drh b2ee272075 Performance enhancements for queries using very large string values.
FossilOrigin-Name: 3661b9cd98565ea23b0cafd030668f09885dcbd54eb12b7b3cd85c289aef4c3c
2023-07-21 22:28:34 +00:00
drh 21f22f9088 Mark an unreachable branch as NEVER().
FossilOrigin-Name: 414a4d660c3b505640227066b468db1268fcb1bd3fad931644a769617c6c00cd
2023-07-21 22:22:13 +00:00
dan 330e36c2c6 Fix fts5 incremental optimization so that it too can handle an index that consists of a single segment with one or more tombstone hash pages.
FossilOrigin-Name: e61c9b083f5e0b6b6ee18f9394581ad816f445dbfb72ed1fe954f4182755a576
2023-07-21 21:10:33 +00:00
dan 4e0c157d02 Ensure the fts5 'optimize' command correctly rewrites any index that consists of a single segment and one or more tombstone hash pages.
FossilOrigin-Name: f4926006b371d9a1439a25384bd50a50c2f1c03f75a7c2c3134ae72abb971c91
2023-07-21 19:33:35 +00:00
drh 89f07b1276 Fix harmless compiler warning introduced by the previous check-in.
FossilOrigin-Name: 06f829e9e57c5aa495f519ad7bb379be611e8e21ef60d36e63d055e45f7a6117
2023-07-21 19:17:26 +00:00
drh 028acd974f Avoid a potentially large strlen() in sqlite3DecOrHexToI64().
FossilOrigin-Name: 5413b02bb629b9cbb76f7e688e94ebcf24276c01436d3feb73ff57c036e1d2aa
2023-07-21 18:38:59 +00:00
drh 569700a72e Further improvements to large string handling in relation to JSON.
FossilOrigin-Name: 1e5df0aa3dae5cadbf1d07c718ae2a5212543300b68e49d35e8c96855a7f619c
2023-07-21 18:09:07 +00:00
drh eee8687a9f Fix harmless compiler warning about signed/unsigned comparison.
FossilOrigin-Name: 75cc3c89ee2dcfefa9421ce60bee77e85d2895918e8c5cfd05c434f8932a99b5
2023-07-21 15:07:56 +00:00
drh 7a2280fe65 Multiple optimizations that try to preserve or infer the zero-terminated
property of TEXT values.  Avoid unnecessary copying of text values destined
to become function parameters.  All changes help improve performance of
doing UPDATEs on large JSON values that are indexed multiple ways.

FossilOrigin-Name: d0278cdedfa04fb0b61838ab9622be8a2c462f58d5c3ebc4c5f802a727d0974e
2023-07-21 15:01:53 +00:00
drh c8f4fec2de More infrastructure changes towards improving JSON cache performance.
Incremental check-in.

FossilOrigin-Name: 1955e66cfc4614df97b8d68b0e662f309513d62dc8aeec71af5a54e66b79c707
2023-07-21 11:09:53 +00:00
stephan bfe6dd0100 Minor internal cleanups in the opfs-sahpool VFS.
FossilOrigin-Name: 74ad31e2908af8225b7aa527dbcd1877423d58163e365317a78453b31e322ea3
2023-07-21 10:51:35 +00:00
stephan 3068d6622e Internal cleanups in the async part of the JS library bootstrap phase.
FossilOrigin-Name: b6d57ab63793241a500ea527c5b3216c54b3ff1972d3adbbf42a9a53bfec0aa1
2023-07-21 09:10:42 +00:00
stephan 35325a37ce Minor internal cleanups in JS code.
FossilOrigin-Name: 21a2ca9fc46bf746874579897872e2a45cb07f278abb670dd22b122f7d6a9a6c
2023-07-20 23:25:32 +00:00
dan 3a51f8c307 Fix some divide-by-zero errors that could occur when handling corrupt tombstone hash records.
FossilOrigin-Name: 7567ca0676f0d45026f5cd4f3fbcd09119c2eaab8ec1711499609c16c452b5e4
2023-07-20 20:29:56 +00:00
dan d6f5aa824e Avoid an infinite loop that could be entered when dealing with corrupt fts5 tombstone hash pages.
FossilOrigin-Name: 69ce2ce035279f2a00c2238187cf4d2a9092c3410f5900e4613fe4e46311169e
2023-07-20 20:09:26 +00:00
drh 51d7869096 Experimental framework upon which to build a better JSON parse structure that
supports cached of modified JSON.  All of these changes are tentative and
subject to change or removal.  Incremental check-in.

FossilOrigin-Name: e384163a4763c10b5838cbf28f9c4d8ea799bbbed6e890e8aa91b2bd1458646d
2023-07-20 17:45:09 +00:00
dan 0b3791b537 Add tests and fixes for the new code on this branch.
FossilOrigin-Name: 5aac50e92e956b15367c75c20c17bc1c75e84e2752bfffe4ad0a266cb9bd3b8a
2023-07-20 16:07:04 +00:00
stephan 0b518b8d65 Filter the OPFS VFSes out of the sqlite3-node.mjs build. Add another level of subdirectory to the sahpool to later enable transparent support of client-provided files under the VFS's root dir. Rework the awkward sahpool-via-oo1 mapping.
FossilOrigin-Name: 080a4d0aba30d8f3802b49be4a113205f069b3bdea8cebf525d654055642ff62
2023-07-20 09:06:42 +00:00
drh f9bfc32b73 Performance enhancements to the JSON parser and code generator.
FossilOrigin-Name: 0a745897d66e9f36ea6e787fad54f0c413d52c226dc96fad36ccd33a0aa1930e
2023-07-19 23:02:45 +00:00
drh 93b3c1f398 Revise the new JSON parser performance test to make the test run 25 times
longer, and thus provide a more repeatable number.

FossilOrigin-Name: bee9e403ae47103938aabb9a7a7e120dfa741b464875965e58788a51fa56a8fe
2023-07-19 20:23:34 +00:00
drh 5e4d77fa88 Describe JSON parser performance testing procedure in test/json/README.md
FossilOrigin-Name: c47056e158073ee746c233dd2729cf224ea9e38821b15aa71ebf97607e602281
2023-07-19 19:23:30 +00:00
dan d1fbaa071b Fix various issues with code added to this branch.
FossilOrigin-Name: 8d09011fa2c6ae9cc88e1766f9aad4578efbf9e0e311b8c6efdffe7a3f88f923
2023-07-19 18:47:02 +00:00
stephan 55f318e53f More internal refactoring and docs for opfs-sahpool.
FossilOrigin-Name: 64ccf6177a019eab46fb3345ad1e8ba80eaf2c9da55767031f9f04ccd16afb4d
2023-07-19 17:47:02 +00:00
stephan 96cb7007a9 Update the development-over-ssh docs for the wasm build.
FossilOrigin-Name: 500109bd0a4c134b91c37f397ff1ee828e09c17f7ecd153f975ede748caee7bb
2023-07-19 17:46:28 +00:00
drh 50834d8b74 Further improvement to JSON parser performance.
FossilOrigin-Name: 144c8ccf6e5bb2527dd98742f0d67e0a16c627e7c67f754ce8ed4c4fb5b8d8b6
2023-07-19 17:24:36 +00:00
drh 8376ae7295 Further performance enhancements to JSON parsing and rendering. Total
performance gain for large JSONs so far on this branch is about 11%.

FossilOrigin-Name: adb4d6b007cbe9d7c9670f5fc196443ebe0f3a89df1f3290ba6247fcf83fe5bd
2023-07-19 15:06:29 +00:00
drh 7a816e771a Performance optimization for parsing large JSONs that contain lots of text.
FossilOrigin-Name: c9fbe0185cd5d64950724b00cd0bfb3a7939a985040465a0f35f445acb6e94a6
2023-07-19 13:50:31 +00:00
stephan eafb4418a0 Major restructuring of opfs-sahpool to improve maintainability and hopefully make it easier to reuse these pieces in the upcoming JSPI-based VFS experiment.
FossilOrigin-Name: 534481cd0c2e6f62dd0a82f25d4b78fdcc671eb70d6966693c98212a6420891c
2023-07-19 13:31:29 +00:00
stephan cd302a5e32 Appearance and legibility improvements for tester1.js.
FossilOrigin-Name: 4fd3ed1b6405f690825c9993e56b199d4a7fb497cf9131815c803150b8b96528
2023-07-19 11:33:52 +00:00
drh 47079b19df Merge the latest trunk enhancements into the sqlite3_stmt_explain branch.
FossilOrigin-Name: 06eae046c1a1155f22590b88942db72f01482f2363da9753f8adfdb322679055
2023-07-19 10:17:42 +00:00
drh 37fd50df1f Two minor improvements to sum(), one of which was inspired by
[forum:/forumpost/af5be98dbc|forum post af5be98dbc].

FossilOrigin-Name: a0d3e7571aded8d1e03908059d2d5aa5d62ec49bff099cb38f6f35df5e4b18b5
2023-07-19 09:52:10 +00:00
stephan 780bc4c557 Internal refactoring in opfs-sahpool. Move OPFS tests to the end of tester1.c-cpp.js.
FossilOrigin-Name: 6bd5a7413dd830ca41b587a2826fb599a2196fb0186646a2333500f950b3cf4d
2023-07-19 08:18:25 +00:00
drh 9246c85b61 Do not use the [/info/609fbb94b8f01d67|viewscan optimization] on a query
that has only a single loop, as the cost adjustments can cause problems
for outer queries.  Proposed fix for the performance regression reported by
[forum:/forumpost/64d36440e473516c|forum post 64d36440e473516c].

FossilOrigin-Name: 76152ad2ffe56034f2fd93d9a1ce9358e1677a7e9cd3dcd9f3a34a5c956a463e
2023-07-18 21:06:19 +00:00
drh 6a9be2cbc7 Improved ".wheretrace" for the VIEWSCAN optimization.
FossilOrigin-Name: 27057ea76b5f72c73fb6f16094736685019643c665b49fd0bb8d60a812ce2338
2023-07-18 20:41:09 +00:00
dan d05bf0fe61 Fix various problems with fts5 contentless_delete=1 tables.
FossilOrigin-Name: 0d005112b8aca9e9eca9d86d5fed9168f6a0218fd290b5489b9e7b05714610f4
2023-07-18 19:52:32 +00:00
stephan aed5abcc0f More opfs-sahpool tests.
FossilOrigin-Name: 60713fa9c4627ef17e0b8778eee37913d2b930c5a06414721a00af30e1395090
2023-07-18 19:47:19 +00:00
stephan 0649a1a05d Fix a shadowed var in opfs-sahpool and add more tests.
FossilOrigin-Name: 9c8a73aff0f291e0c18072372e0d8961d3a05910489598d0d342227d99871954
2023-07-18 19:27:11 +00:00
stephan b0dd9d427f Further refactoring of opfs-sahpool and start integrating it into tester1.c-pp.js.
FossilOrigin-Name: 91c789234963b660ae900f0738906b28a477993709e286d8125b2f4d6101601c
2023-07-18 18:52:41 +00:00
drh 3f47e7551c Merge the FTS5 memory leak fix from trunk.
FossilOrigin-Name: fb65cb73d7ea22a8b20dccfa3abdaaa809eee4fcee6fe4846bd2e598ceb49aa4
2023-07-18 17:43:47 +00:00
drh d1db37a2f3 Enhance the sqlite3_stmt_explain() interface so that avoids unnecessary
reprepare operations.

FossilOrigin-Name: 050f773addd605f6690348c98e9000a9a3663bbc26288a71973fd7b40468e8ab
2023-07-18 17:29:05 +00:00
dan a5b1f34125 Fix memory leak in fts5 introduced by [def41225].
FossilOrigin-Name: 4dcad2db743fdb9ef72871ca5a4d1384f76cb697161b0f5110e2670a83a18e8a
2023-07-18 16:29:34 +00:00
stephan d69e6e6efb Major restructuring of the opfs-sahpool bits to better support multiple instances per app (each sandboxed from each other). Eliminate the pesky promise resolution race condition when two such instances are loaded in parallel.
FossilOrigin-Name: 95e5fa498f71708caeb3394636c4853530a8b2d54406e503f32750732d6815d5
2023-07-18 16:24:51 +00:00
dan cb1d6b997e Merge trunk changes with this branch.
FossilOrigin-Name: 5f66eb4e2603278dcc9dbfe4bf506cba1aa03180cfb492a0dfc3a8be32cc994b
2023-07-18 13:55:55 +00:00
dan 263a8e8c0a Fix a buffer overrun in fts5 that could occur in 'secure-delete' mode when dealing with corrupt database records.
FossilOrigin-Name: def41225b2f10b0294ab506f28bc87873688d4030f7056839298ff18e058ece5
2023-07-18 13:55:19 +00:00
stephan dec4cea24c Refactor opfs-sahpool to support multiple instances, each with a separate VFS name and directory.
FossilOrigin-Name: d036eaf6ac60c576428db40f015733c5d5425f7d613194fd8d9d4d98659077c4
2023-07-18 12:09:16 +00:00
dan 2cccced140 Fix cases where a row is inserted into a contentless_delete=1 fts5 table and then deleted within the same transaction.
FossilOrigin-Name: d928856a226fb7f001e55ff7e8eb58a656b982f1efa811de46c382b8b7cd778c
2023-07-17 18:40:39 +00:00
dan 55e0fd4a9d Do not allow the 'delete' command to be used on contentless_delete=1 fts5 tables.
FossilOrigin-Name: cc694b83408ccb5d42204cb624145c76e95329cbe1d1fe8815c70a7a00af231a
2023-07-17 17:59:58 +00:00
dan 24433bea19 Merge trunk changes into this branch.
FossilOrigin-Name: c4fb2f2ea0afe638fd7cffd89fbdb0a91589577c6f8299c7bbc17ac121be518b
2023-07-17 11:47:42 +00:00
stephan 99d4a2db46 Another -DSQLITE_TEMP_STORE=3 to 2 change for the wasm build.
FossilOrigin-Name: 5ad8f9987c11b5db67edd69a3471ff388140d6268d5d3f5a05dec6bb6d92ac05
2023-07-17 07:43:04 +00:00
stephan ed640943c4 sahpool vfs: rename defaultCapacity to initialCapacity, per feedback.
FossilOrigin-Name: 2c9c95d087c7db2a975ffae15af5ade621411c493145b889f0c9157a55c4952a
2023-07-17 07:14:12 +00:00
stephan 2bf2ff605f Change wasm build's -DSQLITE_TEMP_STORE=3 to -DSQLITE_TEMP_STORE=2, primarily so that clients can optionally shift temp files from memory to OPFS storage.
FossilOrigin-Name: 764430e804332b8f7b2f63a3c0c8acf8f6cc92534858d89de2d310938c1c0d27
2023-07-17 07:13:44 +00:00
stephan 039081123c Remove an extraneous JS script import.
FossilOrigin-Name: 5a205b25912f3feea594a2161a4b8f3955bd29163b39cb0e621b3abdb81fc24d
2023-07-16 21:07:01 +00:00
stephan 8d67d501e3 Add sqlite3-vfs-opfs-sahpool.js to ext/wasm/api/README.md.
FossilOrigin-Name: 58d05201042804c682a417495f13ab47a535e353da6cc307f9d726a289e71ca2
2023-07-16 20:42:19 +00:00
stephan 1ff2903785 Minor doc tweaks.
FossilOrigin-Name: 0cb0d7a9e73e48406f146ebebf211311fd0a0b14b285f0af9e2b645c3d2f172a
2023-07-16 20:20:03 +00:00
stephan d703d25a35 Correct a missing assignment. Minor doc additions.
FossilOrigin-Name: 1c4957d0ef23ff14d4f7bfb33a809dd92712ee9faf77b6052f823eb55de15cf6
2023-07-16 19:29:50 +00:00
stephan a34f27daed Doc tweaks only, no code changes.
FossilOrigin-Name: 044c28dffd45f7c4484686995edd4a1b92151450743968e7d0f662b5c850aa6b
2023-07-16 19:20:45 +00:00
stephan 24873723f4 Add a way to remove the opfs-sahpool's persistent state from OPFS or unlink() an individual file. Doc cleanups.
FossilOrigin-Name: 80982daac3c098033dbc249bb7a17ef84ae218d2d789f8644e7f4af18b553d24
2023-07-16 17:51:43 +00:00
stephan da6a42a921 Move SAH pool configuration options from the library-level config to a config passed to the VFS install routine. Extend and document the PoolUtil object.
FossilOrigin-Name: d2ed99556fa1f40994c1c6bd90d1d5733bebc824b1ebfabe978fae9e18948437
2023-07-16 16:52:09 +00:00
stephan d62c464541 speedtest1.js: only install opfs-sahpool if it's provided via --vfs flag, to avoid locking errors in concurrent speedtest1 tabs with other VFSes. Add opfs-sahpool reserveMinimumCapacity().
FossilOrigin-Name: aa94c8abfbdfc4c7b36554c4b3ea90a5065e7e3f4294c64c8cbf688b4688300d
2023-07-16 14:07:59 +00:00
stephan aa1b2a80ee During "opfs" VFS init, check for URL param opfs-disable and, if set, do not install the VFS. Added per forum suggestion to provide a way to help rule out misinteraction between the "opfs" and "opfs-sahpool" VFSes.
FossilOrigin-Name: 29905b7a75b73e32125bf9116033cae7235a135b668a3b783a3d8dcb0bc80374
2023-07-16 12:02:37 +00:00
stephan 8449dacbd4 Micro-optimizations in opfs-sahpool.
FossilOrigin-Name: 52f23db948ae9694df69c00177b85cb569e9b211350a4a2dbf249e7cd8de700c
2023-07-16 11:49:18 +00:00
stephan d0ae50411f Redefine what the opfs-sahpool installation promise resolves to. Fix addCapacity(). Add utility methods to import/export files.
FossilOrigin-Name: 809c6f4de3653ad7a7751af45a7a0d6cb20c3ee3be80c69833c729242227d970
2023-07-16 10:02:41 +00:00
stephan 28d46cce0b Minor cleanups in the opfs-sahpool VFS.
FossilOrigin-Name: 279e09070918dab7b60c39179ebb7eb931ca6bd4e589b414f436740499a2f910
2023-07-15 21:08:48 +00:00
stephan 38d1db9b79 Correct opfs-sahpool VFS after the pebkac involving the previous speedtest1 runs. Make that VFS explicitly opt-in to avoid certain unfortunate locking situations.
FossilOrigin-Name: 41bf1fe31f2f3d0daa2bac25dc57262a4b90f22fed6fa97e4e92467c32ae02dc
2023-07-15 19:08:58 +00:00
dan 0dac350f3c Store large tombstone hash tables in multiple database records. Ensure the same hash tables handle rowid 0.
FossilOrigin-Name: 4410e60d0c76e057ee962124f9239c6e17fd5ccafdbb4d9b703448eabd7781e3
2023-07-15 18:57:57 +00:00
drh e393f6eff0 Add the experimental sqlite3_stmt_explain(S,E) interface.
FossilOrigin-Name: 5683743ddf0bb051f2fe5d137cc18407e000e77e9faa9010a22e3134b575638b
2023-07-15 16:48:14 +00:00
stephan eadabc6513 speedtest1 JS: only add --memdb flag by default if no --vfs is provided.
FossilOrigin-Name: 676ffe6280c1ce787b04d0cdb4a0664229c6125c601af4b18d1bfa125aac3675
2023-07-15 16:30:46 +00:00
stephan bb65feb869 Rename opfs-sahpool removeCapacity() to reduceCapacity().
FossilOrigin-Name: fff68e9f25a57045e9d636b02ffa073cf1b984b2587d4fce10f6e35c9988469c
2023-07-15 11:37:34 +00:00
stephan 88af76f62d opfs-sahpool VFS now seems to work, in that it runs fine (and blazingly fast) in speedtest1. Add sqlite3.config options for the high-level configurable parts of opfs-sahpool.
FossilOrigin-Name: 5d92d5f4d8ac4cfa37ba473e5cc861628b783bbf1ae4d138bcae8b9d6cc6e798
2023-07-15 11:23:57 +00:00
stephan 100bc4429a More work on porting the sahpool bits. Revert to random backing-store names because it works better with the capacity-reduction algorithm.
FossilOrigin-Name: b4e005fd38b06b8d2f2317b955b93807e80a6a18db5f06d7747978d3bfa11411
2023-07-15 01:02:38 +00:00
stephan bee3213145 Switch opfs-sahpool to use deterministic backing-store file names. Delay VFS registration until after the pool's files are all opened. Fail vfs init if the client's OPFS API has an async FileSystemSyncAccessHandle.close() method (indicating that it's outdated).
FossilOrigin-Name: 7c6697ededee9a64962ac6fd78934c6d6e39258b9558a03c1a6c02bf3be1759e
2023-07-14 23:02:58 +00:00
stephan 92bf6fd4f6 Integrate the opfs-sahpool VFS into the JS build and get it loading (but it's still far from functional).
FossilOrigin-Name: ef96e6b586825a2b3ed011174309cba8ce1031876c86dc59ed87ab9bbc64f57f
2023-07-14 21:48:45 +00:00
stephan 50c61825fc Initial sketches for an alternate OPFS VFS which uses a pool of pre-opened SyncAccessHandles to bypass the need for a dedicated I/O worker and the COOP/COEP HTTP response headers. Currently completely non-functional.
FossilOrigin-Name: a93de9f2a553a3a4edd1b361dd6f465a1b0b5b51f7bb8ede432067aedcfefda4
2023-07-14 21:17:29 +00:00
stephan a02d6d8237 Minor internal cleanups in the OPFS VFS.
FossilOrigin-Name: 984d491eb3fe06f714bf07d6873321f3992a072812b46508e599bfefd39dff3e
2023-07-14 21:06:00 +00:00
stephan 8ba619b5f5 Minor wasm build cleanups.
FossilOrigin-Name: 816b503f093c4e6d92d0eb2f9fbd841acd01cc9bc89ee58d961b56c64f71406a
2023-07-14 12:21:05 +00:00
stephan f4449dc0ba Reintroduce WASMFS+OPFS support, albeit as a separate (less portable) WASM build.
FossilOrigin-Name: 7e4c594e9591101f6b727212c99f561498444e86d3d059e5864a717c614f903a
2023-07-13 18:35:48 +00:00
stephan d655290c99 Minor cleanups related to reintroducing wasmfs support.
FossilOrigin-Name: 6918a927b5ecd1df5916b70d52d8d5ce289869c3f323e577b7dfc7c939084400
2023-07-13 18:13:52 +00:00
stephan bb9549e0be More work on the wasmfs build and its test apps.
FossilOrigin-Name: 953b8557194e9451dcf9f3bb433eafb67961487325e6519e675e73fecf850bfb
2023-07-13 17:27:05 +00:00
stephan d51cefd1bb Get speedtest1 wasmfs variant running again.
FossilOrigin-Name: 0f77e031daf0121f7603409cedf57f138edbb43083cbbcf00d2a423e16de25a7
2023-07-13 16:33:45 +00:00
drh bc6d949aa3 Fix an off-by-one error in tests for cells overflowing the end pages. The
error is completely harmless for the default page cache, but might cause
problems for application-defined page caches that pack pages tightly
together.

FossilOrigin-Name: ce6793e954f291b6f5c29175baf730ce217328de1f0601b8935daac62af5f448
2023-07-13 14:49:39 +00:00
stephan cc336321f8 Significant surgery on the wasm build on the way to incorporating wasmfs as a first-class build option.
FossilOrigin-Name: 215c37fce38cf647e073480689b79d952af7eb8165ae08c7f5caed27003fecfc
2023-07-13 14:08:30 +00:00
stephan d41d454fc1 More work on the wasmfs build. Resolve the inconsistent argument type passed to sqlite3InitModule() in such builds.
FossilOrigin-Name: 4850a9e2d2b41b91e91b9ef99a6d26ddf11a161b4c970fc00d2d095606729a04
2023-07-13 10:41:41 +00:00
stephan 0f0e5700ba Minor cleanups in the wasmfs build.
FossilOrigin-Name: f64477f21040c265061ad9b7f601e74159fd6141e7e4b122c80bf244ab6ddb76
2023-07-13 04:54:18 +00:00
stephan 01bdff7a1c Get wasmfs build and its bare-bones test app working again, albeit currently in ES6 mode only.
FossilOrigin-Name: 647761ed422f196f94facc88bbddd7219a2c1a6301a5f847b0a32d3e405233a7
2023-07-13 04:26:13 +00:00
stephan 81602595a0 Replace 'self' with 'globalThis' in some JS test code.
FossilOrigin-Name: 53eeffc9b1ea8c3ee7d7cdcc220997ed893403e45b35d8ea10805ea3b0435587
2023-07-13 03:46:43 +00:00
stephan 5da49ff562 JS: change the default OPFS VFS journal mode from PERSIST to DELETE, as DELETE is faster in current browsers.
FossilOrigin-Name: a061a43e36b8f3cb7c465fd2e54cfcd9dceb07e232ee78602bbdd1818acd05ea
2023-07-13 01:51:16 +00:00
stephan a5512fb179 Remove obsolete ext/wasm/api/sqlite3-wasi.h.
FossilOrigin-Name: f661a6e0c1ac081ee402e64590eabd427440db1b717be9166bdfc025d0b0d899
2023-07-13 01:48:53 +00:00
dan fc158d3870 Avoid a case of an infinite loop in fts5 when dealing with corrupt records.
FossilOrigin-Name: 0e801f11cd2d50fc710a80c2b3b805c7801e660cff1bcc87be89133d43796524
2023-07-12 20:24:23 +00:00
dan 9f6c0d0027 Fix a crash that may occur when handling corrupt records in contentless_delete=1 mode.
FossilOrigin-Name: 74d7610a8e32ac62a608141ff16bfe517c8d2cdfd43f81ef3a5df2b2003e541a
2023-07-12 19:34:32 +00:00
drh 9faa3bbdbb Get access to SQL functions and virtual tables used for debugging and analysis
of FTS5 using SQLITE_FTS5_DEBUG and without the need for SQLITE_TEST.

FossilOrigin-Name: 383de8e2259adb3f8881a618b8b5e06c1f3b2bd7ac274bd047410ad7d2a18f0f
2023-07-12 18:38:47 +00:00
stephan b3b64179df Internal JS doc fix.
FossilOrigin-Name: 7c5ea02205a360a11fa9043be12c678eb7fe6f6a0b63b09a3dc976beae881e3f
2023-07-12 09:30:13 +00:00
stephan a17b04f840 Correct handling of the speedtest1 --without-rowid flag in the HTML front-end to speedtest1.wasm. Reduce default speedtest1-with-OPFS workload size in ext/wasm/index.html to speed up manual testing.
FossilOrigin-Name: 2daba2cdd3890c87899ff3eb0c1706c814f0ae62e72822eef3b0d4a885679a6b
2023-07-12 09:16:18 +00:00
dan b184c91076 Have contentless_delete=1 tables support REPLACE statements.
FossilOrigin-Name: 2f553a660e00564e51bc4209c92bd3628fb1266f4a52832792fbf91e4234a0ba
2023-07-11 20:57:19 +00:00
dan b92669db73 Have contentless_delete=1 tables support regular DELETE statements, instead of just the special INSERT syntax.
FossilOrigin-Name: fffb8616905501669a94231d5d9f53446bf09553353f2cdab7c43ca54bbb7fa6
2023-07-11 20:19:46 +00:00
drh 8a1104940d Reduce divergence between branches: trunk and wal-shm-exceptions.
FossilOrigin-Name: d38ed6b1fc6b572997d520c36f256c11404fb1bd5f28acab07466bfc96554a8f
2023-07-11 19:54:17 +00:00
dan d23f210377 Use a hash-table instead of a flat list to store tombstone rowids.
FossilOrigin-Name: 948267b066d0dbe667881b3d26a007fa24576da6e57c112676fadeb846c13f0b
2023-07-11 18:55:19 +00:00
drh 66422b57f9 Get fault injection for testing working correctly. Other code clean-up.
FossilOrigin-Name: ff492277ed00c1f637a5b4ccd6d8193ea22f6781f90073861588a2b7d5c045b7
2023-07-11 15:52:52 +00:00
dan 6788c7b7c0 Begin adding support for deleting rows from contentless fts5 tables.
FossilOrigin-Name: e513bea84dfaf2280f7429c9a528b3a1354a46c36e58ab178ca45478975634e0
2023-07-10 20:44:09 +00:00
mistachkin c0b6112d04 Minor style changes to the previous MSVC makefile changes on this branch.
FossilOrigin-Name: 4c950872c870a5968fa4cb8840cf60569a66c0e508811ee79992825ec9c02da3
2023-07-10 19:21:09 +00:00
drh 7994d9dab7 Compile for SEH if the USE_SEH=1 option is specified on the nmake command line.
FossilOrigin-Name: 0e67a82f6cfe681d4676ac5c92d92e95b9dcc3198df21516bccdde002f5e52ae
2023-07-10 19:13:40 +00:00
drh 5d3cc0c09b Fix a warning. Unclear yet if the code is correct this way or not.
FossilOrigin-Name: 2df4fd76ef48bb63389df8d783059af6b8f2cbb2c11d07953b408e71f50b625b
2023-07-10 18:16:49 +00:00
drh cc9380f68d Merge the latest trunk enhancements into the wal-shm-exceptions branch.
FossilOrigin-Name: f655d08d019bab0c578886e91ca24fd956d1b3dea9b9c9104812b3cf62e5e556
2023-07-10 18:05:23 +00:00
drh 4c40b7b819 Always use the "LL" suffix on 64-bit integer literals.
FossilOrigin-Name: 07d95ed60f0a17ea13b4bc19c2ab2ec9052fedd27c9e1e57a1ec6e3a6470e5b7
2023-07-08 17:42:24 +00:00
drh bc532ae15f Fix harmless scan-build warnings.
FossilOrigin-Name: beab3c98639be531744e60440223bb9ee76bc15234aff05e5efb273c8241dfd8
2023-07-08 14:27:55 +00:00
drh 50ba4e3efd Omit the use of memmove() in sqlite3FpDecode() in an attempt to avoid
spurious warnings from valgrind.  Also makes the code slightly smaller and
faster.

FossilOrigin-Name: b409943af00e35ad05906d0a80ffa1225a66f3d807c131e00cfbf6671a03a981
2023-07-07 18:49:08 +00:00
drh ff96718b52 Merge all recent trunk enhancements into the wal-shm-exceptions branch.
FossilOrigin-Name: e59cea2d851be68559e7d01a01cb05684c44d151d022ea9790ce99671016ab00
2023-07-06 17:55:33 +00:00
dan f8be243fef Add experimental ".scanstats vm" command to the shell tool.
FossilOrigin-Name: e727640fb5c17d23b8e27972065b4acbf169c9240298d3ff7aed092b727d052d
2023-06-30 19:14:35 +00:00
dan e6d7ae24e3 Fix an error in the previous commit on this branch.
FossilOrigin-Name: 47c11ca90f98ffc4d91f07e2ab35826a604a2c903008eeddf8b814bb984971f2
2023-06-30 19:13:27 +00:00
dan aec0aa83a9 Add the "nexec" and "ncycle" columns to the bytecode virtual table. For accessing counters collected when SQLITE_DBCONFIG_STMT_SCANSTATUS is enabled.
FossilOrigin-Name: f7b163a319bee9e935a4abf0bd87e16c9974cba5ae75b0cbba63c9da168f2006
2023-06-30 18:59:10 +00:00
dan 51f0c762f0 Fix a case where the system error was not being set correctly.
FossilOrigin-Name: 13a524b351a0812214b223aa4e0a9d778ae7a75cf9875da1147a09c5cb76b9fe
2021-09-11 14:33:05 +00:00
dan 9023444fea If an EXCEPTION_IN_PAGE_ERROR exception is caught, make the underlying OS error code available via sqlite3_system_errno().
FossilOrigin-Name: fdb20e9ee48465b94aa6ac3c5e263ecaa7c3b10f4a193e79f965b7c35944b08b
2021-09-10 21:28:56 +00:00
dan ee7c667a36 Change the structuree exception handler in wal.c to catch EXCEPTION_IN_PAGE_ERROR instead of EXCEPTION_ACCESS_VIOLATION.
FossilOrigin-Name: 0c6ab539cfbc908550fa993a587e85d644b6335853ff9781caae860c461c045d
2021-08-26 21:12:07 +00:00
dan 101a6b87c7 Add test cases to ensure that there is at least one test that causes an exception to be thrown from each SEH_INJECT_FAULT macro.
FossilOrigin-Name: e480146f49004c818ad93464289761d8a7246b502bd079f8140d0a6f684ebe4e
2021-08-21 15:49:15 +00:00
drh 67ae593ffe Use VVA_ONLY() instead of TESTONLY() in SEH macros so that the build
works for coverage testing.

FossilOrigin-Name: 9e9d5b566da72adf780685ea27da7d76a7274e904a533438255af92875c94503
2021-08-20 18:18:12 +00:00
dan 5b6c3c8f7d Add missing SEH_INJECT_FAULT macros.
FossilOrigin-Name: 27e9bdb3a90a5826f45d32e0378e9afa90a16ac762a75236f38218f41bc45ded
2021-08-20 17:23:19 +00:00
dan c6effdbfba Add some assert() statements and fix small issues with code on this branch.
FossilOrigin-Name: bd9b373b177a71f5eaeb7ce8dd1dc755b34951ee1376c24652418bb329bfcb45
2021-08-20 16:19:19 +00:00
dan b15c63cbb9 Fix a problem on this branch causing checkpoints that do not proceed because they can't get the CHECKPOINT lock to return incorrect output values.
FossilOrigin-Name: 5a98820c3d20766813bb4b0da23247ac90f380c07827ff13b61d5b606d8311aa
2021-08-20 11:50:12 +00:00
drh aecd7f9d75 Enable this branch to run both with and without the -DSQLITE_USE_SEH option.
FossilOrigin-Name: 840e0a14ebc4a85d4ede0223287d4c1e4465e9f20c3c43bfc74775dddec1fa8f
2021-08-19 23:10:57 +00:00
dan 8aa22ffe98 If SQLITE_USE_SEH is defined, handle structured-exceptions thrown by MSVC builds if the *-shm file mapping is accessed after it becomes invalid for some reason.
FossilOrigin-Name: 5c5fa47076c5da25ca92248df3a4c11b90a0225a07f05d7220fa95d9bbb36f43
2021-08-19 21:01:15 +00:00
dan 4ffaa7c5de Passive checkpoints do not use the busy-handler. So, in order to minimize visible changes for legacy applications, do not enable blocking locks for passive checkpoints.
FossilOrigin-Name: 9c2b4bdd03716bf492ba85198717f3084ebf187bdb068893bd1ff8662362df89
2020-06-29 19:58:26 +00:00
dan 5289c51050 Change things so that if SQLITE_ENABLE_SETLK_TIMEOUT is defined as 2 instead of 1, all blocking locks are taken for a single millisecond and the default busy-handler invoked as normal.
FossilOrigin-Name: ac381e6eb3c9284e65f7aad66d21bee1bca9ef4123684ccaf069b8a6d157a56d
2020-06-29 18:22:21 +00:00
388 changed files with 47787 additions and 8730 deletions
+93 -31
View File
@@ -57,6 +57,7 @@ LIBTCL = @TCL_LIB_SPEC@
#
READLINE_FLAGS = -DHAVE_READLINE=@TARGET_HAVE_READLINE@ @TARGET_READLINE_INC@
READLINE_FLAGS += -DHAVE_EDITLINE=@TARGET_HAVE_EDITLINE@
READLINE_FLAGS += -DHAVE_LINENOISE=@TARGET_HAVE_LINENOISE@
# The library that programs using readline() must link against.
#
@@ -706,6 +707,24 @@ fuzzcheck$(TEXE): $(FUZZCHECK_SRC) sqlite3.c sqlite3.h $(FUZZCHECK_DEP)
fuzzcheck-asan$(TEXE): $(FUZZCHECK_SRC) sqlite3.c sqlite3.h $(FUZZCHECK_DEP)
$(LTLINK) -o $@ -fsanitize=address $(FUZZCHECK_OPT) $(FUZZCHECK_SRC) sqlite3.c $(TLIBS)
fuzzcheck-ubsan$(TEXE): $(FUZZCHECK_SRC) sqlite3.c sqlite3.h $(FUZZCHECK_DEP)
$(LTLINK) -o $@ -fsanitize=undefined $(FUZZCHECK_OPT) $(FUZZCHECK_SRC) sqlite3.c $(TLIBS)
# Usage: FUZZDB=filename make run-fuzzcheck
#
# Where filename is a fuzzcheck database, this target builds and runs
# fuzzcheck, fuzzcheck-asan, and fuzzcheck-ubsan on that database.
#
# FUZZDB can be a glob pattern of two or more databases. Example:
#
# FUZZDB=test/fuzzdata*.db make run-fuzzcheck
#
run-fuzzcheck: fuzzcheck$(TEXE) fuzzcheck-asan$(TEXE) fuzzcheck-ubsan$(TEXE)
@if test "$(FUZZDB)" = ""; then echo 'ERROR: No FUZZDB specified. Rerun with FUZZDB=filename'; exit 1; fi
./fuzzcheck$(TEXE) --spinner $(FUZZDB)
./fuzzcheck-asan$(TEXE) --spinner $(FUZZDB)
./fuzzcheck-ubsan$(TEXE) --spinner $(FUZZDB)
ossshell$(TEXE): $(TOP)/test/ossfuzz.c $(TOP)/test/ossshell.c sqlite3.c sqlite3.h
$(LTLINK) -o $@ $(FUZZCHECK_OPT) $(TOP)/test/ossshell.c \
$(TOP)/test/ossfuzz.c sqlite3.c $(TLIBS)
@@ -765,13 +784,22 @@ mptest: mptester$(TEXE)
$(MPTEST2) --journalmode DELETE
has_tclsh84:
sh $(TOP)/tool/cktclsh.sh 8.4 $(TCLSH_CMD)
touch has_tclsh84
has_tclsh85:
sh $(TOP)/tool/cktclsh.sh 8.5 $(TCLSH_CMD)
touch has_tclsh85
# This target creates a directory named "tsrc" and fills it with
# copies of all of the C source code and header files needed to
# build on the target system. Some of the C source code and header
# files are automatically generated. This target takes care of
# all that automatic generation.
#
.target_source: $(SRC) $(TOP)/tool/vdbe-compress.tcl fts5.c
.target_source: $(SRC) $(TOP)/tool/vdbe-compress.tcl has_tclsh84 fts5.c
rm -rf tsrc
mkdir tsrc
cp -f $(SRC) tsrc
@@ -781,15 +809,15 @@ mptest: mptester$(TEXE)
cp fts5.c fts5.h tsrc
touch .target_source
sqlite3.c: .target_source $(TOP)/tool/mksqlite3c.tcl src-verify
sqlite3.c: .target_source $(TOP)/tool/mksqlite3c.tcl src-verify has_tclsh84
$(TCLSH_CMD) $(TOP)/tool/mksqlite3c.tcl $(AMALGAMATION_LINE_MACROS)
cp tsrc/sqlite3ext.h .
cp $(TOP)/ext/session/sqlite3session.h .
sqlite3r.h: sqlite3.h
sqlite3r.h: sqlite3.h has_tclsh84
$(TCLSH_CMD) $(TOP)/tool/mksqlite3h.tcl $(TOP) --enable-recover >sqlite3r.h
sqlite3r.c: sqlite3.c sqlite3r.h
sqlite3r.c: sqlite3.c sqlite3r.h has_tclsh84
cp $(TOP)/ext/recover/sqlite3recover.c tsrc/
cp $(TOP)/ext/recover/sqlite3recover.h tsrc/
cp $(TOP)/ext/recover/dbdata.c tsrc/
@@ -804,7 +832,7 @@ tclsqlite3.c: sqlite3.c
echo '#endif /* USE_SYSTEM_SQLITE */' >>tclsqlite3.c
cat $(TOP)/src/tclsqlite.c >>tclsqlite3.c
sqlite3-all.c: sqlite3.c $(TOP)/tool/split-sqlite3c.tcl
sqlite3-all.c: sqlite3.c $(TOP)/tool/split-sqlite3c.tcl has_tclsh84
$(TCLSH_CMD) $(TOP)/tool/split-sqlite3c.tcl
# Rule to build the amalgamation
@@ -1092,10 +1120,10 @@ tclsqlite3$(TEXE): tclsqlite-shell.lo libsqlite3.la
# Rules to build opcodes.c and opcodes.h
#
opcodes.c: opcodes.h $(TOP)/tool/mkopcodec.tcl
opcodes.c: opcodes.h $(TOP)/tool/mkopcodec.tcl has_tclsh84
$(TCLSH_CMD) $(TOP)/tool/mkopcodec.tcl opcodes.h >opcodes.c
opcodes.h: parse.h $(TOP)/src/vdbe.c $(TOP)/tool/mkopcodeh.tcl
opcodes.h: parse.h $(TOP)/src/vdbe.c $(TOP)/tool/mkopcodeh.tcl has_tclsh84
cat parse.h $(TOP)/src/vdbe.c | $(TCLSH_CMD) $(TOP)/tool/mkopcodeh.tcl >opcodes.h
# Rules to build parse.c and parse.h - the outputs of lemon.
@@ -1106,10 +1134,10 @@ parse.c: $(TOP)/src/parse.y lemon$(BEXE)
cp $(TOP)/src/parse.y .
./lemon$(BEXE) $(OPT_FEATURE_FLAGS) $(OPTS) -S parse.y
sqlite3.h: $(TOP)/src/sqlite.h.in $(TOP)/manifest mksourceid$(BEXE) $(TOP)/VERSION
sqlite3.h: $(TOP)/src/sqlite.h.in $(TOP)/manifest mksourceid$(BEXE) $(TOP)/VERSION has_tclsh84
$(TCLSH_CMD) $(TOP)/tool/mksqlite3h.tcl $(TOP) >sqlite3.h
sqlite3rc.h: $(TOP)/src/sqlite3.rc $(TOP)/VERSION
sqlite3rc.h: $(TOP)/src/sqlite3.rc $(TOP)/VERSION has_tclsh84
echo '#ifndef SQLITE_RESOURCE_VERSION' >$@
echo -n '#define SQLITE_RESOURCE_VERSION ' >>$@
cat $(TOP)/VERSION | $(TCLSH_CMD) $(TOP)/tool/replace.tcl exact . , >>$@
@@ -1122,19 +1150,21 @@ keywordhash.h: $(TOP)/tool/mkkeywordhash.c
# Source files that go into making shell.c
SHELL_SRC = \
$(TOP)/src/shell.c.in \
$(TOP)/ext/misc/appendvfs.c \
$(TOP)/ext/consio/console_io.c \
$(TOP)/ext/consio/console_io.h \
$(TOP)/ext/misc/appendvfs.c \
$(TOP)/ext/misc/completion.c \
$(TOP)/ext/misc/decimal.c \
$(TOP)/ext/misc/basexx.c \
$(TOP)/ext/misc/base64.c \
$(TOP)/ext/misc/base85.c \
$(TOP)/ext/misc/decimal.c \
$(TOP)/ext/misc/basexx.c \
$(TOP)/ext/misc/base64.c \
$(TOP)/ext/misc/base85.c \
$(TOP)/ext/misc/fileio.c \
$(TOP)/ext/misc/ieee754.c \
$(TOP)/ext/misc/regexp.c \
$(TOP)/ext/misc/series.c \
$(TOP)/ext/misc/ieee754.c \
$(TOP)/ext/misc/regexp.c \
$(TOP)/ext/misc/series.c \
$(TOP)/ext/misc/shathree.c \
$(TOP)/ext/misc/sqlar.c \
$(TOP)/ext/misc/uint.c \
$(TOP)/ext/misc/uint.c \
$(TOP)/ext/expert/sqlite3expert.c \
$(TOP)/ext/expert/sqlite3expert.h \
$(TOP)/ext/misc/zipfile.c \
@@ -1143,9 +1173,9 @@ SHELL_SRC = \
$(TOP)/ext/recover/dbdata.c \
$(TOP)/ext/recover/sqlite3recover.c \
$(TOP)/ext/recover/sqlite3recover.h \
$(TOP)/src/test_windirent.c
$(TOP)/src/test_windirent.c
shell.c: $(SHELL_SRC) $(TOP)/tool/mkshellc.tcl
shell.c: $(SHELL_SRC) $(TOP)/tool/mkshellc.tcl has_tclsh84
$(TCLSH_CMD) $(TOP)/tool/mkshellc.tcl >shell.c
@@ -1233,7 +1263,7 @@ fts5parse.c: $(TOP)/ext/fts5/fts5parse.y lemon$(BEXE)
fts5parse.h: fts5parse.c
fts5.c: $(FTS5_SRC)
fts5.c: $(FTS5_SRC) has_tclsh84
$(TCLSH_CMD) $(TOP)/ext/fts5/tool/mkfts5c.tcl
cp $(TOP)/ext/fts5/fts5.h .
@@ -1267,7 +1297,7 @@ TESTFIXTURE_SRC1 = sqlite3.c
TESTFIXTURE_SRC = $(TESTSRC) $(TOP)/src/tclsqlite.c
TESTFIXTURE_SRC += $(TESTFIXTURE_SRC$(USE_AMALGAMATION))
testfixture$(TEXE): $(TESTFIXTURE_SRC)
testfixture$(TEXE): has_tclsh85 $(TESTFIXTURE_SRC)
$(LTLINK) -DSQLITE_NO_SYNC=1 $(TEMP_STORE) $(TESTFIXTURE_FLAGS) \
-o $@ $(TESTFIXTURE_SRC) $(LIBTCL) $(TLIBS)
@@ -1291,11 +1321,17 @@ fulltestonly: $(TESTPROGS) fuzztest
./testfixture$(TEXE) $(TOP)/test/full.test
# Fuzz testing
fuzztest: fuzzcheck$(TEXE) $(FUZZDATA) sessionfuzz$(TEXE) $(TOP)/test/sessionfuzz-data1.db
#
# WARNING: When the "fuzztest" target is run by the testrunner.tcl script,
# it does not actually run this code. Instead, it schedules equivalent
# commands. Therefore, if this target is updated, then code in
# testrunner_data.tcl (search for "trd_fuzztest_data") must also be updated.
#
fuzztest: fuzzcheck$(TEXE) $(FUZZDATA) sessionfuzz$(TEXE)
./fuzzcheck$(TEXE) $(FUZZDATA)
./sessionfuzz$(TEXE) run $(TOP)/test/sessionfuzz-data1.db
valgrindfuzz: fuzzcheck$(TEXT) $(FUZZDATA) sessionfuzz$(TEXE) $(TOP)/test/sessionfuzz-data1.db
valgrindfuzz: fuzzcheck$(TEXT) $(FUZZDATA) sessionfuzz$(TEXE)
valgrind ./fuzzcheck$(TEXE) --cell-size-check --limit-mem 10M $(FUZZDATA)
valgrind ./sessionfuzz$(TEXE) run $(TOP)/test/sessionfuzz-data1.db
@@ -1312,11 +1348,23 @@ testrunner: testfixture$(TEXE)
# Runs both fuzztest and testrunner, consecutively.
#
devtest: testfixture$(TEXE) fuzztest testrunner
devtest: srctree-check testfixture$(TEXE) fuzztest testrunner
mdevtest: srctree-check has_tclsh85
$(TCLSH_CMD) $(TOP)/test/testrunner.tcl mdevtest
sdevtest: has_tclsh85
$(TCLSH_CMD) $(TOP)/test/testrunner.tcl sdevtest
# Validate that various generated files in the source tree
# are up-to-date.
#
srctree-check: $(TOP)/tool/srctree-check.tcl
$(TCLSH_CMD) $(TOP)/tool/srctree-check.tcl
# Testing for a release
#
releasetest: testfixture$(TEXE)
releasetest: srctree-check testfixture$(TEXE)
./testfixture$(TEXE) $(TOP)/test/testrunner.tcl release
# Minimal testing that runs in less than 3 minutes
@@ -1327,7 +1375,7 @@ quicktest: ./testfixture$(TEXE)
# This is the common case. Run many tests that do not take too long,
# including fuzzcheck, sqlite3_analyzer, and sqldiff tests.
#
test: fuzztest sourcetest $(TESTPROGS) tcltest
test: srctree-check fuzztest sourcetest $(TESTPROGS) tcltest
# Run a test using valgrind. This can take a really long time
# because valgrind is so much slower than a native machine.
@@ -1345,13 +1393,13 @@ smoketest: $(TESTPROGS) fuzzcheck$(TEXE)
shelltest: $(TESTPROGS)
./testfixture$(TEXT) $(TOP)/test/permutations.test shell
sqlite3_analyzer.c: sqlite3.c $(TOP)/src/tclsqlite.c $(TOP)/tool/spaceanal.tcl $(TOP)/tool/mkccode.tcl $(TOP)/tool/sqlite3_analyzer.c.in
sqlite3_analyzer.c: sqlite3.c $(TOP)/src/tclsqlite.c $(TOP)/tool/spaceanal.tcl $(TOP)/tool/mkccode.tcl $(TOP)/tool/sqlite3_analyzer.c.in has_tclsh85
$(TCLSH_CMD) $(TOP)/tool/mkccode.tcl $(TOP)/tool/sqlite3_analyzer.c.in >sqlite3_analyzer.c
sqlite3_analyzer$(TEXE): sqlite3_analyzer.c
$(LTLINK) sqlite3_analyzer.c -o $@ $(LIBTCL) $(TLIBS)
sqltclsh.c: sqlite3.c $(TOP)/src/tclsqlite.c $(TOP)/tool/sqltclsh.tcl $(TOP)/ext/misc/appendvfs.c $(TOP)/tool/mkccode.tcl $(TOP)/tool/sqltclsh.c.in
sqltclsh.c: sqlite3.c $(TOP)/src/tclsqlite.c $(TOP)/tool/sqltclsh.tcl $(TOP)/ext/misc/appendvfs.c $(TOP)/tool/mkccode.tcl $(TOP)/tool/sqltclsh.c.in has_tclsh85
$(TCLSH_CMD) $(TOP)/tool/mkccode.tcl $(TOP)/tool/sqltclsh.c.in >sqltclsh.c
sqltclsh$(TEXE): sqltclsh.c
@@ -1370,7 +1418,7 @@ CHECKER_DEPS =\
$(TOP)/ext/misc/btreeinfo.c \
$(TOP)/ext/repair/sqlite3_checker.c.in
sqlite3_checker.c: $(CHECKER_DEPS)
sqlite3_checker.c: $(CHECKER_DEPS) has_tclsh85
$(TCLSH_CMD) $(TOP)/tool/mkccode.tcl $(TOP)/ext/repair/sqlite3_checker.c.in >$@
sqlite3_checker$(TEXE): sqlite3_checker.c
@@ -1456,6 +1504,11 @@ amalgamation-tarball: sqlite3.c sqlite3rc.h
snapshot-tarball: sqlite3.c sqlite3rc.h
TOP=$(TOP) sh $(TOP)/tool/mkautoconfamal.sh --snapshot
# Build a ZIP archive containing various command-line tools.
#
tool-zip: testfixture sqlite3 sqldiff sqlite3_analyzer $(TOP)/tool/mktoolzip.tcl
./testfixture $(TOP)/tool/mktoolzip.tcl
# The next two rules are used to support the "threadtest" target. Building
# threadtest runs a few thread-safety tests that are implemented in C. This
# target is invoked by the releasetest.tcl script.
@@ -1513,6 +1566,7 @@ clean:
rm -f LogEst$(TEXE) fts3view$(TEXE) rollback-test$(TEXE) showdb$(TEXE)
rm -f showjournal$(TEXE) showstat4$(TEXE) showwal$(TEXE) speedtest1$(TEXE)
rm -f wordcount$(TEXE) changeset$(TEXE)
rm -f version-info$(TEXT)
rm -f sqlite3.dll sqlite3.lib sqlite3.exp sqlite3.def
rm -f sqlite3.c
rm -f sqlite3rc.h
@@ -1530,6 +1584,7 @@ clean:
rm -f threadtest5
rm -f src-verify
rm -f custom.rws
rm -f has_tclsh84 has_tclsh85
distclean: clean
rm -f sqlite_cfg.h config.log config.status libtool Makefile sqlite3.pc \
@@ -1574,5 +1629,12 @@ fiddle: sqlite3.c shell.c
@echo 'Updating custom dictionary from tool/custom.txt'
aspell --lang=en create master ./custom.rws < $<
misspell: ./custom.rws
misspell: ./custom.rws has_tclsh84
$(TCLSH_CMD) ./tool/spellsift.tcl ./src/*.c ./src/*.h ./src/*.in
#
# tool/version-info: a utility for emitting sqlite3 version info
# in various forms.
#
version-info$(TEXE): $(TOP)/tool/version-info.c Makefile sqlite3.h
$(LTLINK) $(ST_OPT) -o $@ $(TOP)/tool/version-info.c
-6
View File
@@ -22,12 +22,6 @@ TOP = ../sqlite
BCC = gcc -g -O0
#BCC = /opt/ancic/bin/c89 -0
#### If the target operating system supports the "usleep()" system
# call, then define the HAVE_USLEEP macro for all C modules.
#
#USLEEP =
USLEEP = -DHAVE_USLEEP=1
#### If you want the SQLite library to be safe for use within a
# multi-threaded program, then define the following macro
# appropriately:
+44 -5
View File
@@ -52,6 +52,13 @@ MINIMAL_AMALGAMATION = 0
USE_STDCALL = 0
!ENDIF
# Use the USE_SEH=0 option on the nmake command line to omit structured
# exception handling (SEH) support. SEH is on by default.
#
!IFNDEF USE_SEH
USE_SEH = 1
!ENDIF
# Set this non-0 to have the shell executable link against the core dynamic
# link library.
#
@@ -218,6 +225,12 @@ WIN32HEAP = 0
OSTRACE = 0
!ENDIF
# enable address sanitizer using ASAN=1 on the command-line.
#
!IFNDEF ASAN
ASAN = 0
!ENDIF
# Set this to one of the following values to enable various debugging
# features. Each level includes the debugging options from the previous
# levels. Currently, the recognized values for DEBUG are:
@@ -361,6 +374,7 @@ SQLITE_TCL_DEP =
!IFNDEF OPT_FEATURE_FLAGS
!IF $(MINIMAL_AMALGAMATION)==0
OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_FTS3=1
OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_FTS5=1
OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_RTREE=1
OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_GEOPOLY=1
OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_STMTVTAB=1
@@ -389,6 +403,14 @@ OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_MATH_FUNCTIONS
OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_RBU=1
!ENDIF
# Should structured exception handling (SEH) be enabled for WAL mode in
# the core library? It is on by default. Only omit it if the
# USE_SEH=0 option is provided on the nmake command-line.
#
!IF $(USE_SEH)==0
OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_OMIT_SEH=1
!ENDIF
# These are the "extended" SQLite compilation options used when compiling for
# the Windows 10 platform.
#
@@ -877,6 +899,13 @@ RCC = $(RCC) -DSQLITE_WIN32_MALLOC_VALIDATE=1
!ENDIF
!ENDIF
# Address sanitizer if ASAN=1
#
!IF $(ASAN)>0
TCC = $(TCC) /fsanitize=address
!ENDIF
# <<mark>>
# The locations of the Tcl header and library files. Also, the library that
# non-stubs enabled programs using Tcl must link against. These variables
@@ -1565,8 +1594,7 @@ TESTEXT = \
$(TOP)\ext\rtree\test_rtreedoc.c \
$(TOP)\ext\recover\sqlite3recover.c \
$(TOP)\ext\recover\test_recover.c \
$(TOP)\ext\recover\dbdata.c \
fts5.c
$(TOP)\ext\recover\dbdata.c
# If use of zlib is enabled, add the "zipfile.c" source file.
#
@@ -1582,7 +1610,8 @@ TESTSRC2 = \
$(SRC01) \
$(SRC07) \
$(SRC10) \
$(TOP)\ext\async\sqlite3async.c
$(TOP)\ext\async\sqlite3async.c \
fts5.c
# Header files used by all library source files.
#
@@ -1661,6 +1690,8 @@ SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_DQS=0
SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_FTS4=1
SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_EXPLAIN_COMMENTS=1
SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_OFFSET_SQL_FUNC=1
SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_UNKNOWN_SQL_FUNCTION=1
SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_STMT_SCANSTATUS=1
!ENDIF
# <<mark>>
@@ -1792,8 +1823,8 @@ $(SQLITE3EXE): shell.c $(SHELL_CORE_DEP) $(LIBRESOBJS) $(SHELL_CORE_SRC) $(SQLIT
/link $(SQLITE3EXEPDB) $(LDFLAGS) $(LTLINKOPTS) $(SHELL_LINK_OPTS) $(LTLIBPATHS) $(LIBRESOBJS) $(LIBREADLINE) $(LTLIBS) $(TLIBS)
# <<mark>>
sqldiff.exe: $(TOP)\tool\sqldiff.c $(SQLITE3C) $(SQLITE3H)
$(LTLINK) $(NO_WARN) $(TOP)\tool\sqldiff.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS)
sqldiff.exe: $(TOP)\tool\sqldiff.c $(SQLITE3C) $(SQLITE3H) $(LIBRESOBJS)
$(LTLINK) $(NO_WARN) $(TOP)\tool\sqldiff.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS) $(LIBRESOBJS)
dbhash.exe: $(TOP)\tool\dbhash.c $(SQLITE3C) $(SQLITE3H)
$(LTLINK) $(NO_WARN) $(TOP)\tool\dbhash.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS)
@@ -2230,6 +2261,8 @@ keywordhash.h: $(TOP)\tool\mkkeywordhash.c mkkeywordhash.exe
# Source files that go into making shell.c
SHELL_SRC = \
$(TOP)\src\shell.c.in \
$(TOP)\ext\consio\console_io.c \
$(TOP)\ext\consio\console_io.h \
$(TOP)\ext\misc\appendvfs.c \
$(TOP)\ext\misc\completion.c \
$(TOP)\ext\misc\base64.c \
@@ -2439,6 +2472,9 @@ extensiontest: testfixture.exe testloadext.dll
@set PATH=$(LIBTCLPATH);$(PATH)
.\testfixture.exe $(TOP)\test\loadext.test $(TESTOPTS)
tool-zip: testfixture.exe sqlite3.exe sqldiff.exe sqlite3_analyzer.exe $(TOP)\tool\mktoolzip.tcl
.\testfixture.exe $(TOP)\tool\mktoolzip.tcl
coretestprogs: $(TESTPROGS)
testprogs: coretestprogs srcck1.exe fuzzcheck.exe sessionfuzz.exe
@@ -2491,6 +2527,9 @@ testrunner: testfixture.exe
#
devtest: testfixture.exe fuzztest testrunner
mdevtest:
$(TCLSH_CMD) $(TOP)\test\testrunner.tcl mdevtest
# Testing for a release
#
releasetest: testfixture.exe fuzztest
+27 -27
View File
@@ -41,11 +41,10 @@ email to drh at sqlite dot org.
The SQLite source code is in the public domain. See
<https://sqlite.org/copyright.html> for details.
Because SQLite is in the public domain,
we cannot accept pull requests, because
if we did accept a pull request, the changes in that pull request would
carry a copyright and the SQLite source code would no longer be fully in
the public domain.
Because SQLite is in the public domain, we do not normally accept pull
requests, because if we did take a pull request, the changes in that
pull request might carry a copyright and the SQLite source code would
then no longer be fully in the public domain.
## Obtaining The SQLite Source Code
@@ -104,9 +103,9 @@ For example:
mkdir bld ;# Build will occur in a sibling directory
cd bld ;# Change to the build directory
../sqlite/configure ;# Run the configure script
make ;# Run the makefile.
make ;# Builds the "sqlite3" command-line tool
make sqlite3.c ;# Build the "amalgamation" source file
make test ;# Run some tests (requires Tcl)
make devtest ;# Run some tests (requires Tcl)
See the makefile for additional targets.
@@ -116,32 +115,33 @@ script does not work out for you, there is a generic makefile named
can copy and edit to suit your needs. Comments on the generic makefile
show what changes are needed.
## Using MSVC for Windows systems
## Compiling for Windows Using MSVC
On Windows, all applicable build products can be compiled with MSVC.
First open the command prompt window associated with the desired compiler
version (e.g. "Developer Command Prompt for VS2013"). Next, use NMAKE
with the provided "Makefile.msc" to build one of the supported targets.
You will also need a working installation of TCL.
See the [compile-for-windows.md](doc/compile-for-windows.md) document for
additional information about how to install MSVC and TCL and configure your
build environment.
For example, from the parent directory of the source subtree named "sqlite":
If you want to run tests, you need to let SQLite know the location of your
TCL library, using a command like this:
mkdir bld
cd bld
nmake /f ..\sqlite\Makefile.msc TOP=..\sqlite
nmake /f ..\sqlite\Makefile.msc sqlite3.c TOP=..\sqlite
nmake /f ..\sqlite\Makefile.msc sqlite3.dll TOP=..\sqlite
nmake /f ..\sqlite\Makefile.msc sqlite3.exe TOP=..\sqlite
nmake /f ..\sqlite\Makefile.msc test TOP=..\sqlite
set TCLDIR=c:\Tcl
There are several build options that can be set via the NMAKE command
line. For example, to build for WinRT, simply add "FOR_WINRT=1" argument
to the "sqlite3.dll" command line above. When debugging into the SQLite
code, adding the "DEBUG=1" argument to one of the above command lines is
recommended.
SQLite uses "tclsh.exe" as part of the build process, and so that utility
program will need to be somewhere on your %PATH%. The finished SQLite library
does not contain any TCL code, but it does use TCL to help with the build process
and to run tests.
SQLite does not require [Tcl](http://www.tcl.tk/) to run, but a Tcl installation
is required by the makefiles (including those for MSVC). SQLite contains
a lot of generated code and Tcl is used to do much of that code generation.
Build using Makefile.msc. Example:
nmake /f Makefile.msc
nmake /f Makefile.msc sqlite3.c
nmake /f Makefile.msc devtest
nmake /f Makefile.msc releasetest
There are many other makefile targets. See comments in Makefile.msc for
details.
## Source Code Tour
+1 -1
View File
@@ -1 +1 @@
3.43.0
3.45.0
+1 -1
View File
@@ -9,7 +9,7 @@ sqlite3_SOURCES = shell.c sqlite3.h
EXTRA_sqlite3_SOURCES = sqlite3.c
sqlite3_LDADD = @EXTRA_SHELL_OBJ@ @READLINE_LIBS@
sqlite3_DEPENDENCIES = @EXTRA_SHELL_OBJ@
sqlite3_CFLAGS = $(AM_CFLAGS) -DSQLITE_ENABLE_EXPLAIN_COMMENTS -DSQLITE_ENABLE_DBPAGE_VTAB -DSQLITE_ENABLE_STMTVTAB -DSQLITE_ENABLE_DBSTAT_VTAB $(SHELL_CFLAGS)
sqlite3_CFLAGS = $(AM_CFLAGS) -DSQLITE_ENABLE_EXPLAIN_COMMENTS -DSQLITE_DQS=0 -DSQLITE_ENABLE_DBPAGE_VTAB -DSQLITE_ENABLE_STMTVTAB -DSQLITE_ENABLE_DBSTAT_VTAB $(SHELL_CFLAGS)
include_HEADERS = sqlite3.h sqlite3ext.h
+31
View File
@@ -52,6 +52,13 @@ MINIMAL_AMALGAMATION = 0
USE_STDCALL = 0
!ENDIF
# Use the USE_SEH=0 option on the nmake command line to omit structured
# exception handling (SEH) support. SEH is on by default.
#
!IFNDEF USE_SEH
USE_SEH = 1
!ENDIF
# Set this non-0 to have the shell executable link against the core dynamic
# link library.
#
@@ -180,6 +187,12 @@ WIN32HEAP = 0
OSTRACE = 0
!ENDIF
# enable address sanitizer using ASAN=1 on the command-line.
#
!IFNDEF ASAN
ASAN = 0
!ENDIF
# Set this to one of the following values to enable various debugging
# features. Each level includes the debugging options from the previous
# levels. Currently, the recognized values for DEBUG are:
@@ -283,6 +296,7 @@ SQLITE3EXEPDB = /pdb:sqlite3sh.pdb
!IFNDEF OPT_FEATURE_FLAGS
!IF $(MINIMAL_AMALGAMATION)==0
OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_FTS3=1
OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_FTS5=1
OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_RTREE=1
OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_GEOPOLY=1
OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_STMTVTAB=1
@@ -311,6 +325,14 @@ OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_MATH_FUNCTIONS
OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_RBU=1
!ENDIF
# Should structured exception handling (SEH) be enabled for WAL mode in
# the core library? It is on by default. Only omit it if the
# USE_SEH=0 option is provided on the nmake command-line.
#
!IF $(USE_SEH)==0
OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_OMIT_SEH=1
!ENDIF
# These are the "extended" SQLite compilation options used when compiling for
# the Windows 10 platform.
#
@@ -718,6 +740,13 @@ RCC = $(RCC) -DSQLITE_WIN32_MALLOC_VALIDATE=1
!ENDIF
# Address sanitizer if ASAN=1
#
!IF $(ASAN)>0
TCC = $(TCC) /fsanitize=address
!ENDIF
# Compiler options needed for programs that use the readline() library.
#
!IFNDEF READLINE_FLAGS
@@ -959,6 +988,8 @@ SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_DQS=0
SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_FTS4=1
SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_EXPLAIN_COMMENTS=1
SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_OFFSET_SQL_FUNC=1
SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_UNKNOWN_SQL_FUNCTION=1
SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_STMT_SCANSTATUS=1
!ENDIF
+1 -1
View File
@@ -19,7 +19,7 @@ dnl to configure the system for the local environment.
# so that we create the export library with the dll.
#-----------------------------------------------------------------------
AC_INIT([sqlite],[3.42.0])
AC_INIT([sqlite],[3.45.0])
#--------------------------------------------------------------------
# Call TEA_INIT as the first TEA_ macro to set up initial vars.
Vendored
+48 -24
View File
@@ -1,6 +1,6 @@
#! /bin/sh
# Guess values for system-dependent variables and create Makefiles.
# Generated by GNU Autoconf 2.69 for sqlite 3.43.0.
# Generated by GNU Autoconf 2.69 for sqlite 3.45.0.
#
#
# Copyright (C) 1992-1996, 1998-2012 Free Software Foundation, Inc.
@@ -726,8 +726,8 @@ MAKEFLAGS=
# Identity of this package.
PACKAGE_NAME='sqlite'
PACKAGE_TARNAME='sqlite'
PACKAGE_VERSION='3.43.0'
PACKAGE_STRING='sqlite 3.43.0'
PACKAGE_VERSION='3.45.0'
PACKAGE_STRING='sqlite 3.45.0'
PACKAGE_BUGREPORT=''
PACKAGE_URL=''
@@ -776,6 +776,7 @@ OPT_FEATURE_FLAGS
HAVE_ZLIB
USE_AMALGAMATION
TARGET_DEBUG
TARGET_HAVE_LINENOISE
TARGET_HAVE_EDITLINE
TARGET_HAVE_READLINE
TARGET_READLINE_INC
@@ -903,6 +904,7 @@ enable_editline
enable_readline
with_readline_lib
with_readline_inc
with_linenoise
enable_debug
enable_amalgamation
enable_load_extension
@@ -1470,7 +1472,7 @@ if test "$ac_init_help" = "long"; then
# Omit some internal or obsolete options to make the list less imposing.
# This message is too long to be a string in the A/UX 3.1 sh.
cat <<_ACEOF
\`configure' configures sqlite 3.43.0 to adapt to many kinds of systems.
\`configure' configures sqlite 3.45.0 to adapt to many kinds of systems.
Usage: $0 [OPTION]... [VAR=VALUE]...
@@ -1535,7 +1537,7 @@ fi
if test -n "$ac_init_help"; then
case $ac_init_help in
short | recursive ) echo "Configuration of sqlite 3.43.0:";;
short | recursive ) echo "Configuration of sqlite 3.45.0:";;
esac
cat <<\_ACEOF
@@ -1587,6 +1589,7 @@ Optional Packages:
(tclConfig.sh)
--with-readline-lib specify readline library
--with-readline-inc specify readline include paths
--with-linenoise=DIR source directory for linenoise library
Some influential environment variables:
CC C compiler command
@@ -1665,7 +1668,7 @@ fi
test -n "$ac_init_help" && exit $ac_status
if $ac_init_version; then
cat <<\_ACEOF
sqlite configure 3.43.0
sqlite configure 3.45.0
generated by GNU Autoconf 2.69
Copyright (C) 2012 Free Software Foundation, Inc.
@@ -2084,7 +2087,7 @@ cat >config.log <<_ACEOF
This file contains any messages produced by compilers while
running configure, to aid debugging if configure makes a mistake.
It was created by sqlite $as_me 3.43.0, which was
It was created by sqlite $as_me 3.45.0, which was
generated by GNU Autoconf 2.69. Invocation command line was
$ $0 $@
@@ -3942,13 +3945,13 @@ if ${lt_cv_nm_interface+:} false; then :
else
lt_cv_nm_interface="BSD nm"
echo "int some_variable = 0;" > conftest.$ac_ext
(eval echo "\"\$as_me:3945: $ac_compile\"" >&5)
(eval echo "\"\$as_me:3948: $ac_compile\"" >&5)
(eval "$ac_compile" 2>conftest.err)
cat conftest.err >&5
(eval echo "\"\$as_me:3948: $NM \\\"conftest.$ac_objext\\\"\"" >&5)
(eval echo "\"\$as_me:3951: $NM \\\"conftest.$ac_objext\\\"\"" >&5)
(eval "$NM \"conftest.$ac_objext\"" 2>conftest.err > conftest.out)
cat conftest.err >&5
(eval echo "\"\$as_me:3951: output\"" >&5)
(eval echo "\"\$as_me:3954: output\"" >&5)
cat conftest.out >&5
if $GREP 'External.*some_variable' conftest.out > /dev/null; then
lt_cv_nm_interface="MS dumpbin"
@@ -5154,7 +5157,7 @@ ia64-*-hpux*)
;;
*-*-irix6*)
# Find out which ABI we are using.
echo '#line 5157 "configure"' > conftest.$ac_ext
echo '#line 5160 "configure"' > conftest.$ac_ext
if { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$ac_compile\""; } >&5
(eval $ac_compile) 2>&5
ac_status=$?
@@ -6679,11 +6682,11 @@ else
-e 's:.*FLAGS}\{0,1\} :&$lt_compiler_flag :; t' \
-e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \
-e 's:$: $lt_compiler_flag:'`
(eval echo "\"\$as_me:6682: $lt_compile\"" >&5)
(eval echo "\"\$as_me:6685: $lt_compile\"" >&5)
(eval "$lt_compile" 2>conftest.err)
ac_status=$?
cat conftest.err >&5
echo "$as_me:6686: \$? = $ac_status" >&5
echo "$as_me:6689: \$? = $ac_status" >&5
if (exit $ac_status) && test -s "$ac_outfile"; then
# The compiler can only warn and ignore the option if not recognized
# So say no if there are warnings other than the usual output.
@@ -7018,11 +7021,11 @@ else
-e 's:.*FLAGS}\{0,1\} :&$lt_compiler_flag :; t' \
-e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \
-e 's:$: $lt_compiler_flag:'`
(eval echo "\"\$as_me:7021: $lt_compile\"" >&5)
(eval echo "\"\$as_me:7024: $lt_compile\"" >&5)
(eval "$lt_compile" 2>conftest.err)
ac_status=$?
cat conftest.err >&5
echo "$as_me:7025: \$? = $ac_status" >&5
echo "$as_me:7028: \$? = $ac_status" >&5
if (exit $ac_status) && test -s "$ac_outfile"; then
# The compiler can only warn and ignore the option if not recognized
# So say no if there are warnings other than the usual output.
@@ -7123,11 +7126,11 @@ else
-e 's:.*FLAGS}\{0,1\} :&$lt_compiler_flag :; t' \
-e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \
-e 's:$: $lt_compiler_flag:'`
(eval echo "\"\$as_me:7126: $lt_compile\"" >&5)
(eval echo "\"\$as_me:7129: $lt_compile\"" >&5)
(eval "$lt_compile" 2>out/conftest.err)
ac_status=$?
cat out/conftest.err >&5
echo "$as_me:7130: \$? = $ac_status" >&5
echo "$as_me:7133: \$? = $ac_status" >&5
if (exit $ac_status) && test -s out/conftest2.$ac_objext
then
# The compiler can only warn and ignore the option if not recognized
@@ -7178,11 +7181,11 @@ else
-e 's:.*FLAGS}\{0,1\} :&$lt_compiler_flag :; t' \
-e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \
-e 's:$: $lt_compiler_flag:'`
(eval echo "\"\$as_me:7181: $lt_compile\"" >&5)
(eval echo "\"\$as_me:7184: $lt_compile\"" >&5)
(eval "$lt_compile" 2>out/conftest.err)
ac_status=$?
cat out/conftest.err >&5
echo "$as_me:7185: \$? = $ac_status" >&5
echo "$as_me:7188: \$? = $ac_status" >&5
if (exit $ac_status) && test -s out/conftest2.$ac_objext
then
# The compiler can only warn and ignore the option if not recognized
@@ -9558,7 +9561,7 @@ else
lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2
lt_status=$lt_dlunknown
cat > conftest.$ac_ext <<_LT_EOF
#line 9561 "configure"
#line 9564 "configure"
#include "confdefs.h"
#if HAVE_DLFCN_H
@@ -9654,7 +9657,7 @@ else
lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2
lt_status=$lt_dlunknown
cat > conftest.$ac_ext <<_LT_EOF
#line 9657 "configure"
#line 9660 "configure"
#include "confdefs.h"
#if HAVE_DLFCN_H
@@ -11245,6 +11248,27 @@ fi
fi
fi
# Check whether --with-linenoise was given.
if test "${with_linenoise+set}" = set; then :
withval=$with_linenoise; with_linenoise=$withval
else
with_linenoise="no"
fi
if test "x$with_linenoise" != "xno"; then
TARGET_HAVE_READLINE=0
TARGET_HAVE_EDITLINE=0
TARGET_HAVE_LINENOISE=1
TARGET_READLINE_INC="-I${with_linenoise}"
TARGET_READLINE_LIBS="${with_linenoise}/linenoise.c"
echo "using linenoise source code at ${with_linenoise}"
else
TARGET_HAVE_LINENOISE=0
echo "not using linenoise"
fi
@@ -11321,7 +11345,7 @@ fi
{ $as_echo "$as_me:${as_lineno-$LINENO}: checking build type" >&5
$as_echo_n "checking build type... " >&6; }
if test "${enable_debug}" = "yes" ; then
TARGET_DEBUG="-DSQLITE_DEBUG=1 -DSQLITE_ENABLE_SELECTTRACE -DSQLITE_ENABLE_WHERETRACE -O0"
TARGET_DEBUG="-DSQLITE_DEBUG=1 -DSQLITE_ENABLE_SELECTTRACE -DSQLITE_ENABLE_WHERETRACE -O0 -Wall"
{ $as_echo "$as_me:${as_lineno-$LINENO}: result: debug" >&5
$as_echo "debug" >&6; }
else
@@ -12457,7 +12481,7 @@ cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1
# report actual input values of CONFIG_FILES etc. instead of their
# values after options handling.
ac_log="
This file was extended by sqlite $as_me 3.43.0, which was
This file was extended by sqlite $as_me 3.45.0, which was
generated by GNU Autoconf 2.69. Invocation command line was
CONFIG_FILES = $CONFIG_FILES
@@ -12523,7 +12547,7 @@ _ACEOF
cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1
ac_cs_config="`$as_echo "$ac_configure_args" | sed 's/^ //; s/[\\""\`\$]/\\\\&/g'`"
ac_cs_version="\\
sqlite config.status 3.43.0
sqlite config.status 3.45.0
configured by $0, generated by GNU Autoconf 2.69,
with options \\"\$ac_cs_config\\"
+19 -2
View File
@@ -74,7 +74,7 @@
# you don't need (for example BLT) by erasing or commenting out
# the corresponding code.
#
AC_INIT([sqlite],[m4_esyscmd(cat VERSION | tr -d '\n')])
AC_INIT([sqlite],m4_esyscmd(cat VERSION | tr -d '\n'))
dnl Make sure the local VERSION file matches this configure script
sqlite_version_sanity_check=`cat $srcdir/VERSION | tr -d '\n'`
@@ -598,11 +598,28 @@ if test x"$with_readline" != xno; then
TARGET_HAVE_READLINE=1
fi
fi
AC_ARG_WITH([linenoise],
[AS_HELP_STRING([--with-linenoise=DIR],[source directory for linenoise library])],
[with_linenoise=$withval],
[with_linenoise="no"])
if test "x$with_linenoise" != "xno"; then
TARGET_HAVE_READLINE=0
TARGET_HAVE_EDITLINE=0
TARGET_HAVE_LINENOISE=1
TARGET_READLINE_INC="-I${with_linenoise}"
TARGET_READLINE_LIBS="${with_linenoise}/linenoise.c"
echo "using linenoise source code at ${with_linenoise}"
else
TARGET_HAVE_LINENOISE=0
echo "not using linenoise"
fi
AC_SUBST(TARGET_READLINE_LIBS)
AC_SUBST(TARGET_READLINE_INC)
AC_SUBST(TARGET_HAVE_READLINE)
AC_SUBST(TARGET_HAVE_EDITLINE)
AC_SUBST(TARGET_HAVE_LINENOISE)
##########
# Figure out what C libraries are required to compile programs
@@ -615,7 +632,7 @@ AC_SEARCH_LIBS(fdatasync, [rt])
AC_ARG_ENABLE(debug, AS_HELP_STRING([--enable-debug],[enable debugging & verbose explain]))
AC_MSG_CHECKING([build type])
if test "${enable_debug}" = "yes" ; then
TARGET_DEBUG="-DSQLITE_DEBUG=1 -DSQLITE_ENABLE_SELECTTRACE -DSQLITE_ENABLE_WHERETRACE -O0"
TARGET_DEBUG="-DSQLITE_DEBUG=1 -DSQLITE_ENABLE_SELECTTRACE -DSQLITE_ENABLE_WHERETRACE -O0 -Wall"
AC_MSG_RESULT([debug])
else
TARGET_DEBUG="-DNDEBUG"
+148
View File
@@ -0,0 +1,148 @@
# Notes On Compiling SQLite On Windows 11
Here are step-by-step instructions on how to build SQLite from
canonical source on a new Windows 11 PC, as of 2023-11-01:
1. Install Microsoft Visual Studio. The free "community edition"
will work fine. Do a standard install for C++ development.
SQLite only needs the
"cl" compiler and the "nmake" build tool.
2. Under the "Start" menu, find "All Apps" then go to "Visual Studio 20XX"
and find "x64 Native Tools Command Prompt for VS 20XX". Pin that
application to your task bar, as you will use it a lot. Bring up
an instance of this command prompt and do all of the subsequent steps
in that "x64 Native Tools" command prompt. (Or use "x86" if you want
a 32-bit build.) The subsequent steps will not work in a vanilla
DOS prompt. Nor will they work in PowerShell.
3. Install TCL development libraries. This note assumes that you will
install the TCL development libraries in the "`c:\Tcl`" directory.
Make adjustments
if you want TCL installed somewhere else. SQLite needs both the
"tclsh.exe" command-line tool as part of the build process, and
the "tcl86.lib" library in order to run tests. You will need
TCL version 8.6 or later.
<ol type="a">
<li>Get the TCL source archive, perhaps from
[https://www.tcl.tk/software/tcltk/download.html](https://www.tcl.tk/software/tcltk/download.html).
<li>Untar or unzip the source archive. CD into the "win/" subfolder
of the source tree.
<li>Run: `nmake /f makefile.vc release`
<li>Run: `nmake /f makefile.vc INSTALLDIR=c:\Tcl install`
<li>CD to `c:\Tcl\lib`. In that subfolder make a copy of the
"`tcl86t.lib`" file to the alternative name "`tcl86.lib`"
(omitting the second 't'). Leave the copy in the same directory
as the original.
<li>CD to `c:\Tcl\bin`. Make a copy of the "`tclsh86t.exe`"
file into "`tclsh.exe`" (without the "86t") in the same directory.
<li>Add `c:\Tcl\bin` to your %PATH%. To do this, go to Settings
and search for "path". Select "edit environment variables for
your account" and modify your default PATH accordingly.
You will need to close and reopen your command prompts after
making this change.
</ol>
4. Download the SQLite source tree and unpack it. CD into the
toplevel directory of the source tree.
5. Set the TCLDIR environment variable to point to your TCL installation.
Like this:
<ul>
<li> `set TCLDIR=c:\Tcl`
</ul>
6. Run the "`Makefile.msc`" makefile with an appropriate target.
Examples:
<ul>
<li> `nmake /f makefile.msc`
<li> `nmake /f makefile.msc sqlite3.c`
<li> `nmake /f makefile.msc devtest`
<li> `nmake /f makefile.msc releasetest`
</ul>
## 32-bit Builds
Doing a 32-bit build is just like doing a 64-bit build with the
following minor changes:
1. Use the "x86 Native Tools Command Prompt" instead of
"x64 Native Tools Command Prompt". "**x86**" instead of "**x64**".
2. Use a different installation directory for TCL.
The recommended directory is `c:\tcl32`. Thus you end up
with two TCL builds:
<ul>
<li> `c:\tcl` &larr; 64-bit (the default)
<li> `c:\tcl32` &larr; 32-bit
</ul>
3. Ensure that `c:\tcl32\bin` comes before `c:\tcl\bin` on
your PATH environment variable. You can achieve this using
a command like:
<ul>
<li> `set PATH=c:\tcl32\bin;%PATH%`
</ul>
## Building a DLL
The command the developers use for building the deliverable DLL on the
[download page](https://sqlite.org/download.html) is as follows:
> ~~~~
nmake /f Makefile.msc sqlite3.dll USE_NATIVE_LIBPATHS=1 "OPTS=-DSQLITE_ENABLE_FTS3=1 -DSQLITE_ENABLE_FTS4=1 -DSQLITE_ENABLE_FTS5=1 -DSQLITE_ENABLE_RTREE=1 -DSQLITE_ENABLE_JSON1=1 -DSQLITE_ENABLE_GEOPOLY=1 -DSQLITE_ENABLE_SESSION=1 -DSQLITE_ENABLE_PREUPDATE_HOOK=1 -DSQLITE_ENABLE_SERIALIZE=1 -DSQLITE_ENABLE_MATH_FUNCTIONS=1"
~~~~
That command generates both the sqlite3.dll and sqlite3.def files. The same
command works for both 32-bit and 64-bit builds.
## Statically Linking The TCL Library
Some utility programs associated with SQLite need to be linked
with TCL in order to function. The [sqlite3_analyzer.exe program](https://sqlite.org/sqlanalyze.html)
is an example. You can build as described above, and then
enter:
> ~~~~
nmake /f Makefile.msc sqlite3_analyzer.exe
~~~~
And you will end up with a working executable. However, that executable
will depend on having the "tcl86.dll" library somewhere on your %PATH%.
Use the following steps to build an executable that has the TCL library
statically linked so that it does not depend on separate DLL:
1. Use the appropriate "Command Prompt" window - either x86 or
x64, depending on whether you want a 32-bit or 64-bit executable.
2. Untar the TCL source tarball into a fresh directory. CD into
the "win/" subfolder.
3. Run: `nmake /f makefile.vc OPTS=nothreads,static shell`
4. CD into the "Release*" subfolder that is created (note the
wildcard - the full name of the directory might vary). There
you will find the "tcl86s.lib" file. Copy this file into the
same directory that you put the "tcl86.lib" on your initial
installation. (In this document, that directory is
"`C:\Tcl32\lib`" for 32-bit builds and
"`C:\Tcl\lib`" for 64-bit builds.)
5. CD into your SQLite source code directory and build the desired
utility program, but add the following extra arguments to the
nmake command line:
<blockquote><pre>
CCOPTS="-DSTATIC_BUILD" LIBTCL="tcl86s.lib netapi32.lib user32.lib"
</pre></blockquote>
<p>So, for example, to build a statically linked version of
sqlite3_analyzer.exe, you might type:
<blockquote><pre>
nmake /f Makefile.msc CCOPTS="-DSTATIC_BUILD" LIBTCL="tcl86s.lib netapi32.lib user32.lib" sqlite3_analyzer.exe
</pre></blockquote>
6. After your executable is built, you can verify that it does not
depend on the TCL DLL by running:
<blockquote><pre>
dumpbin /dependents sqlite3_analyzer.exe
</pre></blockquote>
+2 -2
View File
@@ -1229,8 +1229,8 @@ first syntax error, of course, if there are no instances of the
<p>Lemon was originally written by Richard Hipp sometime in the late
1980s on a Sun4 Workstation using K&amp;R C.
There was a companion LL(1) parser generator program named "Lime", the
source code to which as been lost.</p>
There was a companion LL(1) parser generator program named "Lime".
The Lime source code has been lost.</p>
<p>The lemon.c source file was originally many separate files that were
compiled together to generate the "lemon" executable. Sometime in the
+284
View File
@@ -0,0 +1,284 @@
# The testrunner.tcl Script
# 1. Overview
testrunner.tcl is a Tcl script used to run multiple SQLite tests using
multiple jobs. It supports the following types of tests:
* Tcl test scripts.
* Tests run with [make] commands. Specifically, at time of writing,
[make fuzztest], [make mptest], [make sourcetest] and [make threadtest].
testrunner.tcl pipes the output of all tests and builds run into log file
**testrunner.log**, created in the cwd directory. Searching this file for
"failed" is a good way to find the output of a failed test.
testrunner.tcl also populates SQLite database **testrunner.db**. This database
contains details of all tests run, running and to be run. A useful query
might be:
```
SELECT * FROM script WHERE state='failed'
```
Running the command:
```
./testfixture $(TESTDIR)/testrunner.tcl status
```
in the directory containing the testrunner.db database runs various queries
to produce a succinct report on the state of a running testrunner.tcl script.
Running:
```
watch ./testfixture $(TESTDIR)/testrunner.tcl status
```
in another terminal is a good way to keep an eye on a long running test.
Sometimes testrunner.tcl uses the [testfixture] binary that it is run with
to run tests (see "Binary Tests" below). Sometimes it builds testfixture and
other binaries in specific configurations to test (see "Source Tests").
# 2. Binary Tests
The commands described in this section all run various combinations of the Tcl
test scripts using the [testfixture] binary used to run the testrunner.tcl
script (i.e. they do not invoke the compiler to build new binaries, or the
[make] command to run tests that are not Tcl scripts). The procedure to run
these tests is therefore:
1. Build the "testfixture" (or "testfixture.exe" for windows) binary using
whatever method seems convenient.
2. Test the binary built in step 1 by running testrunner.tcl with it,
perhaps with various options.
The following sub-sections describe the various options that can be
passed to testrunner.tcl to test binary testfixture builds.
## 2.1. Organization of Tcl Tests
Tcl tests are stored in files that match the pattern *\*.test*. They are
found in both the $TOP/test/ directory, and in the various sub-directories
of the $TOP/ext/ directory of the source tree. Not all *\*.test* files
contain Tcl tests - a handful are Tcl scripts designed to invoke other
*\*.test* files.
The **veryquick** set of tests is a subset of all Tcl test scripts in the
source tree. In includes most tests, but excludes some that are very slow.
Almost all fault-injection tests (those that test the response of the library
to OOM or IO errors) are excluded. It is defined in source file
*test/permutations.test*.
The **full** set of tests includes all Tcl test scripts in the source tree.
To run a "full" test is to run all Tcl test scripts that can be found in the
source tree.
File *permutations.test* defines various test "permutations". A permutation
consists of:
* A subset of Tcl test scripts, and
* Runtime configuration to apply before running each test script
(e.g. enabling auto-vacuum, or disable lookaside).
Running **all** tests is to run all tests in the full test set, plus a dozen
or so permutations. The specific permutations that are run as part of "all"
are defined in file *testrunner_data.tcl*.
## 2.2. Commands to Run Tests
To run the "veryquick" test set, use either of the following:
```
./testfixture $TESTDIR/testrunner.tcl
./testfixture $TESTDIR/testrunner.tcl veryquick
```
To run the "full" test suite:
```
./testfixture $TESTDIR/testrunner.tcl full
```
To run the subset of the "full" test suite for which the test file name matches
a specified pattern (e.g. all tests that start with "fts5"), either of:
```
./testfixture $TESTDIR/testrunner.tcl fts5%
./testfixture $TESTDIR/testrunner.tcl 'fts5*'
```
To run "all" tests (full + permutations):
```
./testfixture $TESTDIR/testrunner.tcl all
```
<a name=binary_test_failures></a>
## 2.3. Investigating Binary Test Failures
If a test fails, testrunner.tcl reports name of the Tcl test script and, if
applicable, the name of the permutation, to stdout. This information can also
be retrieved from either *testrunner.log* or *testrunner.db*.
If there is no permutation, the individual test script may be run with:
```
./testfixture $PATH_TO_SCRIPT
```
Or, if the failure occured as part of a permutation:
```
./testfixture $TESTDIR/testrunner.tcl $PERMUTATION $PATH_TO_SCRIPT
```
TODO: An example instead of "$PERMUTATION" and $PATH\_TO\_SCRIPT?
# 3. Source Code Tests
The commands described in this section invoke the C compiler to build
binaries from the source tree, then use those binaries to run Tcl and
other tests. The advantages of this are that:
* it is possible to test multiple build configurations with a single
command, and
* it ensures that tests are always run using binaries created with the
same set of compiler options.
The testrunner.tcl commands described in this section may be run using
either a *testfixture* (or testfixture.exe) build, or with any other Tcl
shell that supports SQLite 3.31.1 or newer via "package require sqlite3".
TODO: ./configure + Makefile.msc build systems.
## Commands to Run SQLite Tests
The **mdevtest** command is equivalent to running the veryquick tests and
the [make fuzztest] target once for each of two --enable-all builds - one
with debugging enabled and one without:
```
tclsh $TESTDIR/testrunner.tcl mdevtest
```
In other words, it is equivalent to running:
```
$TOP/configure --enable-all --enable-debug
make fuzztest
make testfixture
./testfixture $TOP/test/testrunner.tcl veryquick
# Then, after removing files created by the tests above:
$TOP/configure --enable-all OPTS="-O0"
make fuzztest
make testfixture
./testfixture $TOP/test/testrunner.tcl veryquick
```
The **sdevtest** command is identical to the mdevtest command, except that the
second of the two builds is a sanitizer build. Specifically, this means that
OPTS="-fsanitize=address,undefined" is specified instead of OPTS="-O0":
```
tclsh $TESTDIR/testrunner.tcl sdevtest
```
The **release** command runs lots of tests under lots of builds. It runs
different combinations of builds and tests depending on whether it is run
on Linux, Windows or OSX. Refer to *testrunner\_data.tcl* for the details
of the specific tests run.
```
tclsh $TESTDIR/testrunner.tcl release
```
## Running ZipVFS Tests
testrunner.tcl can build a zipvfs-enabled testfixture and use it to run
tests from the Zipvfs project with the following command:
```
tclsh $TESTDIR/testrunner.tcl --zipvfs $PATH_TO_ZIPVFS
```
This can be combined with any of "mdevtest", "sdevtest" or "release" to
test both SQLite and Zipvfs with a single command:
```
tclsh $TESTDIR/testrunner.tcl --zipvfs $PATH_TO_ZIPVFS mdevtest
```
## Investigating Source Code Test Failures
Investigating a test failure that occurs during source code testing is a
two step process:
1. Recreating the build configuration in which the test failed, and
2. Re-running the actual test.
To recreate a build configuration, use the testrunner.tcl **script** command
to create a build script. A build script is a bash script on Linux or OSX, or
a dos \*.bat file on windows. For example:
```
# Create a script that recreates build configuration "Device-One" on
# Linux or OSX:
tclsh $TESTDIR/testrunner.tcl script Device-One > make.sh
# Create a script that recreates build configuration "Have-Not" on Windows:
tclsh $TESTDIR/testrunner.tcl script Have-Not > make.bat
```
The generated bash or \*.bat file script accepts a single argument - a makefile
target to build. This may be used either to run a [make] command test directly,
or else to build a testfixture (or testfixture.exe) binary with which to
run a Tcl test script, as <a href=#binary_test_failures>described above</a>.
# 4. Controlling CPU Core Utilization
When running either binary or source code tests, testrunner.tcl reports the
number of jobs it intends to use to stdout. e.g.
```
$ ./testfixture $TESTDIR/testrunner.tcl
splitting work across 16 jobs
... more output ...
```
By default, testfixture.tcl attempts to set the number of jobs to the number
of real cores on the machine. This can be overridden using the "--jobs" (or -j)
switch:
```
$ ./testfixture $TESTDIR/testrunner.tcl --jobs 8
splitting work across 8 jobs
... more output ...
```
The number of jobs may also be changed while an instance of testrunner.tcl is
running by exucuting the following command from the directory containing the
testrunner.log and testrunner.db files:
```
$ ./testfixture $TESTDIR/testrunner.tcl njob $NEW_NUMBER_OF_JOBS
```
+678
View File
@@ -0,0 +1,678 @@
/*
** 2023 November 4
**
** The author disclaims copyright to this source code. In place of
** a legal notice, here is a blessing:
**
** May you do good and not evil.
** May you find forgiveness for yourself and forgive others.
** May you share freely, never taking more than you give.
**
********************************************************************************
** This file implements various interfaces used for console and stream I/O
** by the SQLite project command-line tools, as explained in console_io.h .
** Functions prefixed by "SQLITE_INTERNAL_LINKAGE" behave as described there.
*/
#ifndef SQLITE_CDECL
# define SQLITE_CDECL
#endif
#ifndef SHELL_NO_SYSINC
# include <stdarg.h>
# include <string.h>
# include <stdlib.h>
# include <limits.h>
# include <assert.h>
# include "console_io.h"
# include "sqlite3.h"
#endif
#ifndef SQLITE_CIO_NO_TRANSLATE
# if (defined(_WIN32) || defined(WIN32)) && !SQLITE_OS_WINRT
# ifndef SHELL_NO_SYSINC
# include <io.h>
# include <fcntl.h>
# undef WIN32_LEAN_AND_MEAN
# define WIN32_LEAN_AND_MEAN
# include <windows.h>
# endif
# define CIO_WIN_WC_XLATE 1 /* Use WCHAR Windows APIs for console I/O */
# else
# ifndef SHELL_NO_SYSINC
# include <unistd.h>
# endif
# define CIO_WIN_WC_XLATE 0 /* Use plain C library stream I/O at console */
# endif
#else
# define CIO_WIN_WC_XLATE 0 /* Not exposing translation routines at all */
#endif
#if CIO_WIN_WC_XLATE
/* Character used to represent a known-incomplete UTF-8 char group () */
static WCHAR cBadGroup = 0xfffd;
#endif
#if CIO_WIN_WC_XLATE
static HANDLE handleOfFile(FILE *pf){
int fileDesc = _fileno(pf);
union { intptr_t osfh; HANDLE fh; } fid = {
(fileDesc>=0)? _get_osfhandle(fileDesc) : (intptr_t)INVALID_HANDLE_VALUE
};
return fid.fh;
}
#endif
#ifndef SQLITE_CIO_NO_TRANSLATE
typedef struct PerStreamTags {
# if CIO_WIN_WC_XLATE
HANDLE hx;
DWORD consMode;
char acIncomplete[4];
# else
short reachesConsole;
# endif
FILE *pf;
} PerStreamTags;
/* Define NULL-like value for things which can validly be 0. */
# define SHELL_INVALID_FILE_PTR ((FILE *)~0)
# if CIO_WIN_WC_XLATE
# define SHELL_INVALID_CONS_MODE 0xFFFF0000
# endif
# if CIO_WIN_WC_XLATE
# define PST_INITIALIZER { INVALID_HANDLE_VALUE, SHELL_INVALID_CONS_MODE, \
{0,0,0,0}, SHELL_INVALID_FILE_PTR }
# else
# define PST_INITIALIZER { 0, SHELL_INVALID_FILE_PTR }
# endif
/* Quickly say whether a known output is going to the console. */
# if CIO_WIN_WC_XLATE
static short pstReachesConsole(PerStreamTags *ppst){
return (ppst->hx != INVALID_HANDLE_VALUE);
}
# else
# define pstReachesConsole(ppst) 0
# endif
# if CIO_WIN_WC_XLATE
static void restoreConsoleArb(PerStreamTags *ppst){
if( pstReachesConsole(ppst) ) SetConsoleMode(ppst->hx, ppst->consMode);
}
# else
# define restoreConsoleArb(ppst)
# endif
/* Say whether FILE* appears to be a console, collect associated info. */
static short streamOfConsole(FILE *pf, /* out */ PerStreamTags *ppst){
# if CIO_WIN_WC_XLATE
short rv = 0;
DWORD dwCM = SHELL_INVALID_CONS_MODE;
HANDLE fh = handleOfFile(pf);
ppst->pf = pf;
if( INVALID_HANDLE_VALUE != fh ){
rv = (GetFileType(fh) == FILE_TYPE_CHAR && GetConsoleMode(fh,&dwCM));
}
ppst->hx = (rv)? fh : INVALID_HANDLE_VALUE;
ppst->consMode = dwCM;
return rv;
# else
ppst->pf = pf;
ppst->reachesConsole = ( (short)isatty(fileno(pf)) );
return ppst->reachesConsole;
# endif
}
# if CIO_WIN_WC_XLATE
/* Define console modes for use with the Windows Console API. */
# define SHELL_CONI_MODE \
(ENABLE_ECHO_INPUT | ENABLE_INSERT_MODE | ENABLE_LINE_INPUT | 0x80 \
| ENABLE_QUICK_EDIT_MODE | ENABLE_EXTENDED_FLAGS | ENABLE_PROCESSED_INPUT)
# define SHELL_CONO_MODE (ENABLE_PROCESSED_OUTPUT | ENABLE_WRAP_AT_EOL_OUTPUT \
| ENABLE_VIRTUAL_TERMINAL_PROCESSING)
# endif
typedef struct ConsoleInfo {
PerStreamTags pstSetup[3];
PerStreamTags pstDesignated[3];
StreamsAreConsole sacSetup;
} ConsoleInfo;
static short isValidStreamInfo(PerStreamTags *ppst){
return (ppst->pf != SHELL_INVALID_FILE_PTR);
}
static ConsoleInfo consoleInfo = {
{ /* pstSetup */ PST_INITIALIZER, PST_INITIALIZER, PST_INITIALIZER },
{ /* pstDesignated[] */ PST_INITIALIZER, PST_INITIALIZER, PST_INITIALIZER },
SAC_NoConsole /* sacSetup */
};
SQLITE_INTERNAL_LINKAGE FILE* invalidFileStream = (FILE *)~0;
# if CIO_WIN_WC_XLATE
static void maybeSetupAsConsole(PerStreamTags *ppst, short odir){
if( pstReachesConsole(ppst) ){
DWORD cm = odir? SHELL_CONO_MODE : SHELL_CONI_MODE;
SetConsoleMode(ppst->hx, cm);
}
}
# else
# define maybeSetupAsConsole(ppst,odir)
# endif
SQLITE_INTERNAL_LINKAGE void consoleRenewSetup(void){
# if CIO_WIN_WC_XLATE
int ix = 0;
while( ix < 6 ){
PerStreamTags *ppst = (ix<3)?
&consoleInfo.pstSetup[ix] : &consoleInfo.pstDesignated[ix-3];
maybeSetupAsConsole(ppst, (ix % 3)>0);
++ix;
}
# endif
}
SQLITE_INTERNAL_LINKAGE StreamsAreConsole
consoleClassifySetup( FILE *pfIn, FILE *pfOut, FILE *pfErr ){
StreamsAreConsole rv = SAC_NoConsole;
FILE* apf[3] = { pfIn, pfOut, pfErr };
int ix;
for( ix = 2; ix >= 0; --ix ){
PerStreamTags *ppst = &consoleInfo.pstSetup[ix];
if( streamOfConsole(apf[ix], ppst) ){
rv |= (SAC_InConsole<<ix);
}
consoleInfo.pstDesignated[ix] = *ppst;
if( ix > 0 ) fflush(apf[ix]);
}
consoleInfo.sacSetup = rv;
consoleRenewSetup();
return rv;
}
SQLITE_INTERNAL_LINKAGE void SQLITE_CDECL consoleRestore( void ){
# if CIO_WIN_WC_XLATE
static ConsoleInfo *pci = &consoleInfo;
if( pci->sacSetup ){
int ix;
for( ix=0; ix<3; ++ix ){
if( pci->sacSetup & (SAC_InConsole<<ix) ){
PerStreamTags *ppst = &pci->pstSetup[ix];
SetConsoleMode(ppst->hx, ppst->consMode);
}
}
}
# endif
}
#endif /* !defined(SQLITE_CIO_NO_TRANSLATE) */
#ifdef SQLITE_CIO_INPUT_REDIR
/* Say whether given FILE* is among those known, via either
** consoleClassifySetup() or set{Output,Error}Stream, as
** readable, and return an associated PerStreamTags pointer
** if so. Otherwise, return 0.
*/
static PerStreamTags * isKnownReadable(FILE *pf){
static PerStreamTags *apst[] = {
&consoleInfo.pstDesignated[0], &consoleInfo.pstSetup[0], 0
};
int ix = 0;
do {
if( apst[ix]->pf == pf ) break;
} while( apst[++ix] != 0 );
return apst[ix];
}
#endif
#ifndef SQLITE_CIO_NO_TRANSLATE
/* Say whether given FILE* is among those known, via either
** consoleClassifySetup() or set{Output,Error}Stream, as
** writable, and return an associated PerStreamTags pointer
** if so. Otherwise, return 0.
*/
static PerStreamTags * isKnownWritable(FILE *pf){
static PerStreamTags *apst[] = {
&consoleInfo.pstDesignated[1], &consoleInfo.pstDesignated[2],
&consoleInfo.pstSetup[1], &consoleInfo.pstSetup[2], 0
};
int ix = 0;
do {
if( apst[ix]->pf == pf ) break;
} while( apst[++ix] != 0 );
return apst[ix];
}
static FILE *designateEmitStream(FILE *pf, unsigned chix){
FILE *rv = consoleInfo.pstDesignated[chix].pf;
if( pf == invalidFileStream ) return rv;
else{
/* Setting a possibly new output stream. */
PerStreamTags *ppst = isKnownWritable(pf);
if( ppst != 0 ){
PerStreamTags pst = *ppst;
consoleInfo.pstDesignated[chix] = pst;
}else streamOfConsole(pf, &consoleInfo.pstDesignated[chix]);
}
return rv;
}
SQLITE_INTERNAL_LINKAGE FILE *setOutputStream(FILE *pf){
return designateEmitStream(pf, 1);
}
# ifdef CONSIO_SET_ERROR_STREAM
SQLITE_INTERNAL_LINKAGE FILE *setErrorStream(FILE *pf){
return designateEmitStream(pf, 2);
}
# endif
#endif /* !defined(SQLITE_CIO_NO_TRANSLATE) */
#ifndef SQLITE_CIO_NO_SETMODE
# if CIO_WIN_WC_XLATE
static void setModeFlushQ(FILE *pf, short bFlush, int mode){
if( bFlush ) fflush(pf);
_setmode(_fileno(pf), mode);
}
# else
# define setModeFlushQ(f, b, m) if(b) fflush(f)
# endif
SQLITE_INTERNAL_LINKAGE void setBinaryMode(FILE *pf, short bFlush){
setModeFlushQ(pf, bFlush, _O_BINARY);
}
SQLITE_INTERNAL_LINKAGE void setTextMode(FILE *pf, short bFlush){
setModeFlushQ(pf, bFlush, _O_TEXT);
}
# undef setModeFlushQ
#else /* defined(SQLITE_CIO_NO_SETMODE) */
# define setBinaryMode(f, bFlush) do{ if((bFlush)) fflush(f); }while(0)
# define setTextMode(f, bFlush) do{ if((bFlush)) fflush(f); }while(0)
#endif /* defined(SQLITE_CIO_NO_SETMODE) */
#ifndef SQLITE_CIO_NO_TRANSLATE
# if CIO_WIN_WC_XLATE
/* Write buffer cBuf as output to stream known to reach console,
** limited to ncTake char's. Return ncTake on success, else 0. */
static int conZstrEmit(PerStreamTags *ppst, const char *z, int ncTake){
int rv = 0;
if( z!=NULL ){
int nwc = MultiByteToWideChar(CP_UTF8,0, z,ncTake, 0,0);
if( nwc > 0 ){
WCHAR *zw = sqlite3_malloc64(nwc*sizeof(WCHAR));
if( zw!=NULL ){
nwc = MultiByteToWideChar(CP_UTF8,0, z,ncTake, zw,nwc);
if( nwc > 0 ){
/* Translation from UTF-8 to UTF-16, then WCHARs out. */
if( WriteConsoleW(ppst->hx, zw,nwc, 0, NULL) ){
rv = ncTake;
}
}
sqlite3_free(zw);
}
}
}
return rv;
}
/* For {f,o,e}PrintfUtf8() when stream is known to reach console. */
static int conioVmPrintf(PerStreamTags *ppst, const char *zFormat, va_list ap){
char *z = sqlite3_vmprintf(zFormat, ap);
if( z ){
int rv = conZstrEmit(ppst, z, (int)strlen(z));
sqlite3_free(z);
return rv;
}else return 0;
}
# endif /* CIO_WIN_WC_XLATE */
# ifdef CONSIO_GET_EMIT_STREAM
static PerStreamTags * getDesignatedEmitStream(FILE *pf, unsigned chix,
PerStreamTags *ppst){
PerStreamTags *rv = isKnownWritable(pf);
short isValid = (rv!=0)? isValidStreamInfo(rv) : 0;
if( rv != 0 && isValid ) return rv;
streamOfConsole(pf, ppst);
return ppst;
}
# endif
/* Get stream info, either for designated output or error stream when
** chix equals 1 or 2, or for an arbitrary stream when chix == 0.
** In either case, ppst references a caller-owned PerStreamTags
** struct which may be filled in if none of the known writable
** streams is being held by consoleInfo. The ppf parameter is a
** byref output when chix!=0 and a byref input when chix==0.
*/
static PerStreamTags *
getEmitStreamInfo(unsigned chix, PerStreamTags *ppst,
/* in/out */ FILE **ppf){
PerStreamTags *ppstTry;
FILE *pfEmit;
if( chix > 0 ){
ppstTry = &consoleInfo.pstDesignated[chix];
if( !isValidStreamInfo(ppstTry) ){
ppstTry = &consoleInfo.pstSetup[chix];
pfEmit = ppst->pf;
}else pfEmit = ppstTry->pf;
if( !isValidStreamInfo(ppstTry) ){
pfEmit = (chix > 1)? stderr : stdout;
ppstTry = ppst;
streamOfConsole(pfEmit, ppstTry);
}
*ppf = pfEmit;
}else{
ppstTry = isKnownWritable(*ppf);
if( ppstTry != 0 ) return ppstTry;
streamOfConsole(*ppf, ppst);
return ppst;
}
return ppstTry;
}
SQLITE_INTERNAL_LINKAGE int oPrintfUtf8(const char *zFormat, ...){
va_list ap;
int rv;
FILE *pfOut;
PerStreamTags pst = PST_INITIALIZER; /* for unknown streams */
# if CIO_WIN_WC_XLATE
PerStreamTags *ppst = getEmitStreamInfo(1, &pst, &pfOut);
# else
getEmitStreamInfo(1, &pst, &pfOut);
# endif
assert(zFormat!=0);
va_start(ap, zFormat);
# if CIO_WIN_WC_XLATE
if( pstReachesConsole(ppst) ){
rv = conioVmPrintf(ppst, zFormat, ap);
}else{
# endif
rv = vfprintf(pfOut, zFormat, ap);
# if CIO_WIN_WC_XLATE
}
# endif
va_end(ap);
return rv;
}
SQLITE_INTERNAL_LINKAGE int ePrintfUtf8(const char *zFormat, ...){
va_list ap;
int rv;
FILE *pfErr;
PerStreamTags pst = PST_INITIALIZER; /* for unknown streams */
# if CIO_WIN_WC_XLATE
PerStreamTags *ppst = getEmitStreamInfo(2, &pst, &pfErr);
# else
getEmitStreamInfo(2, &pst, &pfErr);
# endif
assert(zFormat!=0);
va_start(ap, zFormat);
# if CIO_WIN_WC_XLATE
if( pstReachesConsole(ppst) ){
rv = conioVmPrintf(ppst, zFormat, ap);
}else{
# endif
rv = vfprintf(pfErr, zFormat, ap);
# if CIO_WIN_WC_XLATE
}
# endif
va_end(ap);
return rv;
}
SQLITE_INTERNAL_LINKAGE int fPrintfUtf8(FILE *pfO, const char *zFormat, ...){
va_list ap;
int rv;
PerStreamTags pst = PST_INITIALIZER; /* for unknown streams */
# if CIO_WIN_WC_XLATE
PerStreamTags *ppst = getEmitStreamInfo(0, &pst, &pfO);
# else
getEmitStreamInfo(0, &pst, &pfO);
# endif
assert(zFormat!=0);
va_start(ap, zFormat);
# if CIO_WIN_WC_XLATE
if( pstReachesConsole(ppst) ){
maybeSetupAsConsole(ppst, 1);
rv = conioVmPrintf(ppst, zFormat, ap);
if( 0 == isKnownWritable(ppst->pf) ) restoreConsoleArb(ppst);
}else{
# endif
rv = vfprintf(pfO, zFormat, ap);
# if CIO_WIN_WC_XLATE
}
# endif
va_end(ap);
return rv;
}
SQLITE_INTERNAL_LINKAGE int fPutsUtf8(const char *z, FILE *pfO){
PerStreamTags pst = PST_INITIALIZER; /* for unknown streams */
# if CIO_WIN_WC_XLATE
PerStreamTags *ppst = getEmitStreamInfo(0, &pst, &pfO);
# else
getEmitStreamInfo(0, &pst, &pfO);
# endif
assert(z!=0);
# if CIO_WIN_WC_XLATE
if( pstReachesConsole(ppst) ){
int rv;
maybeSetupAsConsole(ppst, 1);
rv = conZstrEmit(ppst, z, (int)strlen(z));
if( 0 == isKnownWritable(ppst->pf) ) restoreConsoleArb(ppst);
return rv;
}else {
# endif
return (fputs(z, pfO)<0)? 0 : (int)strlen(z);
# if CIO_WIN_WC_XLATE
}
# endif
}
SQLITE_INTERNAL_LINKAGE int ePutsUtf8(const char *z){
FILE *pfErr;
PerStreamTags pst = PST_INITIALIZER; /* for unknown streams */
# if CIO_WIN_WC_XLATE
PerStreamTags *ppst = getEmitStreamInfo(2, &pst, &pfErr);
# else
getEmitStreamInfo(2, &pst, &pfErr);
# endif
assert(z!=0);
# if CIO_WIN_WC_XLATE
if( pstReachesConsole(ppst) ) return conZstrEmit(ppst, z, (int)strlen(z));
else {
# endif
return (fputs(z, pfErr)<0)? 0 : (int)strlen(z);
# if CIO_WIN_WC_XLATE
}
# endif
}
SQLITE_INTERNAL_LINKAGE int oPutsUtf8(const char *z){
FILE *pfOut;
PerStreamTags pst = PST_INITIALIZER; /* for unknown streams */
# if CIO_WIN_WC_XLATE
PerStreamTags *ppst = getEmitStreamInfo(1, &pst, &pfOut);
# else
getEmitStreamInfo(1, &pst, &pfOut);
# endif
assert(z!=0);
# if CIO_WIN_WC_XLATE
if( pstReachesConsole(ppst) ) return conZstrEmit(ppst, z, (int)strlen(z));
else {
# endif
return (fputs(z, pfOut)<0)? 0 : (int)strlen(z);
# if CIO_WIN_WC_XLATE
}
# endif
}
#endif /* !defined(SQLITE_CIO_NO_TRANSLATE) */
#if !(defined(SQLITE_CIO_NO_UTF8SCAN) && defined(SQLITE_CIO_NO_TRANSLATE))
/* Skip over as much z[] input char sequence as is valid UTF-8,
** limited per nAccept char's or whole characters and containing
** no char cn such that ((1<<cn) & ccm)!=0. On return, the
** sequence z:return (inclusive:exclusive) is validated UTF-8.
** Limit: nAccept>=0 => char count, nAccept<0 => character
*/
SQLITE_INTERNAL_LINKAGE const char*
zSkipValidUtf8(const char *z, int nAccept, long ccm){
int ng = (nAccept<0)? -nAccept : 0;
const char *pcLimit = (nAccept>=0)? z+nAccept : 0;
assert(z!=0);
while( (pcLimit)? (z<pcLimit) : (ng-- != 0) ){
char c = *z;
if( (c & 0x80) == 0 ){
if( ccm != 0L && c < 0x20 && ((1L<<c) & ccm) != 0 ) return z;
++z; /* ASCII */
}else if( (c & 0xC0) != 0xC0 ) return z; /* not a lead byte */
else{
const char *zt = z+1; /* Got lead byte, look at trail bytes.*/
do{
if( pcLimit && zt >= pcLimit ) return z;
else{
char ct = *zt++;
if( ct==0 || (zt-z)>4 || (ct & 0xC0)!=0x80 ){
/* Trailing bytes are too few, too many, or invalid. */
return z;
}
}
} while( ((c <<= 1) & 0x40) == 0x40 ); /* Eat lead byte's count. */
z = zt;
}
}
return z;
}
#endif /*!(defined(SQLITE_CIO_NO_UTF8SCAN)&&defined(SQLITE_CIO_NO_TRANSLATE))*/
#ifndef SQLITE_CIO_NO_TRANSLATE
# ifdef CONSIO_SPUTB
SQLITE_INTERNAL_LINKAGE int
fPutbUtf8(FILE *pfO, const char *cBuf, int nAccept){
assert(pfO!=0);
# if CIO_WIN_WC_XLATE
PerStreamTags pst = PST_INITIALIZER; /* for unknown streams */
PerStreamTags *ppst = getEmitStreamInfo(0, &pst, &pfO);
if( pstReachesConsole(ppst) ){
int rv;
maybeSetupAsConsole(ppst, 1);
rv = conZstrEmit(ppst, cBuf, nAccept);
if( 0 == isKnownWritable(ppst->pf) ) restoreConsoleArb(ppst);
return rv;
}else {
# endif
return (int)fwrite(cBuf, 1, nAccept, pfO);
# if CIO_WIN_WC_XLATE
}
# endif
}
# endif
SQLITE_INTERNAL_LINKAGE int
oPutbUtf8(const char *cBuf, int nAccept){
FILE *pfOut;
PerStreamTags pst = PST_INITIALIZER; /* for unknown streams */
# if CIO_WIN_WC_XLATE
PerStreamTags *ppst = getEmitStreamInfo(1, &pst, &pfOut);
# else
getEmitStreamInfo(1, &pst, &pfOut);
# endif
# if CIO_WIN_WC_XLATE
if( pstReachesConsole(ppst) ){
return conZstrEmit(ppst, cBuf, nAccept);
}else {
# endif
return (int)fwrite(cBuf, 1, nAccept, pfOut);
# if CIO_WIN_WC_XLATE
}
# endif
}
# ifdef CONSIO_EPUTB
SQLITE_INTERNAL_LINKAGE int
ePutbUtf8(const char *cBuf, int nAccept){
FILE *pfErr;
PerStreamTags pst = PST_INITIALIZER; /* for unknown streams */
PerStreamTags *ppst = getEmitStreamInfo(2, &pst, &pfErr);
# if CIO_WIN_WC_XLATE
if( pstReachesConsole(ppst) ){
return conZstrEmit(ppst, cBuf, nAccept);
}else {
# endif
return (int)fwrite(cBuf, 1, nAccept, pfErr);
# if CIO_WIN_WC_XLATE
}
# endif
}
# endif /* defined(CONSIO_EPUTB) */
SQLITE_INTERNAL_LINKAGE char* fGetsUtf8(char *cBuf, int ncMax, FILE *pfIn){
if( pfIn==0 ) pfIn = stdin;
# if CIO_WIN_WC_XLATE
if( pfIn == consoleInfo.pstSetup[0].pf
&& (consoleInfo.sacSetup & SAC_InConsole)!=0 ){
# if CIO_WIN_WC_XLATE==1
# define SHELL_GULP 150 /* Count of WCHARS to be gulped at a time */
WCHAR wcBuf[SHELL_GULP+1];
int lend = 0, noc = 0;
if( ncMax > 0 ) cBuf[0] = 0;
while( noc < ncMax-8-1 && !lend ){
/* There is room for at least 2 more characters and a 0-terminator. */
int na = (ncMax > SHELL_GULP*4+1 + noc)? SHELL_GULP : (ncMax-1 - noc)/4;
# undef SHELL_GULP
DWORD nbr = 0;
BOOL bRC = ReadConsoleW(consoleInfo.pstSetup[0].hx, wcBuf, na, &nbr, 0);
if( bRC && nbr>0 && (wcBuf[nbr-1]&0xF800)==0xD800 ){
/* Last WHAR read is first of a UTF-16 surrogate pair. Grab its mate. */
DWORD nbrx;
bRC &= ReadConsoleW(consoleInfo.pstSetup[0].hx, wcBuf+nbr, 1, &nbrx, 0);
if( bRC ) nbr += nbrx;
}
if( !bRC || (noc==0 && nbr==0) ) return 0;
if( nbr > 0 ){
int nmb = WideCharToMultiByte(CP_UTF8, 0, wcBuf,nbr,0,0,0,0);
if( nmb != 0 && noc+nmb <= ncMax ){
int iseg = noc;
nmb = WideCharToMultiByte(CP_UTF8, 0, wcBuf,nbr,cBuf+noc,nmb,0,0);
noc += nmb;
/* Fixup line-ends as coded by Windows for CR (or "Enter".)
** This is done without regard for any setMode{Text,Binary}()
** call that might have been done on the interactive input.
*/
if( noc > 0 ){
if( cBuf[noc-1]=='\n' ){
lend = 1;
if( noc > 1 && cBuf[noc-2]=='\r' ) cBuf[--noc-1] = '\n';
}
}
/* Check for ^Z (anywhere in line) too, to act as EOF. */
while( iseg < noc ){
if( cBuf[iseg]=='\x1a' ){
noc = iseg; /* Chop ^Z and anything following. */
lend = 1; /* Counts as end of line too. */
break;
}
++iseg;
}
}else break; /* Drop apparent garbage in. (Could assert.) */
}else break;
}
/* If got nothing, (after ^Z chop), must be at end-of-file. */
if( noc > 0 ){
cBuf[noc] = 0;
return cBuf;
}else return 0;
# endif
}else{
# endif
return fgets(cBuf, ncMax, pfIn);
# if CIO_WIN_WC_XLATE
}
# endif
}
#endif /* !defined(SQLITE_CIO_NO_TRANSLATE) */
#undef SHELL_INVALID_FILE_PTR
+280
View File
@@ -0,0 +1,280 @@
/*
** 2023 November 1
**
** The author disclaims copyright to this source code. In place of
** a legal notice, here is a blessing:
**
** May you do good and not evil.
** May you find forgiveness for yourself and forgive others.
** May you share freely, never taking more than you give.
**
********************************************************************************
** This file exposes various interfaces used for console and other I/O
** by the SQLite project command-line tools. These interfaces are used
** at either source conglomeration time, compilation time, or run time.
** This source provides for either inclusion into conglomerated,
** "single-source" forms or separate compilation then linking.
**
** Platform dependencies are "hidden" here by various stratagems so
** that, provided certain conditions are met, the programs using this
** source or object code compiled from it need no explicit conditional
** compilation in their source for their console and stream I/O.
**
** The symbols and functionality exposed here are not a public API.
** This code may change in tandem with other project code as needed.
**
** When this .h file and its companion .c are directly incorporated into
** a source conglomeration (such as shell.c), the preprocessor symbol
** CIO_WIN_WC_XLATE is defined as 0 or 1, reflecting whether console I/O
** translation for Windows is effected for the build.
*/
#ifndef SQLITE_INTERNAL_LINKAGE
# define SQLITE_INTERNAL_LINKAGE extern /* external to translation unit */
# include <stdio.h>
#else
# define SHELL_NO_SYSINC /* Better yet, modify mkshellc.tcl for this. */
#endif
#ifndef SQLITE3_H
# include "sqlite3.h"
#endif
#ifndef SQLITE_CIO_NO_CLASSIFY
/* Define enum for use with following function. */
typedef enum StreamsAreConsole {
SAC_NoConsole = 0,
SAC_InConsole = 1, SAC_OutConsole = 2, SAC_ErrConsole = 4,
SAC_AnyConsole = 0x7
} StreamsAreConsole;
/*
** Classify the three standard I/O streams according to whether
** they are connected to a console attached to the process.
**
** Returns the bit-wise OR of SAC_{In,Out,Err}Console values,
** or SAC_NoConsole if none of the streams reaches a console.
**
** This function should be called before any I/O is done with
** the given streams. As a side-effect, the given inputs are
** recorded so that later I/O operations on them may be done
** differently than the C library FILE* I/O would be done,
** iff the stream is used for the I/O functions that follow,
** and to support the ones that use an implicit stream.
**
** On some platforms, stream or console mode alteration (aka
** "Setup") may be made which is undone by consoleRestore().
*/
SQLITE_INTERNAL_LINKAGE StreamsAreConsole
consoleClassifySetup( FILE *pfIn, FILE *pfOut, FILE *pfErr );
/* A usual call for convenience: */
#define SQLITE_STD_CONSOLE_INIT() consoleClassifySetup(stdin,stdout,stderr)
/*
** After an initial call to consoleClassifySetup(...), renew
** the same setup it effected. (A call not after is an error.)
** This will restore state altered by consoleRestore();
**
** Applications which run an inferior (child) process which
** inherits the same I/O streams may call this function after
** such a process exits to guard against console mode changes.
*/
SQLITE_INTERNAL_LINKAGE void consoleRenewSetup(void);
/*
** Undo any side-effects left by consoleClassifySetup(...).
**
** This should be called after consoleClassifySetup() and
** before the process terminates normally. It is suitable
** for use with the atexit() C library procedure. After
** this call, no console I/O should be done until one of
** console{Classify or Renew}Setup(...) is called again.
**
** Applications which run an inferior (child) process that
** inherits the same I/O streams might call this procedure
** before so that said process will have a console setup
** however users have configured it or come to expect.
*/
SQLITE_INTERNAL_LINKAGE void SQLITE_CDECL consoleRestore( void );
#else /* defined(SQLITE_CIO_NO_CLASSIFY) */
# define consoleClassifySetup(i,o,e)
# define consoleRenewSetup()
# define consoleRestore()
#endif /* defined(SQLITE_CIO_NO_CLASSIFY) */
#ifndef SQLITE_CIO_NO_REDIRECT
/*
** Set stream to be used for the functions below which write
** to "the designated X stream", where X is Output or Error.
** Returns the previous value.
**
** Alternatively, pass the special value, invalidFileStream,
** to get the designated stream value without setting it.
**
** Before the designated streams are set, they default to
** those passed to consoleClassifySetup(...), and before
** that is called they default to stdout and stderr.
**
** It is error to close a stream so designated, then, without
** designating another, use the corresponding {o,e}Emit(...).
*/
SQLITE_INTERNAL_LINKAGE FILE *invalidFileStream;
SQLITE_INTERNAL_LINKAGE FILE *setOutputStream(FILE *pf);
# ifdef CONSIO_SET_ERROR_STREAM
SQLITE_INTERNAL_LINKAGE FILE *setErrorStream(FILE *pf);
# endif
#else
# define setOutputStream(pf)
# define setErrorStream(pf)
#endif /* !defined(SQLITE_CIO_NO_REDIRECT) */
#ifndef SQLITE_CIO_NO_TRANSLATE
/*
** Emit output like fprintf(). If the output is going to the
** console and translation from UTF-8 is necessary, perform
** the needed translation. Otherwise, write formatted output
** to the provided stream almost as-is, possibly with newline
** translation as specified by set{Binary,Text}Mode().
*/
SQLITE_INTERNAL_LINKAGE int fPrintfUtf8(FILE *pfO, const char *zFormat, ...);
/* Like fPrintfUtf8 except stream is always the designated output. */
SQLITE_INTERNAL_LINKAGE int oPrintfUtf8(const char *zFormat, ...);
/* Like fPrintfUtf8 except stream is always the designated error. */
SQLITE_INTERNAL_LINKAGE int ePrintfUtf8(const char *zFormat, ...);
/*
** Emit output like fputs(). If the output is going to the
** console and translation from UTF-8 is necessary, perform
** the needed translation. Otherwise, write given text to the
** provided stream almost as-is, possibly with newline
** translation as specified by set{Binary,Text}Mode().
*/
SQLITE_INTERNAL_LINKAGE int fPutsUtf8(const char *z, FILE *pfO);
/* Like fPutsUtf8 except stream is always the designated output. */
SQLITE_INTERNAL_LINKAGE int oPutsUtf8(const char *z);
/* Like fPutsUtf8 except stream is always the designated error. */
SQLITE_INTERNAL_LINKAGE int ePutsUtf8(const char *z);
/*
** Emit output like fPutsUtf8(), except that the length of the
** accepted char or character sequence is limited by nAccept.
**
** Returns the number of accepted char values.
*/
#ifdef CONSIO_SPUTB
SQLITE_INTERNAL_LINKAGE int
fPutbUtf8(FILE *pfOut, const char *cBuf, int nAccept);
/* Like fPutbUtf8 except stream is always the designated output. */
#endif
SQLITE_INTERNAL_LINKAGE int
oPutbUtf8(const char *cBuf, int nAccept);
/* Like fPutbUtf8 except stream is always the designated error. */
#ifdef CONSIO_EPUTB
SQLITE_INTERNAL_LINKAGE int
ePutbUtf8(const char *cBuf, int nAccept);
#endif
/*
** Collect input like fgets(...) with special provisions for input
** from the console on platforms that require same. Defers to the
** C library fgets() when input is not from the console. Newline
** translation may be done as set by set{Binary,Text}Mode(). As a
** convenience, pfIn==NULL is treated as stdin.
*/
SQLITE_INTERNAL_LINKAGE char* fGetsUtf8(char *cBuf, int ncMax, FILE *pfIn);
/* Like fGetsUtf8 except stream is always the designated input. */
/* SQLITE_INTERNAL_LINKAGE char* iGetsUtf8(char *cBuf, int ncMax); */
#endif /* !defined(SQLITE_CIO_NO_TRANSLATE) */
#ifndef SQLITE_CIO_NO_SETMODE
/*
** Set given stream for binary mode, where newline translation is
** not done, or for text mode where, for some platforms, newlines
** are translated to the platform's conventional char sequence.
** If bFlush true, flush the stream.
**
** An additional side-effect is that if the stream is one passed
** to consoleClassifySetup() as an output, it is flushed first.
**
** Note that binary/text mode has no effect on console I/O
** translation. On all platforms, newline to the console starts
** a new line and CR,LF chars from the console become a newline.
*/
SQLITE_INTERNAL_LINKAGE void setBinaryMode(FILE *, short bFlush);
SQLITE_INTERNAL_LINKAGE void setTextMode(FILE *, short bFlush);
#endif
#ifdef SQLITE_CIO_PROMPTED_IN
typedef struct Prompts {
int numPrompts;
const char **azPrompts;
} Prompts;
/*
** Macros for use of a line editor.
**
** The following macros define operations involving use of a
** line-editing library or simple console interaction.
** A "T" argument is a text (char *) buffer or filename.
** A "N" argument is an integer.
**
** SHELL_ADD_HISTORY(T) // Record text as line(s) of history.
** SHELL_READ_HISTORY(T) // Read history from file named by T.
** SHELL_WRITE_HISTORY(T) // Write history to file named by T.
** SHELL_STIFLE_HISTORY(N) // Limit history to N entries.
**
** A console program which does interactive console input is
** expected to call:
** SHELL_READ_HISTORY(T) before collecting such input;
** SHELL_ADD_HISTORY(T) as record-worthy input is taken;
** SHELL_STIFLE_HISTORY(N) after console input ceases; then
** SHELL_WRITE_HISTORY(T) before the program exits.
*/
/*
** Retrieve a single line of input text from an input stream.
**
** If pfIn is the input stream passed to consoleClassifySetup(),
** and azPrompt is not NULL, then a prompt is issued before the
** line is collected, as selected by the isContinuation flag.
** Array azPrompt[{0,1}] holds the {main,continuation} prompt.
**
** If zBufPrior is not NULL then it is a buffer from a prior
** call to this routine that can be reused, or will be freed.
**
** The result is stored in space obtained from malloc() and
** must either be freed by the caller or else passed back to
** this function as zBufPrior for reuse.
**
** This function may call upon services of a line-editing
** library to interactively collect line edited input.
*/
SQLITE_INTERNAL_LINKAGE char *
shellGetLine(FILE *pfIn, char *zBufPrior, int nLen,
short isContinuation, Prompts azPrompt);
#endif /* defined(SQLITE_CIO_PROMPTED_IN) */
/*
** TBD: Define an interface for application(s) to generate
** completion candidates for use by the line-editor.
**
** This may be premature; the CLI is the only application
** that does this. Yet, getting line-editing melded into
** console I/O is desirable because a line-editing library
** may have to establish console operating mode, possibly
** in a way that interferes with the above functionality.
*/
#if !(defined(SQLITE_CIO_NO_UTF8SCAN)&&defined(SQLITE_CIO_NO_TRANSLATE))
/* Skip over as much z[] input char sequence as is valid UTF-8,
** limited per nAccept char's or whole characters and containing
** no char cn such that ((1<<cn) & ccm)!=0. On return, the
** sequence z:return (inclusive:exclusive) is validated UTF-8.
** Limit: nAccept>=0 => char count, nAccept<0 => character
*/
SQLITE_INTERNAL_LINKAGE const char*
zSkipValidUtf8(const char *z, int nAccept, long ccm);
#endif
+19
View File
@@ -464,4 +464,23 @@ do_execsql_test 5.3 {
t2 t2_idx_0001295b {100 20 5}
}
if 0 {
do_test expert1-6.0 {
catchcmd :memory: {
.expert
select base64('');
.expert
select name from pragma_collation_list order by name collate uint;
}
} {0 {(no new indexes)
SCAN CONSTANT ROW
(no new indexes)
SCAN pragma_collation_list VIRTUAL TABLE INDEX 0:
USE TEMP B-TREE FOR ORDER BY
}}
}
finish_test
+103 -2
View File
@@ -32,7 +32,7 @@
#endif /* !defined(SQLITE_AMALGAMATION) */
#ifndef SQLITE_OMIT_VIRTUALTABLE
#ifndef SQLITE_OMIT_VIRTUALTABLE
typedef sqlite3_int64 i64;
typedef sqlite3_uint64 u64;
@@ -662,6 +662,7 @@ static int idxRegisterVtab(sqlite3expert *p){
0, /* xRelease */
0, /* xRollbackTo */
0, /* xShadowName */
0, /* xIntegrity */
};
return sqlite3_create_module(p->dbv, "expert", &expertModule, (void*)p);
@@ -1818,6 +1819,88 @@ static int idxPopulateStat1(sqlite3expert *p, char **pzErr){
return rc;
}
/*
** Define and possibly pretend to use a useless collation sequence.
** This pretense allows expert to accept SQL using custom collations.
*/
int dummyCompare(void *up1, int up2, const void *up3, int up4, const void *up5){
(void)up1;
(void)up2;
(void)up3;
(void)up4;
(void)up5;
assert(0); /* VDBE should never be run. */
return 0;
}
/* And a callback to register above upon actual need */
void useDummyCS(void *up1, sqlite3 *db, int etr, const char *zName){
(void)up1;
sqlite3_create_collation_v2(db, zName, etr, 0, dummyCompare, 0);
}
#if !defined(SQLITE_OMIT_SCHEMA_PRAGMAS) \
&& !defined(SQLITE_OMIT_INTROSPECTION_PRAGMAS)
/*
** dummy functions for no-op implementation of UDFs during expert's work
*/
void dummyUDF(sqlite3_context *up1, int up2, sqlite3_value **up3){
(void)up1;
(void)up2;
(void)up3;
assert(0); /* VDBE should never be run. */
}
void dummyUDFvalue(sqlite3_context *up1){
(void)up1;
assert(0); /* VDBE should never be run. */
}
/*
** Register UDFs from user database with another.
*/
int registerUDFs(sqlite3 *dbSrc, sqlite3 *dbDst){
sqlite3_stmt *pStmt;
int rc = sqlite3_prepare_v2(dbSrc,
"SELECT name,type,enc,narg,flags "
"FROM pragma_function_list() "
"WHERE builtin==0", -1, &pStmt, 0);
if( rc==SQLITE_OK ){
while( SQLITE_ROW==(rc = sqlite3_step(pStmt)) ){
int nargs = sqlite3_column_int(pStmt,3);
int flags = sqlite3_column_int(pStmt,4);
const char *name = (char*)sqlite3_column_text(pStmt,0);
const char *type = (char*)sqlite3_column_text(pStmt,1);
const char *enc = (char*)sqlite3_column_text(pStmt,2);
if( name==0 || type==0 || enc==0 ){
/* no-op. Only happens on OOM */
}else{
int ienc = SQLITE_UTF8;
int rcf = SQLITE_ERROR;
if( strcmp(enc,"utf16le")==0 ) ienc = SQLITE_UTF16LE;
else if( strcmp(enc,"utf16be")==0 ) ienc = SQLITE_UTF16BE;
ienc |= (flags & (SQLITE_DETERMINISTIC|SQLITE_DIRECTONLY));
if( strcmp(type,"w")==0 ){
rcf = sqlite3_create_window_function(dbDst,name,nargs,ienc,0,
dummyUDF,dummyUDFvalue,0,0,0);
}else if( strcmp(type,"a")==0 ){
rcf = sqlite3_create_function(dbDst,name,nargs,ienc,0,
0,dummyUDF,dummyUDFvalue);
}else if( strcmp(type,"s")==0 ){
rcf = sqlite3_create_function(dbDst,name,nargs,ienc,0,
dummyUDF,0,0);
}
if( rcf!=SQLITE_OK ){
rc = rcf;
break;
}
}
}
sqlite3_finalize(pStmt);
if( rc==SQLITE_DONE ) rc = SQLITE_OK;
}
return rc;
}
#endif
/*
** Allocate a new sqlite3expert object.
*/
@@ -1844,7 +1927,21 @@ sqlite3expert *sqlite3_expert_new(sqlite3 *db, char **pzErrmsg){
sqlite3_db_config(pNew->dbm, SQLITE_DBCONFIG_TRIGGER_EQP, 1, (int*)0);
}
}
/* Allow custom collations to be dealt with through prepare. */
if( rc==SQLITE_OK ) rc = sqlite3_collation_needed(pNew->dbm,0,useDummyCS);
if( rc==SQLITE_OK ) rc = sqlite3_collation_needed(pNew->dbv,0,useDummyCS);
#if !defined(SQLITE_OMIT_SCHEMA_PRAGMAS) \
&& !defined(SQLITE_OMIT_INTROSPECTION_PRAGMAS)
/* Register UDFs from database [db] with [dbm] and [dbv]. */
if( rc==SQLITE_OK ){
rc = registerUDFs(pNew->db, pNew->dbm);
}
if( rc==SQLITE_OK ){
rc = registerUDFs(pNew->db, pNew->dbv);
}
#endif
/* Copy the entire schema of database [db] into [dbm]. */
if( rc==SQLITE_OK ){
@@ -1920,6 +2017,10 @@ int sqlite3_expert_sql(
while( rc==SQLITE_OK && zStmt && zStmt[0] ){
sqlite3_stmt *pStmt = 0;
/* Ensure that the provided statement compiles against user's DB. */
rc = idxPrepareStmt(p->db, &pStmt, pzErr, zStmt);
if( rc!=SQLITE_OK ) break;
sqlite3_finalize(pStmt);
rc = sqlite3_prepare_v2(p->dbv, zStmt, -1, &pStmt, &zStmt);
if( rc==SQLITE_OK ){
if( pStmt ){
+81 -17
View File
@@ -640,6 +640,7 @@ static void fts3DeclareVtab(int *pRc, Fts3Table *p){
zLanguageid = (p->zLanguageid ? p->zLanguageid : "__langid");
sqlite3_vtab_config(p->db, SQLITE_VTAB_CONSTRAINT_SUPPORT, 1);
sqlite3_vtab_config(p->db, SQLITE_VTAB_INNOCUOUS);
/* Create a list of user columns for the virtual table */
zCols = sqlite3_mprintf("%Q, ", p->azColumn[0]);
@@ -3889,6 +3890,8 @@ static int fts3RenameMethod(
rc = sqlite3Fts3PendingTermsFlush(p);
}
p->bIgnoreSavepoint = 1;
if( p->zContentTbl==0 ){
fts3DbExec(&rc, db,
"ALTER TABLE %Q.'%q_content' RENAME TO '%q_content';",
@@ -3916,6 +3919,8 @@ static int fts3RenameMethod(
"ALTER TABLE %Q.'%q_segdir' RENAME TO '%q_segdir';",
p->zDb, p->zName, zName
);
p->bIgnoreSavepoint = 0;
return rc;
}
@@ -3926,12 +3931,28 @@ static int fts3RenameMethod(
*/
static int fts3SavepointMethod(sqlite3_vtab *pVtab, int iSavepoint){
int rc = SQLITE_OK;
UNUSED_PARAMETER(iSavepoint);
assert( ((Fts3Table *)pVtab)->inTransaction );
assert( ((Fts3Table *)pVtab)->mxSavepoint <= iSavepoint );
TESTONLY( ((Fts3Table *)pVtab)->mxSavepoint = iSavepoint );
if( ((Fts3Table *)pVtab)->bIgnoreSavepoint==0 ){
rc = fts3SyncMethod(pVtab);
Fts3Table *pTab = (Fts3Table*)pVtab;
assert( pTab->inTransaction );
assert( pTab->mxSavepoint<=iSavepoint );
TESTONLY( pTab->mxSavepoint = iSavepoint );
if( pTab->bIgnoreSavepoint==0 ){
if( fts3HashCount(&pTab->aIndex[0].hPending)>0 ){
char *zSql = sqlite3_mprintf("INSERT INTO %Q.%Q(%Q) VALUES('flush')",
pTab->zDb, pTab->zName, pTab->zName
);
if( zSql ){
pTab->bIgnoreSavepoint = 1;
rc = sqlite3_exec(pTab->db, zSql, 0, 0, 0);
pTab->bIgnoreSavepoint = 0;
sqlite3_free(zSql);
}else{
rc = SQLITE_NOMEM;
}
}
if( rc==SQLITE_OK ){
pTab->iSavepoint = iSavepoint+1;
}
}
return rc;
}
@@ -3942,12 +3963,11 @@ static int fts3SavepointMethod(sqlite3_vtab *pVtab, int iSavepoint){
** This is a no-op.
*/
static int fts3ReleaseMethod(sqlite3_vtab *pVtab, int iSavepoint){
TESTONLY( Fts3Table *p = (Fts3Table*)pVtab );
UNUSED_PARAMETER(iSavepoint);
UNUSED_PARAMETER(pVtab);
assert( p->inTransaction );
assert( p->mxSavepoint >= iSavepoint );
TESTONLY( p->mxSavepoint = iSavepoint-1 );
Fts3Table *pTab = (Fts3Table*)pVtab;
assert( pTab->inTransaction );
assert( pTab->mxSavepoint >= iSavepoint );
TESTONLY( pTab->mxSavepoint = iSavepoint-1 );
pTab->iSavepoint = iSavepoint;
return SQLITE_OK;
}
@@ -3957,11 +3977,13 @@ static int fts3ReleaseMethod(sqlite3_vtab *pVtab, int iSavepoint){
** Discard the contents of the pending terms table.
*/
static int fts3RollbackToMethod(sqlite3_vtab *pVtab, int iSavepoint){
Fts3Table *p = (Fts3Table*)pVtab;
Fts3Table *pTab = (Fts3Table*)pVtab;
UNUSED_PARAMETER(iSavepoint);
assert( p->inTransaction );
TESTONLY( p->mxSavepoint = iSavepoint );
sqlite3Fts3PendingTermsClear(p);
assert( pTab->inTransaction );
TESTONLY( pTab->mxSavepoint = iSavepoint );
if( (iSavepoint+1)<=pTab->iSavepoint ){
sqlite3Fts3PendingTermsClear(pTab);
}
return SQLITE_OK;
}
@@ -3980,8 +4002,49 @@ static int fts3ShadowName(const char *zName){
return 0;
}
/*
** Implementation of the xIntegrity() method on the FTS3/FTS4 virtual
** table.
*/
static int fts3Integrity(
sqlite3_vtab *pVtab, /* The virtual table to be checked */
const char *zSchema, /* Name of schema in which pVtab lives */
const char *zTabname, /* Name of the pVTab table */
int isQuick, /* True if this is a quick_check */
char **pzErr /* Write error message here */
){
Fts3Table *p = (Fts3Table*)pVtab;
char *zSql;
int rc;
char *zErr = 0;
assert( pzErr!=0 );
assert( *pzErr==0 );
UNUSED_PARAMETER(isQuick);
zSql = sqlite3_mprintf(
"INSERT INTO \"%w\".\"%w\"(\"%w\") VALUES('integrity-check');",
zSchema, zTabname, zTabname);
if( zSql==0 ){
return SQLITE_NOMEM;
}
rc = sqlite3_exec(p->db, zSql, 0, 0, &zErr);
sqlite3_free(zSql);
if( (rc&0xff)==SQLITE_CORRUPT ){
*pzErr = sqlite3_mprintf("malformed inverted index for FTS%d table %s.%s",
p->bFts4 ? 4 : 3, zSchema, zTabname);
}else if( rc!=SQLITE_OK ){
*pzErr = sqlite3_mprintf("unable to validate the inverted index for"
" FTS%d table %s.%s: %s",
p->bFts4 ? 4 : 3, zSchema, zTabname, zErr);
}
sqlite3_free(zErr);
return SQLITE_OK;
}
static const sqlite3_module fts3Module = {
/* iVersion */ 3,
/* iVersion */ 4,
/* xCreate */ fts3CreateMethod,
/* xConnect */ fts3ConnectMethod,
/* xBestIndex */ fts3BestIndexMethod,
@@ -4005,6 +4068,7 @@ static const sqlite3_module fts3Module = {
/* xRelease */ fts3ReleaseMethod,
/* xRollbackTo */ fts3RollbackToMethod,
/* xShadowName */ fts3ShadowName,
/* xIntegrity */ fts3Integrity,
};
/*
+1
View File
@@ -265,6 +265,7 @@ struct Fts3Table {
int nPgsz; /* Page size for host database */
char *zSegmentsTbl; /* Name of %_segments table */
sqlite3_blob *pSegments; /* Blob handle open on %_segments table */
int iSavepoint;
/*
** The following array of hash tables is used to buffer pending index
+2 -1
View File
@@ -545,7 +545,8 @@ int sqlite3Fts3InitAux(sqlite3 *db){
0, /* xSavepoint */
0, /* xRelease */
0, /* xRollbackTo */
0 /* xShadowName */
0, /* xShadowName */
0 /* xIntegrity */
};
int rc; /* Return code */
+2 -1
View File
@@ -362,7 +362,8 @@ int sqlite3Fts3InitTerm(sqlite3 *db){
0, /* xSavepoint */
0, /* xRelease */
0, /* xRollbackTo */
0 /* xShadowName */
0, /* xShadowName */
0 /* xIntegrity */
};
int rc; /* Return code */
+2 -1
View File
@@ -445,7 +445,8 @@ int sqlite3Fts3InitTok(sqlite3 *db, Fts3Hash *pHash, void(*xDestroy)(void*)){
0, /* xSavepoint */
0, /* xRelease */
0, /* xRollbackTo */
0 /* xShadowName */
0, /* xShadowName */
0 /* xIntegrity */
};
int rc; /* Return code */
+20 -8
View File
@@ -3325,7 +3325,6 @@ int sqlite3Fts3PendingTermsFlush(Fts3Table *p){
rc = fts3SegmentMerge(p, p->iPrevLangid, i, FTS3_SEGCURSOR_PENDING);
if( rc==SQLITE_DONE ) rc = SQLITE_OK;
}
sqlite3Fts3PendingTermsClear(p);
/* Determine the auto-incr-merge setting if unknown. If enabled,
** estimate the number of leaf blocks of content to be written
@@ -3347,6 +3346,10 @@ int sqlite3Fts3PendingTermsFlush(Fts3Table *p){
rc = sqlite3_reset(pStmt);
}
}
if( rc==SQLITE_OK ){
sqlite3Fts3PendingTermsClear(p);
}
return rc;
}
@@ -3978,6 +3981,8 @@ static int fts3AppendToNode(
blobGrowBuffer(pPrev, nTerm, &rc);
if( rc!=SQLITE_OK ) return rc;
assert( pPrev!=0 );
assert( pPrev->a!=0 );
nPrefix = fts3PrefixCompress(pPrev->a, pPrev->n, zTerm, nTerm);
nSuffix = nTerm - nPrefix;
@@ -4034,9 +4039,13 @@ static int fts3IncrmergeAppend(
nSpace += sqlite3Fts3VarintLen(nDoclist) + nDoclist;
/* If the current block is not empty, and if adding this term/doclist
** to the current block would make it larger than Fts3Table.nNodeSize
** bytes, write this block out to the database. */
if( pLeaf->block.n>0 && (pLeaf->block.n + nSpace)>p->nNodeSize ){
** to the current block would make it larger than Fts3Table.nNodeSize bytes,
** and if there is still room for another leaf page, write this block out to
** the database. */
if( pLeaf->block.n>0
&& (pLeaf->block.n + nSpace)>p->nNodeSize
&& pLeaf->iBlock < (pWriter->iStart + pWriter->nLeafEst)
){
rc = fts3WriteSegment(p, pLeaf->iBlock, pLeaf->block.a, pLeaf->block.n);
pWriter->nWork++;
@@ -4368,7 +4377,7 @@ static int fts3IncrmergeLoad(
rc = sqlite3Fts3ReadBlock(p, reader.iChild, &aBlock, &nBlock,0);
blobGrowBuffer(&pNode->block,
MAX(nBlock, p->nNodeSize)+FTS3_NODE_PADDING, &rc
);
);
if( rc==SQLITE_OK ){
memcpy(pNode->block.a, aBlock, nBlock);
pNode->block.n = nBlock;
@@ -5218,7 +5227,7 @@ static u64 fts3ChecksumIndex(
int rc;
u64 cksum = 0;
assert( *pRc==SQLITE_OK );
if( *pRc ) return 0;
memset(&filter, 0, sizeof(filter));
memset(&csr, 0, sizeof(csr));
@@ -5433,8 +5442,11 @@ static int fts3SpecialInsert(Fts3Table *p, sqlite3_value *pVal){
rc = fts3DoIncrmerge(p, &zVal[6]);
}else if( nVal>10 && 0==sqlite3_strnicmp(zVal, "automerge=", 10) ){
rc = fts3DoAutoincrmerge(p, &zVal[10]);
}else if( nVal==5 && 0==sqlite3_strnicmp(zVal, "flush", 5) ){
rc = sqlite3Fts3PendingTermsFlush(p);
}
#if defined(SQLITE_DEBUG) || defined(SQLITE_TEST)
}else{
else{
int v;
if( nVal>9 && 0==sqlite3_strnicmp(zVal, "nodesize=", 9) ){
v = atoi(&zVal[9]);
@@ -5452,8 +5464,8 @@ static int fts3SpecialInsert(Fts3Table *p, sqlite3_value *pVal){
if( v>=4 && v<=FTS3_MERGE_COUNT && (v&1)==0 ) p->nMergeCount = v;
rc = SQLITE_OK;
}
#endif
}
#endif
return rc;
}
+6 -6
View File
@@ -263,7 +263,7 @@ struct Fts5PhraseIter {
** See xPhraseFirstColumn above.
*/
struct Fts5ExtensionApi {
int iVersion; /* Currently always set to 3 */
int iVersion; /* Currently always set to 2 */
void *(*xUserData)(Fts5Context*);
@@ -492,8 +492,8 @@ struct Fts5ExtensionApi {
** as separate queries of the FTS index are required for each synonym.
**
** When using methods (2) or (3), it is important that the tokenizer only
** provide synonyms when tokenizing document text (method (2)) or query
** text (method (3)), not both. Doing so will not cause any errors, but is
** provide synonyms when tokenizing document text (method (3)) or query
** text (method (2)), not both. Doing so will not cause any errors, but is
** inefficient.
*/
typedef struct Fts5Tokenizer Fts5Tokenizer;
@@ -541,7 +541,7 @@ struct fts5_api {
int (*xCreateTokenizer)(
fts5_api *pApi,
const char *zName,
void *pContext,
void *pUserData,
fts5_tokenizer *pTokenizer,
void (*xDestroy)(void*)
);
@@ -550,7 +550,7 @@ struct fts5_api {
int (*xFindTokenizer)(
fts5_api *pApi,
const char *zName,
void **ppContext,
void **ppUserData,
fts5_tokenizer *pTokenizer
);
@@ -558,7 +558,7 @@ struct fts5_api {
int (*xCreateFunction)(
fts5_api *pApi,
const char *zName,
void *pContext,
void *pUserData,
fts5_extension_function xFunction,
void (*xDestroy)(void*)
);
+15
View File
@@ -154,6 +154,10 @@ typedef struct Fts5Config Fts5Config;
** attempt to merge together. A value of 1 sets the object to use the
** compile time default. Zero disables auto-merge altogether.
**
** bContentlessDelete:
** True if the contentless_delete option was present in the CREATE
** VIRTUAL TABLE statement.
**
** zContent:
**
** zContentRowid:
@@ -188,6 +192,7 @@ struct Fts5Config {
int nPrefix; /* Number of prefix indexes */
int *aPrefix; /* Sizes in bytes of nPrefix prefix indexes */
int eContent; /* An FTS5_CONTENT value */
int bContentlessDelete; /* "contentless_delete=" option (dflt==0) */
char *zContent; /* content table */
char *zContentRowid; /* "content_rowid=" option value */
int bColumnsize; /* "columnsize=" option value (dflt==1) */
@@ -209,6 +214,7 @@ struct Fts5Config {
char *zRank; /* Name of rank function */
char *zRankArgs; /* Arguments to rank function */
int bSecureDelete; /* 'secure-delete' */
int nDeleteMerge; /* 'deletemerge' */
/* If non-NULL, points to sqlite3_vtab.base.zErrmsg. Often NULL. */
char **pzErrmsg;
@@ -531,6 +537,9 @@ int sqlite3Fts5IndexReset(Fts5Index *p);
int sqlite3Fts5IndexLoadConfig(Fts5Index *p);
int sqlite3Fts5IndexGetOrigin(Fts5Index *p, i64 *piOrigin);
int sqlite3Fts5IndexContentlessDelete(Fts5Index *p, i64 iOrigin, i64 iRowid);
/*
** End of interface to code in fts5_index.c.
**************************************************************************/
@@ -615,6 +624,11 @@ int sqlite3Fts5HashWrite(
*/
void sqlite3Fts5HashClear(Fts5Hash*);
/*
** Return true if the hash is empty, false otherwise.
*/
int sqlite3Fts5HashIsEmpty(Fts5Hash*);
int sqlite3Fts5HashQuery(
Fts5Hash*, /* Hash table to query */
int nPre,
@@ -636,6 +650,7 @@ void sqlite3Fts5HashScanEntry(Fts5Hash *,
);
/*
** End of interface to code in fts5_hash.c.
**************************************************************************/
+47 -12
View File
@@ -110,15 +110,19 @@ static int fts5CInstIterInit(
*/
typedef struct HighlightContext HighlightContext;
struct HighlightContext {
CInstIter iter; /* Coalesced Instance Iterator */
int iPos; /* Current token offset in zIn[] */
/* Constant parameters to fts5HighlightCb() */
int iRangeStart; /* First token to include */
int iRangeEnd; /* If non-zero, last token to include */
const char *zOpen; /* Opening highlight */
const char *zClose; /* Closing highlight */
const char *zIn; /* Input text */
int nIn; /* Size of input text in bytes */
int iOff; /* Current offset within zIn[] */
/* Variables modified by fts5HighlightCb() */
CInstIter iter; /* Coalesced Instance Iterator */
int iPos; /* Current token offset in zIn[] */
int iOff; /* Have copied up to this offset in zIn[] */
int bOpen; /* True if highlight is open */
char *zOut; /* Output value */
};
@@ -151,8 +155,8 @@ static int fts5HighlightCb(
int tflags, /* Mask of FTS5_TOKEN_* flags */
const char *pToken, /* Buffer containing token */
int nToken, /* Size of token in bytes */
int iStartOff, /* Start offset of token */
int iEndOff /* End offset of token */
int iStartOff, /* Start byte offset of token */
int iEndOff /* End byte offset of token */
){
HighlightContext *p = (HighlightContext*)pContext;
int rc = SQLITE_OK;
@@ -168,30 +172,55 @@ static int fts5HighlightCb(
if( p->iRangeStart && iPos==p->iRangeStart ) p->iOff = iStartOff;
}
if( iPos==p->iter.iStart ){
/* If the parenthesis is open, and this token is not part of the current
** phrase, and the starting byte offset of this token is past the point
** that has currently been copied into the output buffer, close the
** parenthesis. */
if( p->bOpen
&& (iPos<=p->iter.iStart || p->iter.iStart<0)
&& iStartOff>p->iOff
){
fts5HighlightAppend(&rc, p, p->zClose, -1);
p->bOpen = 0;
}
/* If this is the start of a new phrase, and the highlight is not open:
**
** * copy text from the input up to the start of the phrase, and
** * open the highlight.
*/
if( iPos==p->iter.iStart && p->bOpen==0 ){
fts5HighlightAppend(&rc, p, &p->zIn[p->iOff], iStartOff - p->iOff);
fts5HighlightAppend(&rc, p, p->zOpen, -1);
p->iOff = iStartOff;
p->bOpen = 1;
}
if( iPos==p->iter.iEnd ){
if( p->iRangeEnd>=0 && p->iter.iStart<p->iRangeStart ){
if( p->bOpen==0 ){
assert( p->iRangeEnd>=0 );
fts5HighlightAppend(&rc, p, p->zOpen, -1);
p->bOpen = 1;
}
fts5HighlightAppend(&rc, p, &p->zIn[p->iOff], iEndOff - p->iOff);
fts5HighlightAppend(&rc, p, p->zClose, -1);
p->iOff = iEndOff;
if( rc==SQLITE_OK ){
rc = fts5CInstIterNext(&p->iter);
}
}
if( p->iRangeEnd>=0 && iPos==p->iRangeEnd ){
if( iPos==p->iRangeEnd ){
if( p->bOpen ){
if( p->iter.iStart>=0 && iPos>=p->iter.iStart ){
fts5HighlightAppend(&rc, p, &p->zIn[p->iOff], iEndOff - p->iOff);
p->iOff = iEndOff;
}
fts5HighlightAppend(&rc, p, p->zClose, -1);
p->bOpen = 0;
}
fts5HighlightAppend(&rc, p, &p->zIn[p->iOff], iEndOff - p->iOff);
p->iOff = iEndOff;
if( iPos>=p->iter.iStart && iPos<p->iter.iEnd ){
fts5HighlightAppend(&rc, p, p->zClose, -1);
}
}
return rc;
@@ -232,6 +261,9 @@ static void fts5HighlightFunction(
if( rc==SQLITE_OK ){
rc = pApi->xTokenize(pFts, ctx.zIn, ctx.nIn, (void*)&ctx,fts5HighlightCb);
}
if( ctx.bOpen ){
fts5HighlightAppend(&rc, &ctx, ctx.zClose, -1);
}
fts5HighlightAppend(&rc, &ctx, &ctx.zIn[ctx.iOff], ctx.nIn - ctx.iOff);
if( rc==SQLITE_OK ){
@@ -510,6 +542,9 @@ static void fts5SnippetFunction(
if( rc==SQLITE_OK ){
rc = pApi->xTokenize(pFts, ctx.zIn, ctx.nIn, (void*)&ctx,fts5HighlightCb);
}
if( ctx.bOpen ){
fts5HighlightAppend(&rc, &ctx, ctx.zClose, -1);
}
if( ctx.iRangeEnd>=(nColSize-1) ){
fts5HighlightAppend(&rc, &ctx, &ctx.zIn[ctx.iOff], ctx.nIn - ctx.iOff);
}else{
+47
View File
@@ -22,6 +22,8 @@
#define FTS5_DEFAULT_CRISISMERGE 16
#define FTS5_DEFAULT_HASHSIZE (1024*1024)
#define FTS5_DEFAULT_DELETE_AUTOMERGE 10 /* default 10% */
/* Maximum allowed page size */
#define FTS5_MAX_PAGE_SIZE (64*1024)
@@ -352,6 +354,16 @@ static int fts5ConfigParseSpecial(
return rc;
}
if( sqlite3_strnicmp("contentless_delete", zCmd, nCmd)==0 ){
if( (zArg[0]!='0' && zArg[0]!='1') || zArg[1]!='\0' ){
*pzErr = sqlite3_mprintf("malformed contentless_delete=... directive");
rc = SQLITE_ERROR;
}else{
pConfig->bContentlessDelete = (zArg[0]=='1');
}
return rc;
}
if( sqlite3_strnicmp("content_rowid", zCmd, nCmd)==0 ){
if( pConfig->zContentRowid ){
*pzErr = sqlite3_mprintf("multiple content_rowid=... directives");
@@ -596,6 +608,28 @@ int sqlite3Fts5ConfigParse(
sqlite3_free(zTwo);
}
/* We only allow contentless_delete=1 if the table is indeed contentless. */
if( rc==SQLITE_OK
&& pRet->bContentlessDelete
&& pRet->eContent!=FTS5_CONTENT_NONE
){
*pzErr = sqlite3_mprintf(
"contentless_delete=1 requires a contentless table"
);
rc = SQLITE_ERROR;
}
/* We only allow contentless_delete=1 if columnsize=0 is not present.
**
** This restriction may be removed at some point.
*/
if( rc==SQLITE_OK && pRet->bContentlessDelete && pRet->bColumnsize==0 ){
*pzErr = sqlite3_mprintf(
"contentless_delete=1 is incompatible with columnsize=0"
);
rc = SQLITE_ERROR;
}
/* If a tokenizer= option was successfully parsed, the tokenizer has
** already been allocated. Otherwise, allocate an instance of the default
** tokenizer (unicode61) now. */
@@ -890,6 +924,18 @@ int sqlite3Fts5ConfigSetValue(
}
}
else if( 0==sqlite3_stricmp(zKey, "deletemerge") ){
int nVal = -1;
if( SQLITE_INTEGER==sqlite3_value_numeric_type(pVal) ){
nVal = sqlite3_value_int(pVal);
}else{
*pbBadkey = 1;
}
if( nVal<0 ) nVal = FTS5_DEFAULT_DELETE_AUTOMERGE;
if( nVal>100 ) nVal = 0;
pConfig->nDeleteMerge = nVal;
}
else if( 0==sqlite3_stricmp(zKey, "rank") ){
const char *zIn = (const char*)sqlite3_value_text(pVal);
char *zRank;
@@ -938,6 +984,7 @@ int sqlite3Fts5ConfigLoad(Fts5Config *pConfig, int iCookie){
pConfig->nUsermerge = FTS5_DEFAULT_USERMERGE;
pConfig->nCrisisMerge = FTS5_DEFAULT_CRISISMERGE;
pConfig->nHashSize = FTS5_DEFAULT_HASHSIZE;
pConfig->nDeleteMerge = FTS5_DEFAULT_DELETE_AUTOMERGE;
zSql = sqlite3Fts5Mprintf(&rc, zSelect, pConfig->zDb, pConfig->zName);
if( zSql ){
+5 -3
View File
@@ -2477,7 +2477,7 @@ Fts5ExprNode *sqlite3Fts5ParseImplicitAnd(
return pRet;
}
#ifdef SQLITE_TEST
#if defined(SQLITE_TEST) || defined(SQLITE_FTS5_DEBUG)
static char *fts5ExprTermPrint(Fts5ExprTerm *pTerm){
sqlite3_int64 nByte = 0;
Fts5ExprTerm *p;
@@ -2583,6 +2583,8 @@ static char *fts5ExprPrintTcl(
if( zRet==0 ) return 0;
}
}else if( pExpr->eType==0 ){
zRet = sqlite3_mprintf("{}");
}else{
char const *zOp = 0;
int i;
@@ -2844,14 +2846,14 @@ static void fts5ExprFold(
sqlite3_result_int(pCtx, sqlite3Fts5UnicodeFold(iCode, bRemoveDiacritics));
}
}
#endif /* ifdef SQLITE_TEST */
#endif /* if SQLITE_TEST || SQLITE_FTS5_DEBUG */
/*
** This is called during initialization to register the fts5_expr() scalar
** UDF with the SQLite handle passed as the only argument.
*/
int sqlite3Fts5ExprInit(Fts5Global *pGlobal, sqlite3 *db){
#ifdef SQLITE_TEST
#if defined(SQLITE_TEST) || defined(SQLITE_FTS5_DEBUG)
struct Fts5ExprFunc {
const char *z;
void (*x)(sqlite3_context*,int,sqlite3_value**);
+24 -5
View File
@@ -432,10 +432,8 @@ static Fts5HashEntry *fts5HashEntryMerge(
}
/*
** Extract all tokens from hash table iHash and link them into a list
** in sorted order. The hash table is cleared before returning. It is
** the responsibility of the caller to free the elements of the returned
** list.
** Link all tokens from hash table iHash into a list in sorted order. The
** tokens are not removed from the hash table.
*/
static int fts5HashEntrySort(
Fts5Hash *pHash,
@@ -475,7 +473,6 @@ static int fts5HashEntrySort(
pList = fts5HashEntryMerge(pList, ap[i]);
}
pHash->nEntry = 0;
sqlite3_free(ap);
*ppSorted = pList;
return SQLITE_OK;
@@ -529,6 +526,28 @@ int sqlite3Fts5HashScanInit(
return fts5HashEntrySort(p, pTerm, nTerm, &p->pScan);
}
#ifdef SQLITE_DEBUG
static int fts5HashCount(Fts5Hash *pHash){
int nEntry = 0;
int ii;
for(ii=0; ii<pHash->nSlot; ii++){
Fts5HashEntry *p = 0;
for(p=pHash->aSlot[ii]; p; p=p->pHashNext){
nEntry++;
}
}
return nEntry;
}
#endif
/*
** Return true if the hash table is empty, false otherwise.
*/
int sqlite3Fts5HashIsEmpty(Fts5Hash *pHash){
assert( pHash->nEntry==fts5HashCount(pHash) );
return pHash->nEntry==0;
}
void sqlite3Fts5HashScanNext(Fts5Hash *p){
assert( !sqlite3Fts5HashScanEof(p) );
p->pScan = p->pScan->pScanNext;
+1392 -315
View File
File diff suppressed because it is too large Load Diff
+131 -19
View File
@@ -117,6 +117,8 @@ struct Fts5FullTable {
Fts5Storage *pStorage; /* Document store */
Fts5Global *pGlobal; /* Global (connection wide) data */
Fts5Cursor *pSortCsr; /* Sort data from this cursor */
int iSavepoint; /* Successful xSavepoint()+1 */
int bInSavepoint;
#ifdef SQLITE_DEBUG
struct Fts5TransactionState ts;
#endif
@@ -405,6 +407,13 @@ static int fts5InitVtab(
pConfig->pzErrmsg = 0;
}
if( rc==SQLITE_OK && pConfig->eContent==FTS5_CONTENT_NORMAL ){
rc = sqlite3_vtab_config(db, SQLITE_VTAB_CONSTRAINT_SUPPORT, (int)1);
}
if( rc==SQLITE_OK ){
rc = sqlite3_vtab_config(db, SQLITE_VTAB_INNOCUOUS);
}
if( rc!=SQLITE_OK ){
fts5FreeVtab(pTab);
pTab = 0;
@@ -1329,6 +1338,9 @@ static int fts5FilterMethod(
pCsr->iFirstRowid = fts5GetRowidLimit(pRowidGe, SMALLEST_INT64);
}
rc = sqlite3Fts5IndexLoadConfig(pTab->p.pIndex);
if( rc!=SQLITE_OK ) goto filter_out;
if( pTab->pSortCsr ){
/* If pSortCsr is non-NULL, then this call is being made as part of
** processing for a "... MATCH <expr> ORDER BY rank" query (ePlan is
@@ -1351,6 +1363,7 @@ static int fts5FilterMethod(
pCsr->pExpr = pTab->pSortCsr->pExpr;
rc = fts5CursorFirst(pTab, pCsr, bDesc);
}else if( pCsr->pExpr ){
assert( rc==SQLITE_OK );
rc = fts5CursorParseRank(pConfig, pCsr, pRank);
if( rc==SQLITE_OK ){
if( bOrderByRank ){
@@ -1522,6 +1535,7 @@ static int fts5SpecialInsert(
Fts5Config *pConfig = pTab->p.pConfig;
int rc = SQLITE_OK;
int bError = 0;
int bLoadConfig = 0;
if( 0==sqlite3_stricmp("delete-all", zCmd) ){
if( pConfig->eContent==FTS5_CONTENT_NORMAL ){
@@ -1533,6 +1547,7 @@ static int fts5SpecialInsert(
}else{
rc = sqlite3Fts5StorageDeleteAll(pTab->pStorage);
}
bLoadConfig = 1;
}else if( 0==sqlite3_stricmp("rebuild", zCmd) ){
if( pConfig->eContent==FTS5_CONTENT_NONE ){
fts5SetVtabError(pTab,
@@ -1542,6 +1557,7 @@ static int fts5SpecialInsert(
}else{
rc = sqlite3Fts5StorageRebuild(pTab->pStorage);
}
bLoadConfig = 1;
}else if( 0==sqlite3_stricmp("optimize", zCmd) ){
rc = sqlite3Fts5StorageOptimize(pTab->pStorage);
}else if( 0==sqlite3_stricmp("merge", zCmd) ){
@@ -1554,6 +1570,8 @@ static int fts5SpecialInsert(
}else if( 0==sqlite3_stricmp("prefix-index", zCmd) ){
pConfig->bPrefixIndex = sqlite3_value_int(pVal);
#endif
}else if( 0==sqlite3_stricmp("flush", zCmd) ){
rc = sqlite3Fts5FlushToDisk(&pTab->p);
}else{
rc = sqlite3Fts5IndexLoadConfig(pTab->p.pIndex);
if( rc==SQLITE_OK ){
@@ -1567,6 +1585,12 @@ static int fts5SpecialInsert(
}
}
}
if( rc==SQLITE_OK && bLoadConfig ){
pTab->p.pConfig->iCookie--;
rc = sqlite3Fts5IndexLoadConfig(pTab->p.pIndex);
}
return rc;
}
@@ -1624,7 +1648,6 @@ static int fts5UpdateMethod(
int eType0; /* value_type() of apVal[0] */
int rc = SQLITE_OK; /* Return code */
int bUpdateOrDelete = 0;
/* A transaction must be open when this is called. */
assert( pTab->ts.eState==1 || pTab->ts.eState==2 );
@@ -1654,7 +1677,14 @@ static int fts5UpdateMethod(
if( pConfig->eContent!=FTS5_CONTENT_NORMAL
&& 0==sqlite3_stricmp("delete", z)
){
rc = fts5SpecialDelete(pTab, apVal);
if( pConfig->bContentlessDelete ){
fts5SetVtabError(pTab,
"'delete' may not be used with a contentless_delete=1 table"
);
rc = SQLITE_ERROR;
}else{
rc = fts5SpecialDelete(pTab, apVal);
}
}else{
rc = fts5SpecialInsert(pTab, z, apVal[2 + pConfig->nCol + 1]);
}
@@ -1671,7 +1701,7 @@ static int fts5UpdateMethod(
** Cases 3 and 4 may violate the rowid constraint.
*/
int eConflict = SQLITE_ABORT;
if( pConfig->eContent==FTS5_CONTENT_NORMAL ){
if( pConfig->eContent==FTS5_CONTENT_NORMAL || pConfig->bContentlessDelete ){
eConflict = sqlite3_vtab_on_conflict(pConfig->db);
}
@@ -1679,8 +1709,12 @@ static int fts5UpdateMethod(
assert( nArg!=1 || eType0==SQLITE_INTEGER );
/* Filter out attempts to run UPDATE or DELETE on contentless tables.
** This is not suported. */
if( eType0==SQLITE_INTEGER && fts5IsContentless(pTab) ){
** This is not suported. Except - they are both supported if the CREATE
** VIRTUAL TABLE statement contained "contentless_delete=1". */
if( eType0==SQLITE_INTEGER
&& pConfig->eContent==FTS5_CONTENT_NONE
&& pConfig->bContentlessDelete==0
){
pTab->p.base.zErrMsg = sqlite3_mprintf(
"cannot %s contentless fts5 table: %s",
(nArg>1 ? "UPDATE" : "DELETE from"), pConfig->zName
@@ -1704,7 +1738,8 @@ static int fts5UpdateMethod(
}
else if( eType0!=SQLITE_INTEGER ){
/* If this is a REPLACE, first remove the current entry (if any) */
/* An INSERT statement. If the conflict-mode is REPLACE, first remove
** the current entry (if any). */
if( eConflict==SQLITE_REPLACE && eType1==SQLITE_INTEGER ){
i64 iNew = sqlite3_value_int64(apVal[1]); /* Rowid to delete */
rc = sqlite3Fts5StorageDelete(pTab->pStorage, iNew, 0);
@@ -1767,8 +1802,7 @@ static int fts5SyncMethod(sqlite3_vtab *pVtab){
Fts5FullTable *pTab = (Fts5FullTable*)pVtab;
fts5CheckTransactionState(pTab, FTS5_SYNC, 0);
pTab->p.pConfig->pzErrmsg = &pTab->p.base.zErrMsg;
fts5TripCursors(pTab);
rc = sqlite3Fts5StorageSync(pTab->pStorage);
rc = sqlite3Fts5FlushToDisk(&pTab->p);
pTab->p.pConfig->pzErrmsg = 0;
return rc;
}
@@ -2535,6 +2569,12 @@ static int fts5ColumnMethod(
sqlite3_result_value(pCtx, sqlite3_column_value(pCsr->pStmt, iCol+1));
}
pConfig->pzErrmsg = 0;
}else if( pConfig->bContentlessDelete && sqlite3_vtab_nochange(pCtx) ){
char *zErr = sqlite3_mprintf("cannot UPDATE a subset of "
"columns on fts5 contentless-delete table: %s", pConfig->zName
);
sqlite3_result_error(pCtx, zErr, -1);
sqlite3_free(zErr);
}
return rc;
}
@@ -2573,8 +2613,12 @@ static int fts5RenameMethod(
sqlite3_vtab *pVtab, /* Virtual table handle */
const char *zName /* New name of table */
){
int rc;
Fts5FullTable *pTab = (Fts5FullTable*)pVtab;
return sqlite3Fts5StorageRename(pTab->pStorage, zName);
pTab->bInSavepoint = 1;
rc = sqlite3Fts5StorageRename(pTab->pStorage, zName);
pTab->bInSavepoint = 0;
return rc;
}
int sqlite3Fts5FlushToDisk(Fts5Table *pTab){
@@ -2588,9 +2632,29 @@ int sqlite3Fts5FlushToDisk(Fts5Table *pTab){
** Flush the contents of the pending-terms table to disk.
*/
static int fts5SavepointMethod(sqlite3_vtab *pVtab, int iSavepoint){
UNUSED_PARAM(iSavepoint); /* Call below is a no-op for NDEBUG builds */
fts5CheckTransactionState((Fts5FullTable*)pVtab, FTS5_SAVEPOINT, iSavepoint);
return sqlite3Fts5FlushToDisk((Fts5Table*)pVtab);
Fts5FullTable *pTab = (Fts5FullTable*)pVtab;
int rc = SQLITE_OK;
char *zSql = 0;
fts5CheckTransactionState(pTab, FTS5_SAVEPOINT, iSavepoint);
if( pTab->bInSavepoint==0 ){
zSql = sqlite3_mprintf("INSERT INTO %Q.%Q(%Q) VALUES('flush')",
pTab->p.pConfig->zDb, pTab->p.pConfig->zName, pTab->p.pConfig->zName
);
if( zSql ){
pTab->bInSavepoint = 1;
rc = sqlite3_exec(pTab->p.pConfig->db, zSql, 0, 0, 0);
pTab->bInSavepoint = 0;
sqlite3_free(zSql);
}else{
rc = SQLITE_NOMEM;
}
if( rc==SQLITE_OK ){
pTab->iSavepoint = iSavepoint+1;
}
}
return rc;
}
/*
@@ -2599,9 +2663,16 @@ static int fts5SavepointMethod(sqlite3_vtab *pVtab, int iSavepoint){
** This is a no-op.
*/
static int fts5ReleaseMethod(sqlite3_vtab *pVtab, int iSavepoint){
UNUSED_PARAM(iSavepoint); /* Call below is a no-op for NDEBUG builds */
fts5CheckTransactionState((Fts5FullTable*)pVtab, FTS5_RELEASE, iSavepoint);
return sqlite3Fts5FlushToDisk((Fts5Table*)pVtab);
Fts5FullTable *pTab = (Fts5FullTable*)pVtab;
int rc = SQLITE_OK;
fts5CheckTransactionState(pTab, FTS5_RELEASE, iSavepoint);
if( (iSavepoint+1)<pTab->iSavepoint ){
rc = sqlite3Fts5FlushToDisk(&pTab->p);
if( rc==SQLITE_OK ){
pTab->iSavepoint = iSavepoint;
}
}
return rc;
}
/*
@@ -2611,11 +2682,14 @@ static int fts5ReleaseMethod(sqlite3_vtab *pVtab, int iSavepoint){
*/
static int fts5RollbackToMethod(sqlite3_vtab *pVtab, int iSavepoint){
Fts5FullTable *pTab = (Fts5FullTable*)pVtab;
UNUSED_PARAM(iSavepoint); /* Call below is a no-op for NDEBUG builds */
int rc = SQLITE_OK;
fts5CheckTransactionState(pTab, FTS5_ROLLBACKTO, iSavepoint);
fts5TripCursors(pTab);
pTab->p.pConfig->pgsz = 0;
return sqlite3Fts5StorageRollback(pTab->pStorage);
if( (iSavepoint+1)<=pTab->iSavepoint ){
rc = sqlite3Fts5StorageRollback(pTab->pStorage);
}
return rc;
}
/*
@@ -2835,9 +2909,46 @@ static int fts5ShadowName(const char *zName){
return 0;
}
/*
** Run an integrity check on the FTS5 data structures. Return a string
** if anything is found amiss. Return a NULL pointer if everything is
** OK.
*/
static int fts5Integrity(
sqlite3_vtab *pVtab, /* the FTS5 virtual table to check */
const char *zSchema, /* Name of schema in which this table lives */
const char *zTabname, /* Name of the table itself */
int isQuick, /* True if this is a quick-check */
char **pzErr /* Write error message here */
){
Fts5FullTable *pTab = (Fts5FullTable*)pVtab;
Fts5Config *pConfig = pTab->p.pConfig;
char *zSql;
char *zErr = 0;
int rc;
assert( pzErr!=0 && *pzErr==0 );
UNUSED_PARAM(isQuick);
zSql = sqlite3_mprintf(
"INSERT INTO \"%w\".\"%w\"(\"%w\") VALUES('integrity-check');",
zSchema, zTabname, pConfig->zName);
if( zSql==0 ) return SQLITE_NOMEM;
rc = sqlite3_exec(pConfig->db, zSql, 0, 0, &zErr);
sqlite3_free(zSql);
if( (rc&0xff)==SQLITE_CORRUPT ){
*pzErr = sqlite3_mprintf("malformed inverted index for FTS5 table %s.%s",
zSchema, zTabname);
}else if( rc!=SQLITE_OK ){
*pzErr = sqlite3_mprintf("unable to validate the inverted index for"
" FTS5 table %s.%s: %s",
zSchema, zTabname, zErr);
}
sqlite3_free(zErr);
return SQLITE_OK;
}
static int fts5Init(sqlite3 *db){
static const sqlite3_module fts5Mod = {
/* iVersion */ 3,
/* iVersion */ 4,
/* xCreate */ fts5CreateMethod,
/* xConnect */ fts5ConnectMethod,
/* xBestIndex */ fts5BestIndexMethod,
@@ -2860,7 +2971,8 @@ static int fts5Init(sqlite3 *db){
/* xSavepoint */ fts5SavepointMethod,
/* xRelease */ fts5ReleaseMethod,
/* xRollbackTo */ fts5RollbackToMethod,
/* xShadowName */ fts5ShadowName
/* xShadowName */ fts5ShadowName,
/* xIntegrity */ fts5Integrity
};
int rc;
+75 -13
View File
@@ -77,10 +77,10 @@ static int fts5StorageGetStmt(
"INSERT INTO %Q.'%q_content' VALUES(%s)", /* INSERT_CONTENT */
"REPLACE INTO %Q.'%q_content' VALUES(%s)", /* REPLACE_CONTENT */
"DELETE FROM %Q.'%q_content' WHERE id=?", /* DELETE_CONTENT */
"REPLACE INTO %Q.'%q_docsize' VALUES(?,?)", /* REPLACE_DOCSIZE */
"REPLACE INTO %Q.'%q_docsize' VALUES(?,?%s)", /* REPLACE_DOCSIZE */
"DELETE FROM %Q.'%q_docsize' WHERE id=?", /* DELETE_DOCSIZE */
"SELECT sz FROM %Q.'%q_docsize' WHERE id=?", /* LOOKUP_DOCSIZE */
"SELECT sz%s FROM %Q.'%q_docsize' WHERE id=?", /* LOOKUP_DOCSIZE */
"REPLACE INTO %Q.'%q_config' VALUES(?,?)", /* REPLACE_CONFIG */
"SELECT %s FROM %s AS T", /* SCAN */
@@ -128,6 +128,19 @@ static int fts5StorageGetStmt(
break;
}
case FTS5_STMT_REPLACE_DOCSIZE:
zSql = sqlite3_mprintf(azStmt[eStmt], pC->zDb, pC->zName,
(pC->bContentlessDelete ? ",?" : "")
);
break;
case FTS5_STMT_LOOKUP_DOCSIZE:
zSql = sqlite3_mprintf(azStmt[eStmt],
(pC->bContentlessDelete ? ",origin" : ""),
pC->zDb, pC->zName
);
break;
default:
zSql = sqlite3_mprintf(azStmt[eStmt], pC->zDb, pC->zName);
break;
@@ -317,9 +330,11 @@ int sqlite3Fts5StorageOpen(
}
if( rc==SQLITE_OK && pConfig->bColumnsize ){
rc = sqlite3Fts5CreateTable(
pConfig, "docsize", "id INTEGER PRIMARY KEY, sz BLOB", 0, pzErr
);
const char *zCols = "id INTEGER PRIMARY KEY, sz BLOB";
if( pConfig->bContentlessDelete ){
zCols = "id INTEGER PRIMARY KEY, sz BLOB, origin INTEGER";
}
rc = sqlite3Fts5CreateTable(pConfig, "docsize", zCols, 0, pzErr);
}
if( rc==SQLITE_OK ){
rc = sqlite3Fts5CreateTable(
@@ -396,7 +411,7 @@ static int fts5StorageDeleteFromIndex(
){
Fts5Config *pConfig = p->pConfig;
sqlite3_stmt *pSeek = 0; /* SELECT to read row iDel from %_data */
int rc; /* Return code */
int rc = SQLITE_OK; /* Return code */
int rc2; /* sqlite3_reset() return code */
int iCol;
Fts5InsertCtx ctx;
@@ -412,7 +427,6 @@ static int fts5StorageDeleteFromIndex(
ctx.pStorage = p;
ctx.iCol = -1;
rc = sqlite3Fts5IndexBeginWrite(p->pIndex, 1, iDel);
for(iCol=1; rc==SQLITE_OK && iCol<=pConfig->nCol; iCol++){
if( pConfig->abUnindexed[iCol-1]==0 ){
const char *zText;
@@ -449,6 +463,37 @@ static int fts5StorageDeleteFromIndex(
return rc;
}
/*
** This function is called to process a DELETE on a contentless_delete=1
** table. It adds the tombstone required to delete the entry with rowid
** iDel. If successful, SQLITE_OK is returned. Or, if an error occurs,
** an SQLite error code.
*/
static int fts5StorageContentlessDelete(Fts5Storage *p, i64 iDel){
i64 iOrigin = 0;
sqlite3_stmt *pLookup = 0;
int rc = SQLITE_OK;
assert( p->pConfig->bContentlessDelete );
assert( p->pConfig->eContent==FTS5_CONTENT_NONE );
/* Look up the origin of the document in the %_docsize table. Store
** this in stack variable iOrigin. */
rc = fts5StorageGetStmt(p, FTS5_STMT_LOOKUP_DOCSIZE, &pLookup, 0);
if( rc==SQLITE_OK ){
sqlite3_bind_int64(pLookup, 1, iDel);
if( SQLITE_ROW==sqlite3_step(pLookup) ){
iOrigin = sqlite3_column_int64(pLookup, 1);
}
rc = sqlite3_reset(pLookup);
}
if( rc==SQLITE_OK && iOrigin!=0 ){
rc = sqlite3Fts5IndexContentlessDelete(p->pIndex, iOrigin, iDel);
}
return rc;
}
/*
** Insert a record into the %_docsize table. Specifically, do:
@@ -469,10 +514,17 @@ static int fts5StorageInsertDocsize(
rc = fts5StorageGetStmt(p, FTS5_STMT_REPLACE_DOCSIZE, &pReplace, 0);
if( rc==SQLITE_OK ){
sqlite3_bind_int64(pReplace, 1, iRowid);
sqlite3_bind_blob(pReplace, 2, pBuf->p, pBuf->n, SQLITE_STATIC);
sqlite3_step(pReplace);
rc = sqlite3_reset(pReplace);
sqlite3_bind_null(pReplace, 2);
if( p->pConfig->bContentlessDelete ){
i64 iOrigin = 0;
rc = sqlite3Fts5IndexGetOrigin(p->pIndex, &iOrigin);
sqlite3_bind_int64(pReplace, 3, iOrigin);
}
if( rc==SQLITE_OK ){
sqlite3_bind_blob(pReplace, 2, pBuf->p, pBuf->n, SQLITE_STATIC);
sqlite3_step(pReplace);
rc = sqlite3_reset(pReplace);
sqlite3_bind_null(pReplace, 2);
}
}
}
return rc;
@@ -536,7 +588,15 @@ int sqlite3Fts5StorageDelete(Fts5Storage *p, i64 iDel, sqlite3_value **apVal){
/* Delete the index records */
if( rc==SQLITE_OK ){
rc = fts5StorageDeleteFromIndex(p, iDel, apVal);
rc = sqlite3Fts5IndexBeginWrite(p->pIndex, 1, iDel);
}
if( rc==SQLITE_OK ){
if( p->pConfig->bContentlessDelete ){
rc = fts5StorageContentlessDelete(p, iDel);
}else{
rc = fts5StorageDeleteFromIndex(p, iDel, apVal);
}
}
/* Delete the %_docsize record */
@@ -1124,7 +1184,9 @@ int sqlite3Fts5StorageSync(Fts5Storage *p){
i64 iLastRowid = sqlite3_last_insert_rowid(p->pConfig->db);
if( p->bTotalsValid ){
rc = fts5StorageSaveTotals(p);
p->bTotalsValid = 0;
if( rc==SQLITE_OK ){
p->bTotalsValid = 0;
}
}
if( rc==SQLITE_OK ){
rc = sqlite3Fts5IndexSync(p->pIndex);
+2 -1
View File
@@ -472,7 +472,8 @@ int sqlite3Fts5TestRegisterTok(sqlite3 *db, fts5_api *pApi){
0, /* xSavepoint */
0, /* xRelease */
0, /* xRollbackTo */
0 /* xShadowName */
0, /* xShadowName */
0 /* xIntegrity */
};
int rc; /* Return code */
+70 -27
View File
@@ -229,6 +229,12 @@ static const unsigned char sqlite3Utf8Trans1[] = {
#endif /* ifndef SQLITE_AMALGAMATION */
#define FTS5_SKIP_UTF8(zIn) { \
if( ((unsigned char)(*(zIn++)))>=0xc0 ){ \
while( (((unsigned char)*zIn) & 0xc0)==0x80 ){ zIn++; } \
} \
}
typedef struct Unicode61Tokenizer Unicode61Tokenizer;
struct Unicode61Tokenizer {
unsigned char aTokenChar[128]; /* ASCII range token characters */
@@ -1264,6 +1270,7 @@ static int fts5PorterTokenize(
typedef struct TrigramTokenizer TrigramTokenizer;
struct TrigramTokenizer {
int bFold; /* True to fold to lower-case */
int iFoldParam; /* Parameter to pass to Fts5UnicodeFold() */
};
/*
@@ -1290,6 +1297,7 @@ static int fts5TriCreate(
}else{
int i;
pNew->bFold = 1;
pNew->iFoldParam = 0;
for(i=0; rc==SQLITE_OK && i<nArg; i+=2){
const char *zArg = azArg[i+1];
if( 0==sqlite3_stricmp(azArg[i], "case_sensitive") ){
@@ -1298,10 +1306,21 @@ static int fts5TriCreate(
}else{
pNew->bFold = (zArg[0]=='0');
}
}else if( 0==sqlite3_stricmp(azArg[i], "remove_diacritics") ){
if( (zArg[0]!='0' && zArg[0]!='1' && zArg[0]!='2') || zArg[1] ){
rc = SQLITE_ERROR;
}else{
pNew->iFoldParam = (zArg[0]!='0') ? 2 : 0;
}
}else{
rc = SQLITE_ERROR;
}
}
if( pNew->iFoldParam!=0 && pNew->bFold==0 ){
rc = SQLITE_ERROR;
}
if( rc!=SQLITE_OK ){
fts5TriDelete((Fts5Tokenizer*)pNew);
pNew = 0;
@@ -1324,40 +1343,62 @@ static int fts5TriTokenize(
TrigramTokenizer *p = (TrigramTokenizer*)pTok;
int rc = SQLITE_OK;
char aBuf[32];
char *zOut = aBuf;
int ii;
const unsigned char *zIn = (const unsigned char*)pText;
const unsigned char *zEof = &zIn[nText];
u32 iCode;
int aStart[3]; /* Input offset of each character in aBuf[] */
UNUSED_PARAM(unusedFlags);
while( 1 ){
char *zOut = aBuf;
int iStart = zIn - (const unsigned char*)pText;
const unsigned char *zNext;
READ_UTF8(zIn, zEof, iCode);
if( iCode==0 ) break;
zNext = zIn;
if( zIn<zEof ){
if( p->bFold ) iCode = sqlite3Fts5UnicodeFold(iCode, 0);
WRITE_UTF8(zOut, iCode);
/* Populate aBuf[] with the characters for the first trigram. */
for(ii=0; ii<3; ii++){
do {
aStart[ii] = zIn - (const unsigned char*)pText;
READ_UTF8(zIn, zEof, iCode);
if( iCode==0 ) return SQLITE_OK;
if( p->bFold ) iCode = sqlite3Fts5UnicodeFold(iCode, p->iFoldParam);
}while( iCode==0 );
WRITE_UTF8(zOut, iCode);
}
/* At the start of each iteration of this loop:
**
** aBuf: Contains 3 characters. The 3 characters of the next trigram.
** zOut: Points to the byte following the last character in aBuf.
** aStart[3]: Contains the byte offset in the input text corresponding
** to the start of each of the three characters in the buffer.
*/
assert( zIn<=zEof );
while( 1 ){
int iNext; /* Start of character following current tri */
const char *z1;
/* Read characters from the input up until the first non-diacritic */
do {
iNext = zIn - (const unsigned char*)pText;
READ_UTF8(zIn, zEof, iCode);
if( iCode==0 ) break;
}else{
break;
}
if( zIn<zEof ){
if( p->bFold ) iCode = sqlite3Fts5UnicodeFold(iCode, 0);
WRITE_UTF8(zOut, iCode);
READ_UTF8(zIn, zEof, iCode);
if( iCode==0 ) break;
if( p->bFold ) iCode = sqlite3Fts5UnicodeFold(iCode, 0);
WRITE_UTF8(zOut, iCode);
}else{
break;
}
rc = xToken(pCtx, 0, aBuf, zOut-aBuf, iStart, iStart + zOut-aBuf);
if( rc!=SQLITE_OK ) break;
zIn = zNext;
if( p->bFold ) iCode = sqlite3Fts5UnicodeFold(iCode, p->iFoldParam);
}while( iCode==0 );
/* Pass the current trigram back to fts5 */
rc = xToken(pCtx, 0, aBuf, zOut-aBuf, aStart[0], iNext);
if( iCode==0 || rc!=SQLITE_OK ) break;
/* Remove the first character from buffer aBuf[]. Append the character
** with codepoint iCode. */
z1 = aBuf;
FTS5_SKIP_UTF8(z1);
memmove(aBuf, z1, zOut - z1);
zOut -= (z1 - aBuf);
WRITE_UTF8(zOut, iCode);
/* Update the aStart[] array */
aStart[0] = aStart[1];
aStart[1] = aStart[2];
aStart[2] = iNext;
}
return rc;
@@ -1380,7 +1421,9 @@ int sqlite3Fts5TokenizerPattern(
){
if( xCreate==fts5TriCreate ){
TrigramTokenizer *p = (TrigramTokenizer*)pTok;
return p->bFold ? FTS5_PATTERN_LIKE : FTS5_PATTERN_GLOB;
if( p->iFoldParam==0 ){
return p->bFold ? FTS5_PATTERN_LIKE : FTS5_PATTERN_GLOB;
}
}
return FTS5_PATTERN_NONE;
}
+2 -1
View File
@@ -783,7 +783,8 @@ int sqlite3Fts5VocabInit(Fts5Global *pGlobal, sqlite3 *db){
/* xSavepoint */ 0,
/* xRelease */ 0,
/* xRollbackTo */ 0,
/* xShadowName */ 0
/* xShadowName */ 0,
/* xIntegrity */ 0
};
void *p = (void*)pGlobal;
+5 -2
View File
@@ -65,7 +65,9 @@ foreach w {a b c d e f} {
do_execsql_test 2.4 {
INSERT INTO t1(t1) VALUES('integrity-check');
}
PRAGMA integrity_check;
PRAGMA integrity_check(t1);
} {ok ok}
#-------------------------------------------------------------------------
@@ -88,6 +90,7 @@ foreach {i x y} {
} {
do_execsql_test 3.$i.1 { INSERT INTO t1 VALUES($x, $y) }
do_execsql_test 3.$i.2 { INSERT INTO t1(t1) VALUES('integrity-check') }
do_execsql_test 3.$i.3 { PRAGMA integrity_check(t1) } ok
if {[set_test_counter errors]} break
}
@@ -135,7 +138,7 @@ foreach {i x y} {
10 {ddd abcde dddd dd c} {dddd c c d abcde}
} {
do_execsql_test 5.$i.1 { INSERT INTO t1 VALUES($x, $y) }
do_execsql_test 5.$i.2 { INSERT INTO t1(t1) VALUES('integrity-check') }
do_execsql_test 5.$i.2 { PRAGMA integrity_check(t1) } ok
if {[set_test_counter errors]} break
}
+27 -1
View File
@@ -307,5 +307,31 @@ do_catchsql_test 10.1.4 {
SELECT group_concat(firstcol(t1), '.') FROM t1 GROUP BY rowid
} {1 {unable to use function firstcol in the requested context}}
finish_test
#-------------------------------------------------------------------------
# Test that xInstCount() works from within an xPhraseQuery() callback.
#
reset_db
proc xCallback {cmd} {
incr ::hitcount [$cmd xInstCount]
return SQLITE_OK
}
proc fts5_hitcount {cmd} {
set ::hitcount 0
$cmd xQueryPhrase 0 xCallback
return $::hitcount
}
sqlite3_fts5_create_function db fts5_hitcount fts5_hitcount
do_execsql_test 11.1 {
CREATE VIRTUAL TABLE x1 USING fts5(z);
INSERT INTO x1 VALUES('one two three');
INSERT INTO x1 VALUES('one two one three one');
INSERT INTO x1 VALUES('one two three');
}
do_execsql_test 11.2 {
SELECT fts5_hitcount(x1) FROM x1('one') LIMIT 1;
} {5}
finish_test
+40
View File
@@ -65,4 +65,44 @@ do_execsql_test 2.1 {
INSERT INTO fts_idx(fts_idx) VALUES('integrity-check');
}
#-------------------------------------------------------------------------
# Tests for OR IGNORE conflict handling.
#
reset_db
foreach_detail_mode $::testprefix {
do_execsql_test 3.0 {
CREATE VIRTUAL TABLE t1 USING fts5(xyz, detail=%DETAIL%);
BEGIN;
INSERT INTO t1(rowid, xyz) VALUES(13, 'thirteen documents');
INSERT INTO t1(rowid, xyz) VALUES(14, 'fourteen documents');
INSERT INTO t1(rowid, xyz) VALUES(15, 'fifteen documents');
COMMIT;
}
set db_cksum [cksum]
foreach {tn sql} {
1 {
INSERT OR IGNORE INTO t1(rowid, xyz) VALUES(14, 'new text');
}
2 {
UPDATE OR IGNORE t1 SET rowid=13 WHERE rowid=15;
}
3 {
INSERT OR IGNORE INTO t1(rowid, xyz)
SELECT 13, 'some text'
UNION ALL
SELECT 14, 'some text'
UNION ALL
SELECT 15, 'some text'
}
} {
do_execsql_test 3.1.$tn.1 $sql
do_test 3.1.$tn.2 { cksum } $db_cksum
}
}
finish_test
-1
View File
@@ -294,4 +294,3 @@ do_catchsql_test 7.2.5 {
} {1 {recursively defined fts5 content table}}
finish_test
+270
View File
@@ -0,0 +1,270 @@
# 2014 Dec 20
#
# The author disclaims copyright to this source code. In place of
# a legal notice, here is a blessing:
#
# May you do good and not evil.
# May you find forgiveness for yourself and forgive others.
# May you share freely, never taking more than you give.
#
#***********************************************************************
#
# This file contains tests for the content= and content_rowid= options.
#
source [file join [file dirname [info script]] fts5_common.tcl]
set testprefix fts5contentless
# If SQLITE_ENABLE_FTS5 is defined, omit this file.
ifcapable !fts5 {
finish_test
return
}
# Check that it is not possible to specify "contentless_delete=1" for
# anything other than a contentless table.
#
set res(0) {0 {}}
set res(1) {1 {contentless_delete=1 requires a contentless table}}
foreach {tn sql bError} {
1 "(a, b, contentless_delete=1)" 1
2 "(a, b, contentless_delete=1, content=abc)" 1
3 "(a, b, contentless_delete=1, content=)" 0
4 "(content=, contentless_delete=1, a)" 0
5 "(content='', contentless_delete=1, hello)" 0
} {
execsql { BEGIN }
do_catchsql_test 1.$tn "CREATE VIRTUAL TABLE t1 USING fts5 $sql" $res($bError)
execsql { ROLLBACK }
}
# Check that it is not possible to specify "contentless_delete=1"
# along with columnsize=1.
#
set res(0) {0 {}}
set res(1) {1 {contentless_delete=1 is incompatible with columnsize=0}}
foreach {tn sql bError} {
2 "(a, b, content='', contentless_delete=1, columnsize=0)" 1
} {
execsql { BEGIN }
do_catchsql_test 1.$tn "CREATE VIRTUAL TABLE t1 USING fts5 $sql" $res($bError)
execsql { ROLLBACK }
}
# Check that if contentless_delete=1 is specified, then the "origin"
# column is added to the %_docsize table.
reset_db
do_execsql_test 3.0 {
CREATE VIRTUAL TABLE x1 USING fts5(c, content='');
CREATE VIRTUAL TABLE x2 USING fts5(c, content='', contentless_delete=1);
}
do_execsql_test 3.1 {
SELECT sql FROM sqlite_schema WHERE name IN ('x1_docsize', 'x2_docsize');
} {
{CREATE TABLE 'x1_docsize'(id INTEGER PRIMARY KEY, sz BLOB)}
{CREATE TABLE 'x2_docsize'(id INTEGER PRIMARY KEY, sz BLOB, origin INTEGER)}
}
do_execsql_test 3.2.1 {
SELECT hex(block) FROM x1_data WHERE id=10
} {00000000000000}
do_execsql_test 3.2.2 {
SELECT hex(block) FROM x2_data WHERE id=10
} {00000000FF000001000000}
do_execsql_test 3.3 {
INSERT INTO x2 VALUES('first text');
INSERT INTO x2 VALUES('second text');
}
do_execsql_test 3.4 {
SELECT id, origin FROM x2_docsize
} {1 1 2 2}
do_execsql_test 3.5 {
SELECT level, segment, loc1, loc2 FROM fts5_structure(
(SELECT block FROM x2_data WHERE id=10)
)
} {
0 0 1 1
0 1 2 2
}
do_execsql_test 3.6 {
INSERT INTO x2(x2) VALUES('optimize');
}
do_execsql_test 3.7 {
SELECT level, segment, loc1, loc2 FROM fts5_structure(
(SELECT block FROM x2_data WHERE id=10)
)
} {
1 0 1 2
}
do_execsql_test 3.8 {
DELETE FROM x2 WHERE rowid=2;
}
do_execsql_test 3.9 {
SELECT rowid FROM x2('text')
} {1}
#--------------------------------------------------------------------------
reset_db
proc document {n} {
set vocab [list A B C D E F G H I J K L M N O P Q R S T U V W X Y Z]
set ret [list]
for {set ii 0} {$ii < $n} {incr ii} {
lappend ret [lindex $vocab [expr int(rand()*[llength $vocab])]]
}
set ret
}
set nRow 1000
do_execsql_test 4.0 {
CREATE TABLE t1(x);
CREATE VIRTUAL TABLE ft USING fts5(x, content='', contentless_delete=1);
INSERT INTO ft(ft, rank) VALUES('pgsz', 100);
}
do_test 4.1 {
for {set ii 0} {$ii < $nRow} {incr ii} {
set doc [document 6]
execsql {
INSERT INTO t1 VALUES($doc);
INSERT INTO ft VALUES($doc);
}
}
} {}
foreach v {A B C D E F G H I J K L M N O P Q R S T U V W X Y Z} {
set L1 [execsql {SELECT rowid FROM t1 WHERE x LIKE '%'||$v||'%'}]
set L2 [execsql {SELECT rowid FROM ft($v)}]
do_test 4.2.$v { set L1 } $L2
}
do_test 4.3 {
for {set ii 1} {$ii < $nRow} {incr ii 2} {
execsql {
DELETE FROM ft WHERE rowid=$ii;
DELETE FROM t1 WHERE rowid=$ii;
}
}
} {}
foreach v {A B C D E F G H I J K L M N O P Q R S T U V W X Y Z} {
set L1 [execsql {SELECT rowid FROM t1 WHERE x LIKE '%'||$v||'%'}]
set L2 [execsql {SELECT rowid FROM ft($v)}]
do_test 4.4.$v { set L1 } $L2
}
do_execsql_test 4.5 {
INSERT INTO ft(ft) VALUES('optimize');
} {}
foreach v {A B C D E F G H I J K L M N O P Q R S T U V W X Y Z} {
set L1 [execsql {SELECT rowid FROM t1 WHERE x LIKE '%'||$v||'%'}]
set L2 [execsql {SELECT rowid FROM ft($v)}]
do_test 4.6.$v { set L1 } $L2
}
#execsql_pp { SELECT fts5_decode(id, block) FROM ft_data }
#-------------------------------------------------------------------------
reset_db
do_execsql_test 5.0 {
CREATE VIRTUAL TABLE ft USING fts5(x, content='', contentless_delete=1);
INSERT INTO ft(rowid, x) VALUES(1, 'one two three');
INSERT INTO ft(rowid, x) VALUES(2, 'one two four');
INSERT INTO ft(rowid, x) VALUES(3, 'one two five');
INSERT INTO ft(rowid, x) VALUES(4, 'one two seven');
INSERT INTO ft(rowid, x) VALUES(5, 'one two eight');
}
do_execsql_test 5.1 {
DELETE FROM ft WHERE rowid=2
}
do_execsql_test 5.2 {
SELECT rowid FROM ft
} {1 3 4 5}
do_catchsql_test 5.3 {
UPDATE ft SET x='four six' WHERE rowid=3
} {0 {}}
do_execsql_test 5.4 {
SELECT rowid FROM ft('one');
} {1 4 5}
do_execsql_test 5.5 {
REPLACE INTO ft(rowid, x) VALUES(3, 'four six');
SELECT rowid FROM ft('one');
} {1 4 5}
do_execsql_test 5.6 {
REPLACE INTO ft(rowid, x) VALUES(6, 'one two eleven');
SELECT rowid FROM ft('one');
} {1 4 5 6}
#-------------------------------------------------------------------------
reset_db
do_execsql_test 6.0 {
CREATE VIRTUAL TABLE ft USING fts5(x, content='', contentless_delete=1);
INSERT INTO ft(rowid, x) VALUES(1, 'one two three');
INSERT INTO ft(rowid, x) VALUES(2, 'one two four');
}
do_test 6.1 {
db eval { SELECT rowid FROM ft('one two') } {
if {$rowid==1} {
db eval { INSERT INTO ft(rowid, x) VALUES(3, 'one two four') }
}
}
} {}
#-------------------------------------------------------------------------
reset_db
do_execsql_test 7.0 {
CREATE VIRTUAL TABLE ft USING fts5(x, content='', contentless_delete=1);
}
set lRowid [list -450 0 1 2 42]
do_test 7.1 {
execsql BEGIN
foreach r $lRowid {
execsql { INSERT INTO ft(rowid, x) VALUES($r, 'one one one'); }
}
execsql COMMIT
} {}
do_test 7.2 {
execsql BEGIN
foreach r $lRowid {
execsql { REPLACE INTO ft(rowid, x) VALUES($r, 'two two two'); }
}
execsql COMMIT
} {}
do_execsql_test 7.3 { SELECT rowid FROM ft('one'); } {}
do_execsql_test 7.4 { SELECT rowid FROM ft('two'); } $lRowid
#-------------------------------------------------------------------------
reset_db
do_execsql_test 8.0 {
CREATE VIRTUAL TABLE ft USING fts5(x, content='', contentless_delete=1);
INSERT INTO ft VALUES('hello world');
INSERT INTO ft VALUES('one two three');
}
do_catchsql_test 8.1 {
INSERT INTO ft(ft, rowid, x) VALUES('delete', 1, 'hello world');
} {1 {'delete' may not be used with a contentless_delete=1 table}}
do_execsql_test 8.2 {
BEGIN;
INSERT INTO ft(rowid, x) VALUES(3, 'four four four');
DELETE FROM ft WHERE rowid=3;
COMMIT;
SELECT rowid FROM ft('four');
} {}
finish_test
+207
View File
@@ -0,0 +1,207 @@
# 2023 July 19
#
# The author disclaims copyright to this source code. In place of
# a legal notice, here is a blessing:
#
# May you do good and not evil.
# May you find forgiveness for yourself and forgive others.
# May you share freely, never taking more than you give.
#
#***********************************************************************
#
# This file contains tests for the content= and content_rowid= options.
#
source [file join [file dirname [info script]] fts5_common.tcl]
set testprefix fts5contentless2
# If SQLITE_ENABLE_FTS5 is defined, omit this file.
ifcapable !fts5 {
finish_test
return
}
proc vocab {} {
list aaa bbb ccc ddd eee fff ggg hhh iii jjj kkk lll mmm nnn ooo ppp
}
proc document {nToken} {
set doc [list]
set vocab [vocab]
for {set ii 0} {$ii < $nToken} {incr ii} {
lappend doc [lindex $vocab [expr int(rand()*[llength $vocab])]]
}
set doc
}
db func document document
proc contains {doc token} {
expr {[lsearch $doc $token]>=0}
}
db func contains contains
proc do_compare_tables_test {tn} {
uplevel [list do_test $tn {
foreach v [vocab] {
set l1 [execsql { SELECT rowid FROM t1 WHERE contains(doc, $v) }]
set l2 [execsql { SELECT rowid FROM t2($v) }]
if {$l1!=$l2} { error "1: query mismatch ($l1) ($l2)" }
set w "[string range $v 0 1]*"
set l1 [execsql { SELECT rowid FROM t1 WHERE contains(doc, $w) }]
set l2 [execsql { SELECT rowid FROM t2($w) }]
if {$l1!=$l2} { error "2: query mismatch ($l1) ($l2)" }
set w "[string range $v 0 0]*"
set l1 [execsql { SELECT rowid FROM t1 WHERE contains(doc, $w) }]
set l2 [execsql { SELECT rowid FROM t2($w) }]
if {$l1!=$l2} { error "2: query mismatch ($l1) ($l2)" }
set l1 [execsql {
SELECT rowid FROM t1 WHERE contains(doc, $v) ORDER BY rowid DESC
}]
set l2 [execsql { SELECT rowid FROM t2($v) ORDER BY rowid DESC }]
if {$l1!=$l2} { error "1: query mismatch ($l1) ($l2)" }
}
set {} {}
} {}]
}
proc lshuffle {in} {
set L [list]
set ret [list]
foreach elem $in { lappend L [list [expr rand()] $elem] }
foreach pair [lsort -index 0 $L] { lappend ret [lindex $pair 1] }
set ret
}
expr srand(0)
do_execsql_test 1.0 {
CREATE VIRTUAL TABLE t2 USING fts5(
doc, prefix=2, content=, contentless_delete=1
);
CREATE TABLE t1(doc);
CREATE TRIGGER tr1 AFTER DELETE ON t1 BEGIN
DELETE FROM t2 WHERE rowid = old.rowid;
END;
}
set SMALLEST64 -9223372036854775808
set LARGEST64 9223372036854775807
foreach {tn r1 r2} {
1 0 50
2 $SMALLEST64 $SMALLEST64+50
3 $LARGEST64-50 $LARGEST64
4 -50 -1
} {
set r1 [expr $r1]
set r2 [expr $r2]
do_test 1.1.$tn {
execsql BEGIN
for {set ii $r1} {$ii <= $r2} {incr ii} {
execsql { INSERT INTO t1(rowid, doc) VALUES ($ii, document(8)); }
}
execsql COMMIT
} {}
}
do_test 1.2 {
db eval { SELECT rowid, doc FROM t1 } {
execsql { INSERT INTO t2(rowid, doc) VALUES($rowid, $doc) }
}
} {}
foreach {tn rowid} {
1 $SMALLEST64
2 0
3 -5
4 -30
5 $LARGEST64
6 $LARGEST64-1
} {
set rowid [expr $rowid]
do_execsql_test 1.3.$tn.1 {
DELETE FROM t1 WHERE rowid=$rowid
}
do_compare_tables_test 1.3.$tn.2
}
set iTest 1
foreach r [lshuffle [execsql {SELECT rowid FROM t1}]] {
if {($iTest % 50)==0} {
execsql { INSERT INTO t2(t2) VALUES('optimize') }
}
if {($iTest % 5)==0} {
execsql { INSERT INTO t2(t2, rank) VALUES('merge', 5) }
}
do_execsql_test 1.4.$iTest.1($r) {
DELETE FROM t1 WHERE rowid=$r
}
do_compare_tables_test 1.4.$iTest.2
incr iTest
}
do_execsql_test 1.5 {
SELECT * FROM t1
} {}
#-------------------------------------------------------------------------
reset_db
db func document document
do_execsql_test 2.0 {
CREATE VIRTUAL TABLE t2 USING fts5(doc, content=, contentless_delete=1);
WITH s(i) AS (
SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<1000
)
INSERT INTO t2(rowid, doc) SELECT i, i || ' ' || i FROM s;
}
do_execsql_test 2.1 {
BEGIN;
DELETE FROM t2 WHERE rowid=32;
DELETE FROM t2 WHERE rowid=64;
DELETE FROM t2 WHERE rowid=96;
DELETE FROM t2 WHERE rowid=128;
DELETE FROM t2 WHERE rowid=160;
DELETE FROM t2 WHERE rowid=192;
COMMIT;
}
do_execsql_test 2.2 {
SELECT * FROM t2('128');
} {}
#-------------------------------------------------------------------------
foreach {tn step} {
1 3
2 7
3 15
} {
set step [expr $step]
reset_db
db func document document
do_execsql_test 3.$tn.0 {
CREATE VIRTUAL TABLE t2 USING fts5(doc, content=, contentless_delete=1);
INSERT INTO t2(t2, rank) VALUES('pgsz', 100);
WITH s(i) AS (
SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<1000
)
INSERT INTO t2(rowid, doc) SELECT i, i || ' ' || i FROM s;
}
do_execsql_test 3.$tn.1 {
DELETE FROM t2 WHERE (rowid % $step)==0
}
do_execsql_test 3.$tn.2 {
SELECT * FROM t2( $step * 5 )
} {}
}
finish_test
+195
View File
@@ -0,0 +1,195 @@
# 2023 July 21
#
# The author disclaims copyright to this source code. In place of
# a legal notice, here is a blessing:
#
# May you do good and not evil.
# May you find forgiveness for yourself and forgive others.
# May you share freely, never taking more than you give.
#
#***********************************************************************
#
# This file contains tests for the content= and content_rowid= options.
#
source [file join [file dirname [info script]] fts5_common.tcl]
set testprefix fts5contentless3
# If SQLITE_ENABLE_FTS5 is defined, omit this file.
ifcapable !fts5 {
finish_test
return
}
do_execsql_test 1.0 {
CREATE VIRTUAL TABLE ft USING fts5(x, content=, contentless_delete=1);
BEGIN;
INSERT INTO ft VALUES('one one one');
INSERT INTO ft VALUES('two two two');
INSERT INTO ft VALUES('three three three');
INSERT INTO ft VALUES('four four four');
INSERT INTO ft VALUES('five five five');
INSERT INTO ft VALUES('six six six');
INSERT INTO ft VALUES('seven seven seven');
INSERT INTO ft VALUES('eight eight eight');
INSERT INTO ft VALUES('nine nine nine');
COMMIT;
DELETE FROM ft WHERE rowid=3;
}
proc myhex {hex} { binary decode hex $hex }
db func myhex myhex
do_execsql_test 1.1 {
UPDATE ft_data SET block =
myhex('04000000 00000001' ||
'01020304 01020304 01020304 01020304' ||
'01020304 01020304 01020304 01020304'
)
WHERE id = (SELECT max(id) FROM ft_data);
}
do_execsql_test 1.2 {
DELETE FROM ft WHERE rowid=1
}
do_execsql_test 1.3 {
SELECT rowid FROM ft('two');
} {2}
do_execsql_test 1.3 {
UPDATE ft_data SET block =
myhex('08000000 00000001' ||
'0000000001020304 0000000001020304 0000000001020304 0000000001020304' ||
'0000000001020304 0000000001020304 0000000001020304 0000000001020304'
)
WHERE id = (SELECT max(id) FROM ft_data);
}
do_execsql_test 1.4 {
SELECT rowid FROM ft('two');
} {2}
do_execsql_test 1.5 {
DELETE FROM ft WHERE rowid=4
}
do_execsql_test 1.6 {
UPDATE ft_data SET block = myhex('04000000 00000000')
WHERE id = (SELECT max(id) FROM ft_data);
}
do_execsql_test 1.7 {
SELECT rowid FROM ft('two');
} {2}
do_execsql_test 1.8 {
UPDATE ft_data SET block = myhex('04000000 00000000')
WHERE id = (SELECT max(id) FROM ft_data);
}
do_execsql_test 1.9 {
DELETE FROM ft WHERE rowid=8
} {}
#-------------------------------------------------------------------------
reset_db
do_execsql_test 2.0 {
CREATE VIRTUAL TABLE ft USING fts5(x, content=, contentless_delete=1);
INSERT INTO ft VALUES('one one one');
INSERT INTO ft VALUES('two two two');
INSERT INTO ft VALUES('three three three');
INSERT INTO ft VALUES('four four four');
INSERT INTO ft VALUES('five five five');
INSERT INTO ft VALUES('six six six');
INSERT INTO ft VALUES('seven seven seven');
INSERT INTO ft VALUES('eight eight eight');
INSERT INTO ft VALUES('nine nine nine');
}
do_execsql_test 2.1 {
INSERT INTO ft(ft) VALUES('optimize');
}
do_execsql_test 2.2 {
SELECT count(*) FROM ft_data
} {3}
do_execsql_test 2.3 {
DELETE FROM ft WHERE rowid=5
}
do_execsql_test 2.4 {
SELECT count(*) FROM ft_data
} {4}
# Check that an 'optimize' works (rewrites the index) if there is a single
# segment with one or more tombstone hash pages.
do_execsql_test 2.5 {
INSERT INTO ft(ft) VALUES('optimize');
}
do_execsql_test 2.6 {
SELECT count(*) FROM ft_data
} {3}
# Check that an 'optimize' is a no-op if there is a single segment
# and no tombstone hash pages.
do_execsql_test 2.7 {
INSERT INTO ft(ft) VALUES('optimize');
SELECT rowid FROM ft_data;
} [db eval {SELECT rowid FROM ft_data}]
#-------------------------------------------------------------------------
reset_db
do_execsql_test 3.0 {
CREATE VIRTUAL TABLE ft USING fts5(x, content=, contentless_delete=1);
INSERT INTO ft(ft, rank) VALUES('pgsz', 64);
WITH s(i) AS (
SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<1000
)
INSERT INTO ft(rowid, x) SELECT i, i||' '||i||' '||i||' '||i FROM s;
INSERT INTO ft(ft) VALUES('optimize');
}
do_execsql_test 3.1 {
SELECT count(*) FROM ft_data
} {200}
do_execsql_test 3.2 {
DELETE FROM ft WHERE (rowid % 50)==0;
SELECT count(*) FROM ft_data;
} {203}
do_execsql_test 3.3 {
INSERT INTO ft(ft, rank) VALUES('merge', 500);
SELECT rowid FROM ft_data;
} [db eval {SELECT rowid FROM ft_data}]
do_execsql_test 3.4 {
INSERT INTO ft(ft, rank) VALUES('merge', -1000);
SELECT count(*) FROM ft_data;
} {197}
do_execsql_test 3.5 {
DELETE FROM ft WHERE (rowid % 50)==1;
SELECT count(*) FROM ft_data;
} {200}
do_execsql_test 3.6 {
SELECT level, segment, npgtombstone FROM fts5_structure(
(SELECT block FROM ft_data WHERE id=10)
)
} {1 0 3}
do_test 3.6 {
while 1 {
set nChange [db total_changes]
execsql { INSERT INTO ft(ft, rank) VALUES('merge', -5) }
if {([db total_changes] - $nChange)<2} break
}
} {}
do_execsql_test 3.7 {
SELECT level, segment, npgtombstone FROM fts5_structure(
(SELECT block FROM ft_data WHERE id=10)
)
} {2 0 0}
finish_test
+247
View File
@@ -0,0 +1,247 @@
# 2023 July 21
#
# The author disclaims copyright to this source code. In place of
# a legal notice, here is a blessing:
#
# May you do good and not evil.
# May you find forgiveness for yourself and forgive others.
# May you share freely, never taking more than you give.
#
#***********************************************************************
#
# This file contains tests for the content= and content_rowid= options.
#
source [file join [file dirname [info script]] fts5_common.tcl]
set testprefix fts5contentless4
# If SQLITE_ENABLE_FTS5 is defined, omit this file.
ifcapable !fts5 {
finish_test
return
}
proc document {n} {
set vocab [list A B C D E F G H I J K L M N O P Q R S T U V W X Y Z]
set ret [list]
for {set ii 0} {$ii < $n} {incr ii} {
lappend ret [lindex $vocab [expr int(rand()*[llength $vocab])]]
}
set ret
}
db func document document
do_execsql_test 1.0 {
CREATE VIRTUAL TABLE ft USING fts5(x, content='', contentless_delete=1);
INSERT INTO ft(ft, rank) VALUES('pgsz', 240);
WITH s(i) AS (
SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<1000
)
INSERT INTO ft SELECT document(12) FROM s;
}
do_execsql_test 1.1 {
INSERT INTO ft(ft) VALUES('optimize');
}
do_execsql_test 1.2 {
SELECT level, segment, nentry, nentrytombstone FROM fts5_structure((
SELECT block FROM ft_data WHERE id=10
))
} {0 0 1000 0}
do_execsql_test 1.3 {
DELETE FROM ft WHERE rowid < 50
}
do_execsql_test 1.4 {
SELECT level, segment, nentry, nentrytombstone FROM fts5_structure((
SELECT block FROM ft_data WHERE id=10
))
} {0 0 1000 49}
do_execsql_test 1.5 {
DELETE FROM ft WHERE rowid < 1000
}
do_execsql_test 1.6 {
SELECT level, segment, nentry, nentrytombstone FROM fts5_structure((
SELECT block FROM ft_data WHERE id=10
))
} {1 0 1 0}
#--------------------------------------------------------------------------
reset_db
db func document document
do_execsql_test 2.0 {
CREATE VIRTUAL TABLE ft USING fts5(x, content='', contentless_delete=1);
}
do_test 2.1 {
for {set ii 0} {$ii < 5000} {incr ii} {
execsql { INSERT INTO ft VALUES( document(12) ) }
}
} {}
do_execsql_test 2.2 {
SELECT sum(nentry) - sum(nentrytombstone) FROM fts5_structure((
SELECT block FROM ft_data WHERE id=10
))
} {5000}
for {set ii 5000} {$ii >= 0} {incr ii -100} {
do_execsql_test 2.3.$ii {
DELETE FROM ft WHERE rowid > $ii
}
do_execsql_test 2.3.$ii.2 {
SELECT
CAST((total(nentry) - total(nentrytombstone)) AS integer)
FROM
fts5_structure( (SELECT block FROM ft_data WHERE id=10) )
} $ii
}
execsql_pp {
SELECT * FROM fts5_structure((
SELECT block FROM ft_data WHERE id=10
))
}
do_test 2.4 {
for {set ii 0} {$ii < 5000} {incr ii} {
execsql { INSERT INTO ft VALUES( document(12) ) }
}
} {}
for {set ii 1} {$ii <= 5000} {incr ii 10} {
do_execsql_test 2.3.$ii {
DELETE FROM ft WHERE rowid = $ii;
INSERT INTO ft VALUES( document(12) );
INSERT INTO ft(ft, rank) VALUES('merge', -10);
}
do_execsql_test 2.3.$ii.2 {
SELECT
CAST((total(nentry) - total(nentrytombstone)) AS integer)
FROM
fts5_structure( (SELECT block FROM ft_data WHERE id=10) )
} 5000
}
#-------------------------------------------------------------------------
reset_db
db func document document
do_execsql_test 3.0 {
CREATE VIRTUAL TABLE ft USING fts5(x, content='', contentless_delete=1);
WITH s(i) AS (
SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<100
)
INSERT INTO ft SELECT document(12) FROM s;
}
do_catchsql_test 3.1 {
INSERT INTO ft(ft, rank) VALUES('deletemerge', 'text');
} {1 {SQL logic error}}
do_catchsql_test 3.2 {
INSERT INTO ft(ft, rank) VALUES('deletemerge', 50);
} {0 {}}
do_execsql_test 3.3 {
SELECT * FROM ft_config WHERE k='deletemerge'
} {deletemerge 50}
do_catchsql_test 3.4 {
INSERT INTO ft(ft, rank) VALUES('deletemerge', 101);
} {0 {}}
do_execsql_test 3.5 {
SELECT * FROM ft_config WHERE k='deletemerge'
} {deletemerge 101}
do_execsql_test 3.6 {
DELETE FROM ft WHERE rowid<95
}
do_execsql_test 3.7 {
SELECT nentrytombstone, nentry FROM fts5_structure((
SELECT block FROM ft_data WHERE id=10
))
} {94 100}
do_execsql_test 3.8 {
DELETE FROM ft WHERE rowid=95
}
do_execsql_test 3.9 {
SELECT nentrytombstone, nentry FROM fts5_structure((
SELECT block FROM ft_data WHERE id=10
))
} {95 100}
do_execsql_test 3.10 {
DELETE FROM ft;
WITH s(i) AS (
SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<100
)
INSERT INTO ft SELECT document(12) FROM s;
INSERT INTO ft(ft, rank) VALUES('deletemerge', 50);
}
do_execsql_test 3.11 {
DELETE FROM ft WHERE rowid<95
}
do_execsql_test 3.12 {
SELECT nentrytombstone, nentry FROM fts5_structure((
SELECT block FROM ft_data WHERE id=10
))
} {0 6}
#-------------------------------------------------------------------------
reset_db
db func document document
do_execsql_test 4.0 {
CREATE VIRTUAL TABLE x1 USING fts5(x, content='', contentless_delete=1);
INSERT INTO x1(x1, rank) VALUES('usermerge', 16);
INSERT INTO x1(x1, rank) VALUES('deletemerge', 40);
INSERT INTO x1 VALUES('one');
INSERT INTO x1 VALUES('two');
INSERT INTO x1 VALUES('three');
INSERT INTO x1 VALUES('four');
INSERT INTO x1 VALUES('five');
INSERT INTO x1 VALUES('six');
INSERT INTO x1 VALUES('seven');
INSERT INTO x1 VALUES('eight');
INSERT INTO x1 VALUES('nine');
INSERT INTO x1 VALUES('ten');
}
do_execsql_test 4.1 {
SELECT level, segment FROM fts5_structure((
SELECT block FROM x1_data WHERE id=10
))
} {
0 0 0 1 0 2 0 3 0 4 0 5 0 6 0 7 0 8 0 9
}
for {set ii 1} {$ii < 4} {incr ii} {
do_execsql_test 4.2.$ii {
DELETE FROM x1 WHERE rowid = $ii;
INSERT INTO x1(x1, rank) VALUES('merge', 5);
SELECT level, segment FROM fts5_structure((
SELECT block FROM x1_data WHERE id=10
))
} {
0 0 0 1 0 2 0 3 0 4 0 5 0 6 0 7 0 8 0 9
}
}
do_execsql_test 4.3 {
DELETE FROM x1 WHERE rowid = $ii;
INSERT INTO x1(x1, rank) VALUES('merge', 5);
SELECT level, segment, nentry FROM fts5_structure((
SELECT block FROM x1_data WHERE id=10
))
} {
1 0 6
}
finish_test
+58
View File
@@ -0,0 +1,58 @@
# 2023 August 7
#
# The author disclaims copyright to this source code. In place of
# a legal notice, here is a blessing:
#
# May you do good and not evil.
# May you find forgiveness for yourself and forgive others.
# May you share freely, never taking more than you give.
#
#***********************************************************************
#
# This file contains tests for the content= and content_rowid= options.
#
source [file join [file dirname [info script]] fts5_common.tcl]
set testprefix fts5contentless5
# If SQLITE_ENABLE_FTS5 is defined, omit this file.
ifcapable !fts5 {
finish_test
return
}
do_execsql_test 1.0 {
CREATE VIRTUAL TABLE t1 USING fts5(a, b, c, content='', contentless_delete=1);
INSERT INTO t1 VALUES('A', 'B', 'C');
INSERT INTO t1 VALUES('D', 'E', 'F');
INSERT INTO t1 VALUES('G', 'H', 'I');
}
do_execsql_test 1.01 {
CREATE TABLE t2(x, y);
INSERT INTO t2 VALUES('x', 'y');
}
# explain_i "UPDATE t1 SET a='a' WHERE t1.rowid=1"
breakpoint
explain_i "UPDATE t1 SET a='a' FROM t2 WHERE t1.rowid=1 AND b IS NULL"
#breakpoint
#explain_i "UPDATE t1 SET a='a' WHERE b IS NULL AND rowid=?"
foreach {tn up err} {
1 "UPDATE t1 SET a='a', b='b', c='c' WHERE rowid=1" 0
2 "UPDATE t1 SET a='a', b='b' WHERE rowid=1" 1
3 "UPDATE t1 SET b='b', c='c' WHERE rowid=1" 1
4 "UPDATE t1 SET a='a', c='c' WHERE rowid=1" 1
5 "UPDATE t1 SET a='a', c='c' WHERE t1.rowid=1 AND b IS NULL" 1
6 "UPDATE t1 SET a='a' FROM t2 WHERE t1.rowid=1" 1
7 "UPDATE t1 SET a='a', b='b', c='c' FROM t2 WHERE t1.rowid=1" 0
} {
set res(0) {0 {}}
set res(1) {1 {cannot UPDATE a subset of columns on fts5 contentless-delete table: t1}}
do_catchsql_test 1.$tn $up $res($err)
}
finish_test
+4
View File
@@ -48,6 +48,10 @@ do_test 1.3 {
}
catchsql { INSERT INTO t1(t1) VALUES('integrity-check') }
} {1 {database disk image is malformed}}
do_execsql_test 1.3b {
PRAGMA integrity_check(t1);
} {{malformed inverted index for FTS5 table main.t1}}
do_test 1.4 {
db_restore_and_reopen
+3
View File
@@ -167,6 +167,9 @@ foreach {tn hdr} {
do_test 3.$tn.$tn2.2 {
catchsql { INSERT INTO x3(x3) VALUES('integrity-check') }
} {1 {database disk image is malformed}}
do_execsql_test 3.$tn.$tn2.3 {
PRAGMA integrity_check(x3);
} {{malformed inverted index for FTS5 table main.x3}}
}
execsql ROLLBACK
+85
View File
@@ -880,6 +880,91 @@ do_catchsql_test 5.4 {
UPDATE t1 SET content=randomblob(500);
} {1 {database disk image is malformed}}
#-------------------------------------------------------------------------
reset_db
do_test 6.0 {
sqlite3 db {}
db deserialize [decode_hexdb {
.open --hexdb
| size 32768 pagesize 4096 filename crash-42fa37b694d45a.db
| page 1 offset 0
| 0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00 SQLite format 3.
| 16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 07 .....@ ........
| 96: 00 00 00 00 0d 00 00 00 07 0d d2 00 0f c4 0f 6d ...............m
| 112: 0f 02 0e ab 0e 4e 0d f6 0d d2 00 00 00 00 00 00 .....N..........
| 3536: 00 00 22 07 06 17 11 11 01 31 74 61 62 6c 65 74 .........1tablet
| 3552: 32 74 32 07 43 52 45 41 54 45 20 54 41 42 4c 45 2t2.CREATE TABLE
| 3568: 20 74 32 28 78 29 56 06 06 17 1f 1f 01 7d 74 61 t2(x)V.......ta
| 3584: 62 6c 65 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63 blet1_configt1_c
| 3600: 6f 6e 66 69 67 06 43 52 45 41 54 45 20 54 41 42 onfig.CREATE TAB
| 3616: 4c 45 20 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b LE 't1_config'(k
| 3632: 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 76 29 PRIMARY KEY, v)
| 3648: 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 5b 05 WITHOUT ROWID[.
| 3664: 07 17 21 21 01 81 01 74 61 62 6c 65 74 31 5f 64 ..!!...tablet1_d
| 3680: 6f 63 73 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65 ocsizet1_docsize
| 3696: 05 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74 .CREATE TABLE 't
| 3712: 31 5f 64 6f 63 73 69 7a 65 27 28 69 64 20 49 4e 1_docsize'(id IN
| 3728: 54 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45 TEGER PRIMARY KE
| 3744: 59 2c 20 73 7a 20 42 4c 4f 42 29 55 04 06 17 21 Y, sz BLOB)U...!
| 3760: 21 01 77 74 61 62 6c 65 74 31 5f 63 6f 6e 74 65 !.wtablet1_conte
| 3776: 6e 74 74 31 5f 63 6f 6e 74 65 6e 74 04 43 52 45 ntt1_content.CRE
| 3792: 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63 6f ATE TABLE 't1_co
| 3808: 6e 74 65 6e 74 27 28 69 64 20 49 4e 54 45 47 45 ntent'(id INTEGE
| 3824: 52 20 50 52 49 4d 41 52 49 20 4b 45 59 2c 20 63 R PRIMARI KEY, c
| 3840: 30 29 69 03 07 17 19 19 01 81 2d 74 61 62 6c 65 0)i.......-table
| 3856: 74 31 5f 69 64 78 74 31 5f 69 64 78 03 43 52 45 t1_idxt1_idx.CRE
| 3872: 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 69 64 ATE TABLE 't1_id
| 3888: 78 27 28 73 65 67 69 64 2c 20 74 65 72 6d 2c 20 x'(segid, term,
| 3904: 70 67 6e 6f 2c 20 50 52 49 4d 41 52 59 20 4b 45 pgno, PRIMARY KE
| 3920: 59 28 73 65 67 69 64 2c 20 74 65 72 6d 29 29 20 Y(segid, term))
| 3936: 57 49 54 48 4f 55 54 20 52 4f 57 49 44 55 02 07 WITHOUT ROWIDU..
| 3952: 17 1b 1b 01 81 01 74 61 62 6c 65 74 31 5f 64 61 ......tablet1_da
| 3968: 74 61 74 31 5f 64 61 74 61 02 43 52 45 41 54 45 tat1_data.CREATE
| 3984: 20 54 41 42 4c 45 20 27 74 31 5f 64 61 74 61 27 TABLE 't1_data'
| 4000: 28 69 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d (id INTEGER PRIM
| 4016: 41 52 b9 20 4b 45 59 2c 20 62 6c 6f 63 6b 20 42 AR. KEY, block B
| 4032: 4c 4f 42 29 3a 01 06 17 11 11 08 63 74 61 62 6c LOB):......ctabl
| 4048: 65 74 31 74 31 43 52 45 41 54 45 20 56 49 52 54 et1t1CREATE VIRT
| 4064: 55 41 4c 20 54 41 42 4c 45 20 74 31 20 55 53 49 UAL TABLE t1 USI
| 4080: 4e 47 20 66 74 73 35 28 63 6f 6e 74 65 6e 74 29 NG fts5(content)
| page 2 offset 4096
| 0: 0d 00 00 00 03 0f bd 00 0f e8 0f ef 0f bd f0 00 ................
| 16: 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
| 4016: 00 00 00 00 00 00 00 00 00 00 00 00 00 24 84 80 .............$..
| 4032: 80 80 80 01 03 00 4e 00 10 00 1e 06 30 61 62 61 ......N.....0aba
| 4048: 63 6c 01 02 02 04 02 66 74 02 5f 02 04 04 6e 64 cl.....ft._...nd
| 4064: 6f 6e 02 02 02 04 0a 07 05 01 03 00 10 03 03 0f on..............
| 4080: 0a 03 00 24 00 00 00 00 01 01 01 00 01 01 01 11 ...$............
| page 3 offset 8192
| 0: 0a 00 00 00 01 0f 00 01 00 00 00 00 00 00 00 00 ................
| 4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0c 01 02 ................
| page 4 offset 12288
| 0: 0d 00 00 00 03 0f e0 00 0f f6 0f ec 0f e0 00 00 ................
| 4064: 0a 03 03 00 1b 61 62 61 6e 64 6f 6e 08 02 03 00 .....abandon....
| 4080: 17 61 62 61 66 74 08 01 03 00 17 61 62 61 63 6b .abaft.....aback
| page 5 offset 16384
| 0: 0d 00 00 00 03 0f ee 00 0f fa 0f 00 00 00 00 00 ................
| 4064: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 04 03 ................
| 4080: 03 00 0e 01 04 02 03 00 0e 01 04 01 03 00 0e 01 ................
| page 6 offset 20480
| 0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00 ................
| 4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04 ........version.
| page 7 offset 24576
| 0: 0d 00 00 10 03 0f d6 00 0f 00 00 00 00 00 00 00 ................
| 4048: 00 00 00 00 00 00 09 03 02 1b 72 65 62 75 69 6c ..........rebuil
| 4064: 64 11 02 02 2b 69 6e 74 65 67 72 69 74 79 2d 63 d...+integrity-c
| 4080: 68 65 63 6b 0a 01 02 1d 6f 70 74 69 6d 00 00 00 heck....optim...
| page 8 offset 28672
| 0: 00 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
| end crash-42fa37b694d45a.db
}]} {}
do_execsql_test 6.1 {
INSERT INTO t1(t1,rank) VALUES('secure-delete',1);
}
do_catchsql_test 6.2 {
UPDATE t1 SET content=randomblob(500) WHERE t1;
} {1 {constraint failed}}
sqlite3_fts5_may_be_corrupt 0
finish_test
+3
View File
@@ -95,6 +95,9 @@ do_execsql_test 3.3 {
SELECT rowid, bm25(e1) FROM e1 WHERE e1 MATCH '"/" OR "just"' ORDER BY rank;
} {1 -1e-06}
do_execsql_test 3.4 "
SELECT fts5_expr_tcl('e AND \" \"');
" {{AND [nearset -- {e}] [{}]}}
finish_test
+111
View File
@@ -0,0 +1,111 @@
# 2023 July 20
#
# The author disclaims copyright to this source code. In place of
# a legal notice, here is a blessing:
#
# May you do good and not evil.
# May you find forgiveness for yourself and forgive others.
# May you share freely, never taking more than you give.
#
#*************************************************************************
#
# This file is focused on OOM errors. Particularly those that may occur
# when using contentless_delete=1 databases.
#
source [file join [file dirname [info script]] fts5_common.tcl]
source $testdir/malloc_common.tcl
set testprefix fts5faultF
# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
ifcapable !fts5 {
finish_test
return
}
faultsim_save_and_close
do_faultsim_test 1 -prep {
faultsim_restore_and_reopen
} -body {
execsql {
CREATE VIRTUAL TABLE t1 USING fts5(x, y, content=, contentless_delete=1)
}
} -test {
faultsim_test_result {0 {}} {1 {vtable constructor failed: t1}}
}
reset_db
do_execsql_test 2.0 {
CREATE VIRTUAL TABLE t1 USING fts5(doc, content=, contentless_delete=1);
BEGIN;
INSERT INTO t1(rowid, doc) VALUES(1, 'a b c d');
INSERT INTO t1(rowid, doc) VALUES(2, 'a b c d');
INSERT INTO t1(rowid, doc) VALUES(3, 'a b c d');
INSERT INTO t1(rowid, doc) VALUES(4, 'a b c d');
COMMIT;
DELETE FROM t1 WHERE rowid IN (2, 4);
}
do_faultsim_test 2 -prep {
sqlite3 db test.db
execsql { SELECT rowid FROM t1 }
} -body {
execsql {
SELECT rowid FROM t1('b c');
}
} -test {
faultsim_test_result {0 {1 3}}
}
#-------------------------------------------------------------------------
reset_db
do_execsql_test 3.0 {
CREATE VIRTUAL TABLE t1 USING fts5(doc, content=, contentless_delete=1);
BEGIN;
INSERT INTO t1(rowid, doc) VALUES(1, 'a b c d');
INSERT INTO t1(rowid, doc) VALUES(2, 'a b c d');
INSERT INTO t1(rowid, doc) VALUES(3, 'a b c d');
INSERT INTO t1(rowid, doc) VALUES(4, 'a b c d');
COMMIT;
}
faultsim_save_and_close
do_faultsim_test 3 -prep {
faultsim_restore_and_reopen
execsql { SELECT rowid FROM t1 }
} -body {
execsql {
INSERT INTO t1(rowid, doc) VALUES(5, 'a b c d');
}
} -test {
faultsim_test_result {0 {}}
}
#-------------------------------------------------------------------------
reset_db
do_execsql_test 4.0 {
CREATE VIRTUAL TABLE t1 USING fts5(doc, content=, contentless_delete=1);
INSERT INTO t1(t1, rank) VALUES('pgsz', 64);
WITH s(i) AS (
SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<1000
)
INSERT INTO t1(rowid, doc) SELECT i, 'a b c d' FROM s;
}
do_execsql_test 4.1 { DELETE FROM t1 WHERE rowid <= 25 }
faultsim_save_and_close
do_faultsim_test 4 -faults oom-t* -prep {
faultsim_restore_and_reopen
execsql { SELECT rowid FROM t1 }
} -body {
execsql {
DELETE FROM t1 WHERE rowid < 100
}
} -test {
faultsim_test_result {0 {}}
}
finish_test
+76
View File
@@ -0,0 +1,76 @@
# 2010 June 15
#
# The author disclaims copyright to this source code. In place of
# a legal notice, here is a blessing:
#
# May you do good and not evil.
# May you find forgiveness for yourself and forgive others.
# May you share freely, never taking more than you give.
#
#***********************************************************************
#
source [file join [file dirname [info script]] fts5_common.tcl]
source $testdir/malloc_common.tcl
set testprefix fts5faultG
# If SQLITE_ENABLE_FTS5 is defined, omit this file.
ifcapable !fts5 {
finish_test
return
}
set ::testprefix fts5faultG
do_execsql_test 1.0 {
CREATE VIRTUAL TABLE t1 USING fts5(a);
INSERT INTO t1 VALUES('test renaming the table');
INSERT INTO t1 VALUES(' after it has been written');
INSERT INTO t1 VALUES(' actually other stuff instead');
}
faultsim_save_and_close
do_faultsim_test 1 -faults oom* -prep {
faultsim_restore_and_reopen
execsql {
BEGIN;
DELETE FROM t1 WHERE rowid=2;
}
} -body {
execsql {
DELETE FROM t1;
}
} -test {
catchsql { COMMIT }
faultsim_integrity_check
faultsim_test_result {0 {}}
}
reset_db
do_execsql_test 2.0 {
CREATE VIRTUAL TABLE t1 USING fts5(a, content=, contentless_delete=1);
BEGIN;
INSERT INTO t1 VALUES('here''s some text');
INSERT INTO t1 VALUES('useful stuff, text');
INSERT INTO t1 VALUES('what would we do without text!');
COMMIT;
}
faultsim_save_and_close
do_faultsim_test 2 -faults oom* -prep {
faultsim_restore_and_reopen
execsql {
BEGIN;
DELETE FROM t1 WHERE rowid=2;
}
} -body {
execsql {
INSERT INTO t1(t1) VALUES('optimize');
}
} -test {
faultsim_integrity_check
faultsim_test_result {0 {}}
}
finish_test
+38
View File
@@ -77,6 +77,9 @@ do_catchsql_test 4.2 {
UPDATE aa_docsize SET sz = X'44' WHERE rowid = 3;
INSERT INTO aa(aa) VALUES('integrity-check');
} {1 {database disk image is malformed}}
do_execsql_test 4.2.1 {
PRAGMA integrity_check(aa);
} {{malformed inverted index for FTS5 table main.aa}}
do_catchsql_test 4.3 {
ROLLBACK;
@@ -317,4 +320,39 @@ do_catchsql_test 10.5.3 {
INSERT INTO vt0(vt0) VALUES('integrity-check');
} {0 {}}
reset_db
proc slang {in} {return [string map {th d e eh} $in]}
db function slang -deterministic -innocuous slang
do_execsql_test 11.0 {
CREATE TABLE t1(a INTEGER PRIMARY KEY, b TEXT, c TEXT AS (slang(b)));
INSERT INTO t1(b) VALUES('the quick fox jumps over the lazy brown dog');
SELECT c FROM t1;
} {{deh quick fox jumps ovehr deh lazy brown dog}}
do_execsql_test 11.1 {
CREATE VIRTUAL TABLE t2 USING fts5(content="t1", c);
INSERT INTO t2(t2) VALUES('rebuild');
SELECT rowid FROM t2 WHERE t2 MATCH 'deh';
} {1}
do_execsql_test 11.2 {
PRAGMA integrity_check(t2);
} {ok}
db close
sqlite3 db test.db
# FIX ME?
#
# FTS5 integrity-check does not care if the content table is unreadable or
# does not exist. It only looks for internal inconsistencies in the
# inverted index.
#
do_execsql_test 11.3 {
PRAGMA integrity_check(t2);
} {ok}
do_execsql_test 11.4 {
DROP TABLE t1;
PRAGMA integrity_check(t2);
} {ok}
finish_test
+7 -3
View File
@@ -44,12 +44,12 @@ do_catchsql_test 1.2.2 {
do_catchsql_test 1.3.1 {
SELECT highlight(t1, 4, '<b>', '</b>') FROM t1('*reads');
} {1 {no such cursor: 1}}
} {1 {no such cursor: 2}}
do_catchsql_test 1.3.2 {
SELECT a FROM t1
WHERE rank = (SELECT highlight(t1, 4, '<b>', '</b>') FROM t1('*reads'));
} {1 {no such cursor: 1}}
} {1 {no such cursor: 2}}
db close
sqlite3 db test.db
@@ -424,10 +424,12 @@ do_execsql_test -db db2 15.3 {
SAVEPOINT one;
} {}
do_execsql_test 15.4 END
do_test 15.4 {
do_test 15.5 {
list [catch { db2 eval COMMIT } msg] $msg
} {0 {}}
db2 close
#-------------------------------------------------------------------------
reset_db
forcedelete test.db2
@@ -469,6 +471,8 @@ do_execsql_test -db db2 16.6 {
SELECT * FROM x1
} {abc def}
db2 close
#-------------------------------------------------------------------------
reset_db
do_execsql_test 17.1 {
+1 -20
View File
@@ -9,7 +9,7 @@
#
#***********************************************************************
#
# TESTRUNNER: slow
# TESTRUNNER: superslow
#
source [file join [file dirname [info script]] fts5_common.tcl]
@@ -42,23 +42,4 @@ do_execsql_test 1.2 {
SELECT count(*) FROM t1('mno')
} $nLoop
do_execsql_test 2.0 {
CREATE VIRTUAL TABLE t2 USING fts5(x);
INSERT INTO t2(t2, rank) VALUES('pgsz', 32);
}
do_test 2.1 {
for {set ii 0} {$ii < $nLoop} {incr ii} {
execsql {
INSERT INTO t2 VALUES('abc def ghi');
INSERT INTO t2 VALUES('jkl mno pqr');
INSERT INTO t2(t2, rank) VALUES('merge', -1);
}
}
} {}
do_execsql_test 2.2 {
SELECT count(*) FROM t2('mno')
} $nLoop
finish_test
+45
View File
@@ -0,0 +1,45 @@
# 2023 Aug 27
#
# The author disclaims copyright to this source code. In place of
# a legal notice, here is a blessing:
#
# May you do good and not evil.
# May you find forgiveness for yourself and forgive others.
# May you share freely, never taking more than you give.
#
#***********************************************************************
#
# TESTRUNNER: superslow
#
source [file join [file dirname [info script]] fts5_common.tcl]
set testprefix fts5optimize2
# If SQLITE_ENABLE_FTS5 is defined, omit this file.
ifcapable !fts5 {
finish_test
return
}
set nLoop 2500
do_execsql_test 1.0 {
CREATE VIRTUAL TABLE t2 USING fts5(x);
INSERT INTO t2(t2, rank) VALUES('pgsz', 32);
}
do_test 1.1 {
for {set ii 0} {$ii < $nLoop} {incr ii} {
execsql {
INSERT INTO t2 VALUES('abc def ghi');
INSERT INTO t2 VALUES('jkl mno pqr');
INSERT INTO t2(t2, rank) VALUES('merge', -1);
}
}
} {}
do_execsql_test 1.2 {
SELECT count(*) FROM t2('mno')
} $nLoop
finish_test
+30
View File
@@ -52,6 +52,36 @@ do_execsql_test 2.1 {
SELECT * FROM t2('to*');
} {top to tommy}
#-------------------------------------------------------------------------
foreach {tn newrowid} {
1 122
2 123
3 124
} {
reset_db
do_execsql_test 3.$tn.0 {
CREATE VIRTUAL TABLE t12 USING fts5(x);
INSERT INTO t12(rowid, x) VALUES(123, 'wwww');
}
do_execsql_test 3.$tn.1 {
BEGIN;
DELETE FROM t12 WHERE rowid=123;
SELECT * FROM t12('wwww*');
INSERT INTO t12(rowid, x) VALUES($newrowid, 'wwww');
SELECT * FROM t12('wwww*');
END;
} {wwww}
do_execsql_test 3.$tn.2 {
INSERT INTO t12(t12) VALUES('integrity-check');
}
do_execsql_test 3.$tn.3 {
SELECT rowid FROM t12('wwww*');
} $newrowid
}
finish_test
finish_test
+24
View File
@@ -180,4 +180,28 @@ do_execsql_test 6.1 {
{table table table} {the table names.} {rank on an fts5 table}
}
#-------------------------------------------------------------------------
# forum post: https://sqlite.org/forum/forumpost/a2dd636330
#
reset_db
do_execsql_test 1.0 {
CREATE VIRTUAL TABLE t USING fts5 (a, b);
INSERT INTO t (a, b) VALUES ('data1', 'sentence1'), ('data2', 'sentence2');
INSERT INTO t(t, rank) VALUES ('rank', 'bm25(10.0,1.0)');
}
sqlite3 db2 test.db
do_execsql_test -db db2 1.1 {
SELECT *, rank<0.0 FROM t('data*') ORDER BY RANK;
} {data1 sentence1 1 data2 sentence2 1}
do_execsql_test 1.2 {
INSERT INTO t(t, rank) VALUES ('rank', 'bm25(10.0,1.0)');
}
do_execsql_test -db db2 1.3 {
SELECT *, rank<0.0 FROM t('data*') ORDER BY RANK;
} {data1 sentence1 1 data2 sentence2 1}
db2 close
finish_test
+1 -1
View File
@@ -71,7 +71,7 @@ ifcapable fts3 {
do_catchsql_test 3.2 {
DROP TABLE vt1;
} {1 {SQL logic error}}
} {0 {}}
do_execsql_test 3.3 {
SAVEPOINT x;
+70
View File
@@ -273,6 +273,76 @@ do_execsql_test 5.3 {
do_execsql_test 5.4 { SELECT rowid FROM t1('abc'); } 2
do_execsql_test 5.5 { SELECT rowid FROM t1('aa'); } 2
#-------------------------------------------------------------------------
# Tests for the bug fixed by https://sqlite.org/src/info/4b60a1c3
#
reset_db
do_execsql_test 6.0 {
CREATE VIRTUAL TABLE fts USING fts5(content);
INSERT INTO fts(fts, rank) VALUES ('secure-delete', 1);
INSERT INTO fts(rowid, content) VALUES
(3407, 'profile profile profile profile profile profile profile profile pull pulling pulling really');
DELETE FROM fts WHERE rowid IS 3407;
INSERT INTO fts(fts) VALUES ('integrity-check');
}
foreach {tn detail} {
1 full
2 column
3 none
} {
do_execsql_test 6.1.$detail "
DROP TABLE IF EXISTS t1;
CREATE VIRTUAL TABLE t1 USING fts5(x, detail=$detail);
"
do_execsql_test 6.2.$detail {
INSERT INTO t1(t1, rank) VALUES('secure-delete', 1);
}
for {set ii 1} {$ii < 100} {incr ii} {
do_execsql_test 6.3.$detail.$ii.1 {
BEGIN;
INSERT INTO t1(rowid, x) VALUES(10, 'word1');
WITH s(i) AS (
SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<CAST($ii AS integer)
)
INSERT INTO t1(x) SELECT 'word3' FROM s;
COMMIT;
INSERT INTO t1(t1) VALUES('optimize');
}
do_execsql_test 6.3.$detail.$ii.2 {
DELETE FROM t1 WHERE rowid=10;
INSERT INTO t1(t1) VALUES ('integrity-check');
}
do_execsql_test 6.3.$detail.$ii.3 {
DELETE FROM t1;
}
do_execsql_test 6.3.$detail.$ii.4 {
BEGIN;
INSERT INTO t1(rowid, x) VALUES(10, 'tokenA');
WITH s(i) AS (
SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<CAST($ii AS integer)
)
INSERT INTO t1(x) SELECT group_concat('tokenB ') FROM s;
COMMIT;
INSERT INTO t1(t1) VALUES('optimize');
}
do_execsql_test 6.3.$detail.$ii.5 {
DELETE FROM t1 WHERE rowid=10;
INSERT INTO t1(t1) VALUES ('integrity-check');
}
do_execsql_test 6.3.$detail.$ii.6 {
DELETE FROM t1;
}
}
}
finish_test
+87 -1
View File
@@ -18,7 +18,7 @@ db progress 1 progress_handler
set ::PHC 0
proc progress_handler {args} {
incr ::PHC
if {($::PHC % 100000)==0} breakpoint
# if {($::PHC % 100000)==0} breakpoint
return 0
}
@@ -50,6 +50,92 @@ do_test 1.3 {
expr $phc(1)*5 < $phc(2)
} {1}
#-------------------------------------------------------------------------
reset_db
do_execsql_test 2.0 {
CREATE VIRTUAL TABLE t1 USING fts5(x);
INSERT INTO t1(t1, rank) VALUES('secure-delete', $sd)
}
do_execsql_test 2.1 {
BEGIN;
INSERT INTO t1(rowid, x) VALUES(-100000, 'abc def ghi');
INSERT INTO t1(rowid, x) VALUES(-99999, 'abc def ghi');
INSERT INTO t1(rowid, x) VALUES(9223372036854775800, 'abc def ghi');
COMMIT;
}
do_execsql_test 2.2 {
SELECT rowid FROM t1('def')
} {-100000 -99999 9223372036854775800}
#-------------------------------------------------------------------------
reset_db
do_execsql_test 3.0 {
CREATE VIRTUAL TABLE t1 USING fts5(x);
INSERT INTO t1(t1, rank) VALUES('secure-delete', $sd)
}
do_execsql_test 3.1 {
BEGIN;
INSERT INTO t1(rowid, x)
VALUES(51869, 'when whenever where weress what turn'),
(51871, 'to were');
COMMIT;
}
do_execsql_test 3.2 {
DELETE FROM t1 WHERE rowid=51871;
INSERT INTO t1(t1) VALUES('integrity-check');
}
#-------------------------------------------------------------------------
reset_db
do_execsql_test 4.0 {
CREATE VIRTUAL TABLE t1 USING fts5(x);
INSERT INTO t1(rowid, x) VALUES(10, 'one two');
}
do_execsql_test 4.1 {
UPDATE t1 SET x = 'one three' WHERE rowid=10;
INSERT INTO t1(t1, rank) VALUES('secure-delete', 1);
}
do_execsql_test 4.2 {
DELETE FROM t1 WHERE rowid=10;
}
do_execsql_test 4.3 {
INSERT INTO t1(t1) VALUES('integrity-check');
}
#-------------------------------------------------------------------------
reset_db
do_execsql_test 5.0 {
CREATE VIRTUAL TABLE t1 USING fts5(content);
INSERT INTO t1(t1,rank) VALUES('secure-delete',1);
INSERT INTO t1 VALUES('active'),('boomer'),('atom'),('atomic'),
('alpha channel backup abandon test aback boomer atom alpha active');
DELETE FROM t1 WHERE t1 MATCH 'abandon';
}
do_execsql_test 5.1 {
INSERT INTO t1(t1) VALUES('rebuild');
}
do_execsql_test 5.2 {
DELETE FROM t1 WHERE rowid NOTNULL<5;
}
db close
sqlite3 db test.db
do_execsql_test 5.3 {
PRAGMA integrity_check;
} {ok}
finish_test
+116
View File
@@ -0,0 +1,116 @@
# 2023 Feb 17
#
# The author disclaims copyright to this source code. In place of
# a legal notice, here is a blessing:
#
# May you do good and not evil.
# May you find forgiveness for yourself and forgive others.
# May you share freely, never taking more than you give.
#
#*************************************************************************
#
# TESTRUNNER: slow
#
source [file join [file dirname [info script]] fts5_common.tcl]
ifcapable !fts5 { finish_test ; return }
set ::testprefix fts5secure7
set NVOCAB 500
set NDOC [expr 1000]
set NREP 100
set nDeletePerRep [expr 5]
set VOCAB [list]
proc select_one {list} {
set n [llength $list]
lindex $list [expr {abs(int(rand()*$n))}]
}
proc init_vocab {} {
set L [split "abcdefghijklmnopqrstuvwxyz" {}]
set nL [llength $L]
for {set i 0} {$i < $::NVOCAB} {incr i} {
set n [expr {6 + int(rand()*8)}]
set word ""
for {set j 0} {$j < $n} {incr j} {
append word [select_one $L]
}
lappend ::VOCAB $word
}
}
proc get_word {} {
select_one $::VOCAB
}
proc get_document {nWord} {
set ret [list]
for {set i 0} {$i < $nWord} {incr i} {
lappend ret [get_word]
}
return $ret
}
init_vocab
db func document [list get_document 12]
do_execsql_test 1.0 {
CREATE VIRTUAL TABLE t1 USING fts5(body);
INSERT INTO t1(t1, rank) VALUES('secure-delete', 1);
}
do_execsql_test 1.1 {
WITH s(i) AS (
SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<$NDOC
)
INSERT INTO t1 SELECT document() FROM s;
}
for {set iRep 0} {$iRep < $NREP} {incr iRep} {
set lRowid [db eval {SELECT rowid FROM t1}]
for {set iDel 0} {$iDel < $nDeletePerRep} {incr iDel} {
set idx [select_one $lRowid]
db eval {
DELETE FROM t1 WHERE rowid=$idx
}
}
db eval {
WITH s(i) AS (
SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<$nDeletePerRep
)
INSERT INTO t1 SELECT document() FROM s;
}
do_execsql_test 1.2.$iRep {
INSERT INTO t1(t1) VALUES('integrity-check');
}
}
reset_db
db func document [list get_document 12]
do_execsql_test 2.0 {
CREATE VIRTUAL TABLE t1 USING fts5(body);
INSERT INTO t1(t1, rank) VALUES('secure-delete', 1);
INSERT INTO t1(t1, rank) VALUES('pgsz', 128);
}
do_execsql_test 2.1 {
WITH s(i) AS (
SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<$NDOC
)
INSERT INTO t1 SELECT document() FROM s;
}
for {set ii 0} {$ii < $NDOC} {incr ii} {
set lRowid [db eval {SELECT rowid FROM t1}]
set idx [select_one $lRowid]
db eval { DELETE FROM t1 WHERE rowid=$idx }
do_execsql_test 2.2.$ii {
INSERT INTO t1(t1) VALUES('integrity-check');
}
}
finish_test
+51
View File
@@ -0,0 +1,51 @@
# 2023 Nov 23
#
# The author disclaims copyright to this source code. In place of
# a legal notice, here is a blessing:
#
# May you do good and not evil.
# May you find forgiveness for yourself and forgive others.
# May you share freely, never taking more than you give.
#
#*************************************************************************
#
source [file join [file dirname [info script]] fts5_common.tcl]
ifcapable !fts5 { finish_test ; return }
set ::testprefix fts5secure8
proc sql_repeat {txt n} {
string repeat $txt $n
}
db func repeat sql_repeat
do_execsql_test 1.0 {
CREATE VIRTUAL TABLE ft USING fts5(x);
INSERT INTO ft(ft, rank) VALUES('pgsz', 64);
INSERT INTO ft(rowid, x) VALUES(100, 'hello world');
INSERT INTO ft(rowid, x) VALUES(200, 'one day');
BEGIN;
INSERT INTO ft(rowid, x) VALUES(45, 'one two three');
UPDATE ft SET x = repeat('hello world ', 500) WHERE rowid=100;
COMMIT
}
do_execsql_test 1.1 {
INSERT INTO ft(ft, rank) VALUES('secure-delete', 1);
DELETE FROM ft WHERE rowid=100;
}
do_execsql_test 1.2 {
PRAGMA integrity_check;
} {ok}
finish_test
+3
View File
@@ -122,6 +122,9 @@ foreach {tn expr} {
4.1 "NEAR(one two, 2)"
4.2 "NEAR(one two three, 2)"
4.3 "NEAR(eight nine, 1) OR NEAR(six seven, 1)"
5.1 "one + two"
5.2 "1 + two"
} {
if {[fts5_expr_ok $expr ss]==0} {
do_test 1.$tok.$tn.OMITTED { list } [list]
+89
View File
@@ -0,0 +1,89 @@
# 2023 Nov 03
#
# The author disclaims copyright to this source code. In place of
# a legal notice, here is a blessing:
#
# May you do good and not evil.
# May you find forgiveness for yourself and forgive others.
# May you share freely, never taking more than you give.
#
#***********************************************************************
#
# Tests focusing on the built-in fts5 tokenizers.
#
source [file join [file dirname [info script]] fts5_common.tcl]
set testprefix fts5tokenizer2
# If SQLITE_ENABLE_FTS5 is defined, omit this file.
ifcapable !fts5 {
finish_test
return
}
sqlite3_fts5_create_tokenizer db tst get_tst_tokenizer
proc get_tst_tokenizer {args} {
return "tst_tokenizer"
}
proc tst_tokenizer {flags txt} {
set token ""
set lTok [list]
foreach c [split $txt {}] {
if {$token==""} {
append token $c
} else {
set t1 [string is upper $token]
set t2 [string is upper $c]
if {$t1!=$t2} {
lappend lTok $token
set token ""
}
append token $c
}
}
if {$token!=""} { lappend lTok $token }
set iOff 0
foreach t $lTok {
set n [string length $t]
sqlite3_fts5_token $t $iOff [expr $iOff+$n]
incr iOff $n
}
}
do_execsql_test 1.0 {
CREATE VIRTUAL TABLE t1 USING fts5(t, tokenize=tst);
}
do_execsql_test 1.1 {
INSERT INTO t1 VALUES('AAdontBBmess');
}
do_execsql_test 1.2 {
SELECT snippet(t1, 0, '>', '<', '...', 4) FROM t1('BB');
} {AAdont>BB<mess}
do_execsql_test 1.3 {
SELECT highlight(t1, 0, '>', '<') FROM t1('BB');
} {AAdont>BB<mess}
do_execsql_test 1.4 {
SELECT highlight(t1, 0, '>', '<') FROM t1('AA');
} {>AA<dontBBmess}
do_execsql_test 1.5 {
SELECT highlight(t1, 0, '>', '<') FROM t1('dont');
} {AA>dont<BBmess}
do_execsql_test 1.6 {
SELECT highlight(t1, 0, '>', '<') FROM t1('mess');
} {AAdontBB>mess<}
do_execsql_test 1.7 {
SELECT highlight(t1, 0, '>', '<') FROM t1('BB mess');
} {AAdont>BBmess<}
finish_test
+38
View File
@@ -215,4 +215,42 @@ do_execsql_test 7.2 {
SELECT rowid FROM f WHERE filename GLOB '*ир*';
} {20}
#-------------------------------------------------------------------------
reset_db
do_execsql_test 8.0 {
CREATE VIRTUAL TABLE t1 USING fts5(y, tokenize=trigram);
INSERT INTO t1 VALUES('abcdefghijklm');
}
foreach {tn match res} {
1 "abc ghi" "(abc)def(ghi)jklm"
2 "def ghi" "abc(defghi)jklm"
3 "efg ghi" "abcd(efghi)jklm"
4 "efghi" "abcd(efghi)jklm"
5 "abcd jklm" "(abcd)efghi(jklm)"
6 "ijkl jklm" "abcdefgh(ijklm)"
7 "ijk ijkl hijk" "abcdefg(hijkl)m"
} {
do_execsql_test 8.1.$tn {
SELECT highlight(t1, 0, '(', ')') FROM t1($match)
} $res
}
do_execsql_test 8.2 {
CREATE VIRTUAL TABLE ft2 USING fts5(a, tokenize="trigram");
INSERT INTO ft2 VALUES('abc x cde');
INSERT INTO ft2 VALUES('abc cde');
INSERT INTO ft2 VALUES('abcde');
}
do_execsql_test 8.3 {
SELECT highlight(ft2, 0, '[', ']') FROM ft2 WHERE ft2 MATCH 'abc AND cde';
} {
{[abc] x [cde]}
{[abc] [cde]}
{[abcde]}
}
finish_test
+109
View File
@@ -0,0 +1,109 @@
# 2023 October 24
#
# The author disclaims copyright to this source code. In place of
# a legal notice, here is a blessing:
#
# May you do good and not evil.
# May you find forgiveness for yourself and forgive others.
# May you share freely, never taking more than you give.
#
#*************************************************************************
#
# Tests for the fts5 "trigram" tokenizer.
#
source [file join [file dirname [info script]] fts5_common.tcl]
ifcapable !fts5 { finish_test ; return }
set ::testprefix fts5trigram2
do_execsql_test 1.0 "
CREATE VIRTUAL TABLE t1 USING fts5(y, tokenize='trigram remove_diacritics 1');
INSERT INTO t1 VALUES('abc\u0303defghijklm');
INSERT INTO t1 VALUES('a\u0303b\u0303c\u0303defghijklm');
"
do_execsql_test 1.1 {
SELECT highlight(t1, 0, '(', ')') FROM t1('abc');
} [list \
"(abc\u0303)defghijklm" \
"(a\u0303b\u0303c\u0303)defghijklm" \
]
do_execsql_test 1.2 {
SELECT highlight(t1, 0, '(', ')') FROM t1('bcde');
} [list \
"a(bc\u0303de)fghijklm" \
"a\u0303(b\u0303c\u0303de)fghijklm" \
]
do_execsql_test 1.3 {
SELECT highlight(t1, 0, '(', ')') FROM t1('cdef');
} [list \
"ab(c\u0303def)ghijklm" \
"a\u0303b\u0303(c\u0303def)ghijklm" \
]
do_execsql_test 1.4 {
SELECT highlight(t1, 0, '(', ')') FROM t1('def');
} [list \
"abc\u0303(def)ghijklm" \
"a\u0303b\u0303c\u0303(def)ghijklm" \
]
#-------------------------------------------------------------------------
do_catchsql_test 2.0 {
CREATE VIRTUAL TABLE t2 USING fts5(
z, tokenize='trigram case_sensitive 1 remove_diacritics 1'
);
} {1 {error in tokenizer constructor}}
do_execsql_test 2.1 {
CREATE VIRTUAL TABLE t2 USING fts5(
z, tokenize='trigram case_sensitive 0 remove_diacritics 1'
);
}
do_execsql_test 2.2 "
INSERT INTO t2 VALUES('\u00E3bcdef');
INSERT INTO t2 VALUES('b\u00E3cdef');
INSERT INTO t2 VALUES('bc\u00E3def');
INSERT INTO t2 VALUES('bcd\u00E3ef');
"
do_execsql_test 2.3 {
SELECT highlight(t2, 0, '(', ')') FROM t2('abc');
} "(\u00E3bc)def"
do_execsql_test 2.4 {
SELECT highlight(t2, 0, '(', ')') FROM t2('bac');
} "(b\u00E3c)def"
do_execsql_test 2.5 {
SELECT highlight(t2, 0, '(', ')') FROM t2('bca');
} "(bc\u00E3)def"
do_execsql_test 2.6 "
SELECT highlight(t2, 0, '(', ')') FROM t2('\u00E3bc');
" "(\u00E3bc)def"
#-------------------------------------------------------------------------
do_execsql_test 3.0 {
CREATE VIRTUAL TABLE t3 USING fts5(
z, tokenize='trigram remove_diacritics 1'
);
} {}
do_execsql_test 3.1 "
INSERT INTO t3 VALUES ('\u0303abc\u0303');
"
do_execsql_test 3.2 {
SELECT highlight(t3, 0, '(', ')') FROM t3('abc');
} "\u0303(abc\u0303)"
#-------------------------------------------------------------------------
do_execsql_test 4.0 {
CREATE VIRTUAL TABLE t4 USING fts5(z, tokenize=trigram);
} {}
breakpoint
do_execsql_test 4.1 {
INSERT INTO t4 VALUES('ABCD');
} {}
finish_test
+505
View File
@@ -0,0 +1,505 @@
# Quick-and-dirty makefile to bootstrap the sqlite3-jni project. This
# build assumes a Linux-like system.
default: all
JAVA_HOME ?= $(HOME)/jdk/current
# e.g. /usr/lib/jvm/default-javajava-19-openjdk-amd64
JDK_HOME ?= $(JAVA_HOME)
# ^^^ JDK_HOME is not as widely used as JAVA_HOME
bin.jar := $(JDK_HOME)/bin/jar
bin.java := $(JDK_HOME)/bin/java
bin.javac := $(JDK_HOME)/bin/javac
bin.javadoc := $(JDK_HOME)/bin/javadoc
ifeq (,$(wildcard $(JDK_HOME)))
$(error set JDK_HOME to the top-most dir of your JDK installation.)
endif
MAKEFILE := $(lastword $(MAKEFILE_LIST))
$(MAKEFILE):
package.jar := sqlite3-jni.jar
dir.top := ../..
dir.tool := ../../tool
dir.jni := $(patsubst %/,%,$(dir $(MAKEFILE)))
dir.src := $(dir.jni)/src
dir.src.c := $(dir.src)/c
dir.bld := $(dir.jni)/bld
dir.bld.c := $(dir.bld)
dir.src.jni := $(dir.src)/org/sqlite/jni
dir.src.capi := $(dir.src.jni)/capi
dir.src.fts5 := $(dir.src.jni)/fts5
dir.tests := $(dir.src)/tests
mkdir ?= mkdir -p
$(dir.bld.c):
$(mkdir) $@
javac.flags ?= -Xlint:unchecked -Xlint:deprecation
java.flags ?=
javac.flags += -encoding utf8
# -------------^^^^^^^^^^^^^^ required for Windows builds
jnicheck ?= 1
ifeq (1,$(jnicheck))
java.flags += -Xcheck:jni
endif
classpath := $(dir.src)
CLEAN_FILES := $(package.jar)
DISTCLEAN_FILES := $(dir.jni)/*~ $(dir.src.c)/*~ $(dir.src.jni)/*~
sqlite3-jni.h := $(dir.src.c)/sqlite3-jni.h
.NOTPARALLEL: $(sqlite3-jni.h)
CApi.java := $(dir.src.capi)/CApi.java
SQLTester.java := $(dir.src.capi)/SQLTester.java
CApi.class := $(CApi.java:.java=.class)
SQLTester.class := $(SQLTester.java:.java=.class)
########################################################################
# The future of FTS5 customization in this API is as yet unclear.
# The pieces are all in place, and are all thin proxies so not much
# complexity, but some semantic changes were required in porting
# which are largely untested.
#
# Reminder: this flag influences the contents of $(sqlite3-jni.h),
# which is checked in. Please do not check in changes to that file in
# which the fts5 APIs have been stripped unless that feature is
# intended to be stripped for good.
enable.fts5 ?= 1
ifeq (,$(wildcard $(dir.tests)/*))
enable.tester := 0
else
enable.tester := 1
endif
# bin.version-info = binary to output various sqlite3 version info
# building the distribution zip file.
bin.version-info := $(dir.top)/version-info
.NOTPARALLEL: $(bin.version-info)
$(bin.version-info): $(dir.tool)/version-info.c $(sqlite3.h) $(dir.top)/Makefile
$(MAKE) -C $(dir.top) version-info
# Be explicit about which Java files to compile so that we can work on
# in-progress files without requiring them to be in a compilable statae.
JAVA_FILES.main := $(patsubst %,$(dir.src.jni)/annotation/%,\
Experimental.java \
NotNull.java \
Nullable.java \
) $(patsubst %,$(dir.src.capi)/%,\
AbstractCollationCallback.java \
AggregateFunction.java \
AuthorizerCallback.java \
AutoExtensionCallback.java \
BusyHandlerCallback.java \
CollationCallback.java \
CollationNeededCallback.java \
CommitHookCallback.java \
ConfigLogCallback.java \
ConfigSqlLogCallback.java \
NativePointerHolder.java \
OutputPointer.java \
PrepareMultiCallback.java \
PreupdateHookCallback.java \
ProgressHandlerCallback.java \
ResultCode.java \
RollbackHookCallback.java \
ScalarFunction.java \
SQLFunction.java \
CallbackProxy.java \
CApi.java \
TableColumnMetadata.java \
TraceV2Callback.java \
UpdateHookCallback.java \
ValueHolder.java \
WindowFunction.java \
XDestroyCallback.java \
sqlite3.java \
sqlite3_blob.java \
sqlite3_context.java \
sqlite3_stmt.java \
sqlite3_value.java \
) $(patsubst %,$(dir.src.jni)/wrapper1/%,\
AggregateFunction.java \
ScalarFunction.java \
SqlFunction.java \
Sqlite.java \
SqliteException.java \
ValueHolder.java \
WindowFunction.java \
)
JAVA_FILES.unittest := $(patsubst %,$(dir.src.jni)/%,\
capi/Tester1.java \
wrapper1/Tester2.java \
)
ifeq (1,$(enable.fts5))
JAVA_FILES.unittest += $(patsubst %,$(dir.src.fts5)/%,\
TesterFts5.java \
)
JAVA_FILES.main += $(patsubst %,$(dir.src.fts5)/%,\
fts5_api.java \
fts5_extension_function.java \
fts5_tokenizer.java \
Fts5.java \
Fts5Context.java \
Fts5ExtensionApi.java \
Fts5PhraseIter.java \
Fts5Tokenizer.java \
XTokenizeCallback.java \
)
endif
JAVA_FILES.tester := $(SQLTester.java)
JAVA_FILES.package.info := \
$(dir.src.jni)/package-info.java \
$(dir.src.jni)/annotation/package-info.java
CLASS_FILES.main := $(JAVA_FILES.main:.java=.class)
CLASS_FILES.unittest := $(JAVA_FILES.unittest:.java=.class)
CLASS_FILES.tester := $(JAVA_FILES.tester:.java=.class)
JAVA_FILES += $(JAVA_FILES.main) $(JAVA_FILES.unittest)
ifeq (1,$(enable.tester))
JAVA_FILES += $(JAVA_FILES.tester)
endif
CLASS_FILES :=
define CLASSFILE_DEPS
all: $(1).class
$(1).class: $(1).java
CLASS_FILES += $(1).class
endef
$(foreach B,$(basename \
$(JAVA_FILES.main) $(JAVA_FILES.unittest) $(JAVA_FILES.tester)),\
$(eval $(call CLASSFILE_DEPS,$(B))))
$(CLASS_FILES): $(MAKEFILE)
$(bin.javac) $(javac.flags) -h $(dir.bld.c) -cp $(classpath) $(JAVA_FILES)
#.PHONY: classfiles
########################################################################
# Set up sqlite3.c and sqlite3.h...
#
# To build with SEE (https://sqlite.org/see), either put sqlite3-see.c
# in the top of this build tree or pass
# sqlite3.c=PATH_TO_sqlite3-see.c to the build. Note that only
# encryption modules with no 3rd-party dependencies will currently
# work here: AES256-OFB, AES128-OFB, and AES128-CCM. Not
# coincidentally, those 3 modules are included in the sqlite3-see.c
# bundle.
#
# A custom sqlite3.c must not have any spaces in its name.
# $(sqlite3.canonical.c) must point to the sqlite3.c in
# the sqlite3 canonical source tree, as that source file
# is required for certain utility and test code.
sqlite3.canonical.c := $(firstword $(wildcard $(dir.src.c)/sqlite3.c) $(dir.top)/sqlite3.c)
sqlite3.canonical.h := $(firstword $(wildcard $(dir.src.c)/sqlite3.h) $(dir.top)/sqlite3.h)
sqlite3.c := $(sqlite3.canonical.c)
sqlite3.h := $(sqlite3.canonical.h)
#ifeq (,$(shell grep sqlite3_activate_see $(sqlite3.c) 2>/dev/null))
# SQLITE_C_IS_SEE := 0
#else
# SQLITE_C_IS_SEE := 1
# $(info This is an SEE build.)
#endif
.NOTPARALLEL: $(sqlite3.h)
$(sqlite3.h):
$(MAKE) -C $(dir.top) sqlite3.c
$(sqlite3.c): $(sqlite3.h)
opt.threadsafe ?= 1
opt.fatal-oom ?= 1
opt.debug ?= 1
opt.metrics ?= 1
SQLITE_OPT = \
-DSQLITE_THREADSAFE=$(opt.threadsafe) \
-DSQLITE_TEMP_STORE=2 \
-DSQLITE_USE_URI=1 \
-DSQLITE_OMIT_LOAD_EXTENSION \
-DSQLITE_OMIT_DEPRECATED \
-DSQLITE_OMIT_SHARED_CACHE \
-DSQLITE_C=$(sqlite3.c) \
-DSQLITE_JNI_FATAL_OOM=$(opt.fatal-oom) \
-DSQLITE_JNI_ENABLE_METRICS=$(opt.metrics)
opt.extras ?= 1
ifeq (1,$(opt.extras))
SQLITE_OPT += -DSQLITE_ENABLE_RTREE \
-DSQLITE_ENABLE_EXPLAIN_COMMENTS \
-DSQLITE_ENABLE_STMTVTAB \
-DSQLITE_ENABLE_DBPAGE_VTAB \
-DSQLITE_ENABLE_DBSTAT_VTAB \
-DSQLITE_ENABLE_BYTECODE_VTAB \
-DSQLITE_ENABLE_OFFSET_SQL_FUNC \
-DSQLITE_ENABLE_PREUPDATE_HOOK \
-DSQLITE_ENABLE_NORMALIZE \
-DSQLITE_ENABLE_SQLLOG \
-DSQLITE_ENABLE_COLUMN_METADATA
endif
ifeq (1,$(opt.debug))
SQLITE_OPT += -DSQLITE_DEBUG -g -DDEBUG -UNDEBUG
else
SQLITE_OPT += -Os
endif
ifeq (1,$(enable.fts5))
SQLITE_OPT += -DSQLITE_ENABLE_FTS5
endif
sqlite3-jni.c := $(dir.src.c)/sqlite3-jni.c
sqlite3-jni.o := $(dir.bld.c)/sqlite3-jni.o
sqlite3-jni.h := $(dir.src.c)/sqlite3-jni.h
package.dll := $(dir.bld.c)/libsqlite3-jni.so
# All javac-generated .h files must be listed in $(sqlite3-jni.h.in):
sqlite3-jni.h.in :=
# $(java.with.jni) lists all Java files which contain JNI decls:
java.with.jni :=
define ADD_JNI_H
sqlite3-jni.h.in += $$(dir.bld.c)/org_sqlite_jni$(3)_$(2).h
java.with.jni += $(1)/$(2).java
$$(dir.bld.c)/org_sqlite_jni$(3)_$(2).h: $(1)/$(2).java
endef
# Invoke ADD_JNI_H once for each Java file which includes JNI
# declarations:
$(eval $(call ADD_JNI_H,$(dir.src.capi),CApi,_capi))
$(eval $(call ADD_JNI_H,$(dir.src.capi),SQLTester,_capi))
ifeq (1,$(enable.fts5))
$(eval $(call ADD_JNI_H,$(dir.src.fts5),Fts5ExtensionApi,_fts5))
$(eval $(call ADD_JNI_H,$(dir.src.fts5),fts5_api,_fts5))
$(eval $(call ADD_JNI_H,$(dir.src.fts5),fts5_tokenizer,_fts5))
endif
$(sqlite3-jni.h.in): $(dir.bld.c)
#package.dll.cfiles :=
package.dll.cflags = \
-std=c99 \
-fPIC \
-I. \
-I$(dir $(sqlite3.h)) \
-I$(dir.src.c) \
-I$(JDK_HOME)/include \
$(patsubst %,-I%,$(patsubst %.h,,$(wildcard $(JDK_HOME)/include/*))) \
-Wall
# The gross $(patsubst...) above is to include the platform-specific
# subdir which lives under $(JDK_HOME)/include and is a required
# include path for client-level code.
#
# Using (-Wall -Wextra) triggers an untennable number of
# gcc warnings from sqlite3.c for mundane things like
# unused parameters.
########################################################################
ifeq (1,$(enable.tester))
package.dll.cflags += -DSQLITE_JNI_ENABLE_SQLTester
endif
$(sqlite3-jni.h): $(sqlite3-jni.h.in) $(MAKEFILE)
@cat $(sqlite3-jni.h.in) > $@.tmp
@if cmp $@ $@.tmp >/dev/null; then \
rm -f $@.tmp; \
echo "$@ not modified"; \
else \
mv $@.tmp $@; \
echo "Updated $@"; \
fi
@if [ x1 != x$(enable.fts5) ]; then \
echo "*** REMINDER:"; \
echo "*** enable.fts5=0, so please do not check in changes to $@."; \
fi
$(package.dll): $(sqlite3-jni.h) $(sqlite3.c) $(sqlite3.h)
$(package.dll): $(sqlite3-jni.c) $(MAKEFILE)
$(CC) $(package.dll.cflags) $(SQLITE_OPT) \
$(sqlite3-jni.c) -shared -o $@
all: $(package.dll)
.PHONY: test test-one
Tester1.flags ?=
Tester2.flags ?=
test.flags.jvm = -ea -Djava.library.path=$(dir.bld.c) \
$(java.flags) -cp $(classpath)
test.deps := $(CLASS_FILES) $(package.dll)
test-one: $(test.deps)
$(bin.java) $(test.flags.jvm) org.sqlite.jni.capi.Tester1 $(Tester1.flags)
$(bin.java) $(test.flags.jvm) org.sqlite.jni.wrapper1.Tester2 $(Tester2.flags)
test-sqllog: $(test.deps)
@echo "Testing with -sqllog..."
$(bin.java) $(test.flags.jvm) org.sqlite.jni.capi.Tester1 $(Tester1.flags) -sqllog
test-mt: $(test.deps)
@echo "Testing in multi-threaded mode:";
$(bin.java) $(test.flags.jvm) org.sqlite.jni.capi.Tester1 \
-t 7 -r 50 -shuffle $(Tester1.flags)
$(bin.java) $(test.flags.jvm) org.sqlite.jni.wrapper1.Tester2 \
-t 7 -r 50 -shuffle $(Tester2.flags)
test: test-one test-mt
tests: test test-sqllog
tester.scripts := $(sort $(wildcard $(dir.src)/tests/*.test))
tester.flags ?= # --verbose
.PHONY: tester tester-local tester-ext
ifeq (1,$(enable.tester))
tester-local: $(CLASS_FILES.tester) $(package.dll)
$(bin.java) -ea -Djava.library.path=$(dir.bld.c) \
$(java.flags) -cp $(classpath) \
org.sqlite.jni.capi.SQLTester $(tester.flags) $(tester.scripts)
tester: tester-local
else
tester:
@echo "SQLTester support is disabled."
endif
tester.extdir.default := $(dir.tests)/ext
tester.extdir ?= $(tester.extdir.default)
tester.extern-scripts := $(wildcard $(tester.extdir)/*.test)
ifneq (,$(tester.extern-scripts))
tester-ext:
$(bin.java) -ea -Djava.library.path=$(dir.bld.c) \
$(java.flags) -cp $(classpath) \
org.sqlite.jni.capi.SQLTester $(tester.flags) $(tester.extern-scripts)
else
tester-ext:
@echo "******************************************************"; \
echo "*** Include the out-of-tree test suite in the 'tester'"; \
echo "*** target by either symlinking its directory to"; \
echo "*** $(tester.extdir.default) or passing it to make"; \
echo "*** as tester.extdir=/path/to/that/dir."; \
echo "******************************************************";
endif
tester-ext: tester-local
tester: tester-ext
tests: tester
########################################################################
# Build each SQLITE_THREADMODE variant and run all tests against them.
multitest: clean
define MULTIOPT
multitest: multitest-$(1)
multitest-$(1):
$$(MAKE) opt.debug=$$(opt.debug) $(patsubst %,opt.%,$(2)) \
tests clean enable.fts5=1
endef
$(eval $(call MULTIOPT,01,threadsafe=0 oom=1))
$(eval $(call MULTIOPT,00,threadsafe=0 oom=0))
$(eval $(call MULTIOPT,11,threadsafe=1 oom=1))
$(eval $(call MULTIOPT,10,threadsafe=1 oom=0))
$(eval $(call MULTIOPT,21,threadsafe=2 oom=1))
$(eval $(call MULTIOPT,20,threadsafe=2 oom=0))
########################################################################
# jar bundle...
package.jar.in := $(abspath $(dir.src)/jar.in)
CLEAN_FILES += $(package.jar.in)
JAVA_FILES.jar := $(JAVA_FILES.main) $(JAVA_FILES.unittest) $(JAVA_FILES.package.info)
CLASS_FILES.jar := $(filter-out %/package-info.class,$(JAVA_FILES.jar:.java=.class))
$(package.jar.in): $(package.dll) $(MAKEFILE)
ls -1 \
$(dir.src.jni)/*/*.java $(dir.src.jni)/*/*.class \
| sed -e 's,^$(dir.src)/,,' | sort > $@
$(package.jar): $(CLASS_FILES.jar) $(MAKEFILE) $(package.jar.in)
@rm -f $(dir.src)/c/*~ $(dir.src.jni)/*~
cd $(dir.src); $(bin.jar) -cfe ../$@ org.sqlite.jni.capi.Tester1 @$(package.jar.in)
@ls -la $@
@echo "To use this jar you will need the -Djava.library.path=DIR/CONTAINING/libsqlite3-jni.so flag."
@echo "e.g. java -Djava.library.path=bld -jar $@"
jar: $(package.jar)
run-jar: $(package.jar) $(package.dll)
$(bin.java) -Djava.library.path=$(dir.bld) -jar $(package.jar) $(run-jar.flags)
########################################################################
# javadoc...
dir.doc := $(dir.jni)/javadoc
doc.index := $(dir.doc)/index.html
javadoc.exclude := -exclude org.sqlite.jni.fts5
# ^^^^ 2023-09-13: elide the fts5 parts from the public docs for
# the time being, as it's not clear where the Java bindings for
# those bits are going.
# javadoc.exclude += -exclude org.sqlite.jni.capi
# ^^^^ exclude the capi API only for certain builds (TBD)
$(doc.index): $(JAVA_FILES.main) $(MAKEFILE)
@if [ -d $(dir.doc) ]; then rm -fr $(dir.doc)/*; fi
$(bin.javadoc) -cp $(classpath) -d $(dir.doc) -quiet \
-subpackages org.sqlite.jni $(javadoc.exclude)
@echo "javadoc output is in $@"
.PHONY: doc javadoc docserve
.FORCE: doc
doc: $(doc.index)
javadoc: $(doc.index)
# Force rebild of docs
redoc:
@rm -f $(doc.index)
@$(MAKE) doc
docserve: $(doc.index)
cd $(dir.doc) && althttpd -max-age 1 -page index.html
########################################################################
# Clean up...
CLEAN_FILES += $(dir.bld.c)/* \
$(dir.src.jni)/*.class \
$(dir.src.jni)/*/*.class \
$(package.dll) \
hs_err_pid*.log
.PHONY: clean distclean
clean:
-rm -f $(CLEAN_FILES)
distclean: clean
-rm -f $(DISTCLEAN_FILES)
-rm -fr $(dir.bld.c) $(dir.doc)
########################################################################
# disttribution bundle rules...
ifeq (,$(filter snapshot,$(MAKECMDGOALS)))
dist-name-prefix := sqlite-jni
else
dist-name-prefix := sqlite-jni-snapshot-$(shell /usr/bin/date +%Y%m%d)
endif
dist-name := $(dist-name-prefix)-TEMP
dist-dir.top := $(dist-name)
dist-dir.src := $(dist-dir.top)/src
dist.top.extras := \
README.md
.PHONY: dist snapshot
dist: \
$(bin.version-info) $(sqlite3.canonical.c) \
$(package.jar) $(MAKEFILE)
@echo "Making end-user deliverables..."
@echo "****************************************************************************"; \
echo "*** WARNING: be sure to build this with JDK8 (javac 1.8) for compatibility."; \
echo "*** reasons!"; $$($(bin.javac) -version); \
echo "****************************************************************************"
@rm -fr $(dist-dir.top)
@mkdir -p $(dist-dir.src)
@cp -p $(dist.top.extras) $(dist-dir.top)/.
@cp -p jar-dist.make $(dist-dir.top)/Makefile
@cp -p $(dir.src.c)/*.[ch] $(dist-dir.src)/.
@cp -p $(sqlite3.canonical.c) $(sqlite3.canonical.h) $(dist-dir.src)/.
@set -e; \
vnum=$$($(bin.version-info) --download-version); \
vjar=$$($(bin.version-info) --version); \
vdir=$(dist-name-prefix)-$$vnum; \
arczip=$$vdir.zip; \
cp -p $(package.jar) $(dist-dir.top)/sqlite3-jni-$${vjar}.jar; \
echo "Making $$arczip ..."; \
rm -fr $$arczip $$vdir; \
mv $(dist-dir.top) $$vdir; \
zip -qr $$arczip $$vdir; \
rm -fr $$vdir; \
ls -la $$arczip; \
set +e; \
unzip -lv $$arczip || echo "Missing unzip app? Not fatal."
snapshot: dist
.PHONY: dist-clean
clean: dist-clean
dist-clean:
rm -fr $(dist-name) $(wildcard sqlite-jni-*.zip)
+316
View File
@@ -0,0 +1,316 @@
SQLite3 via JNI
========================================================================
This directory houses a Java Native Interface (JNI) binding for the
sqlite3 API. If you are reading this from the distribution ZIP file,
links to resources in the canonical source tree will note work. The
canonical copy of this file can be browsed at:
<https://sqlite.org/src/doc/trunk/ext/jni/README.md>
Technical support is available in the forum:
<https://sqlite.org/forum>
> **FOREWARNING:** this subproject is very much in development and
subject to any number of changes. Please do not rely on any
information about its API until this disclaimer is removed. The JNI
bindings released with version 3.43 are a "tech preview." Once
finalized, strong backward compatibility guarantees will apply.
Project goals/requirements:
- A [1-to-1(-ish) mapping of the C API](#1to1ish) to Java via JNI,
insofar as cross-language semantics allow for. A closely-related
goal is that [the C documentation](https://sqlite.org/c3ref/intro.html)
should be usable as-is, insofar as possible, for the JNI binding.
- Support Java as far back as version 8 (2014).
- Environment-independent. Should work everywhere both Java
and SQLite3 do.
- No 3rd-party dependencies beyond the JDK. That includes no
build-level dependencies for specific IDEs and toolchains. We
welcome the addition of build files for arbitrary environments
insofar as they neither interfere with each other nor become
a maintenance burden for the sqlite developers.
Non-goals:
- Creation of high-level OO wrapper APIs. Clients are free to create
them off of the C-style API.
- Virtual tables are unlikely to be supported due to the amount of
glue code needed to fit them into Java.
- Support for mixed-mode operation, where client code accesses SQLite
both via the Java-side API and the C API via their own native
code. Such cases would be a minefield of potential mis-interactions
between this project's JNI bindings and mixed-mode client code.
Hello World
-----------------------------------------------------------------------
```java
import org.sqlite.jni.*;
import static org.sqlite.jni.CApi.*;
...
final sqlite3 db = sqlite3_open(":memory:");
try {
final int rc = sqlite3_errcode(db);
if( 0 != rc ){
if( null != db ){
System.out.print("Error opening db: "+sqlite3_errmsg(db));
}else{
System.out.print("Error opening db: rc="+rc);
}
... handle error ...
}
// ... else use the db ...
}finally{
// ALWAYS close databases using sqlite3_close() or sqlite3_close_v2()
// when done with them. All of their active statement handles must
// first have been passed to sqlite3_finalize().
sqlite3_close_v2(db);
}
```
Building
========================================================================
The canonical builds assumes a Linux-like environment and requires:
- GNU Make
- A JDK supporting Java 8 or higher
- A modern C compiler. gcc and clang should both work.
Put simply:
```console
$ export JAVA_HOME=/path/to/jdk/root
$ make
$ make test
$ make clean
```
The jar distribution can be created with `make jar`, but note that it
does not contain the binary DLL file. A different DLL is needed for
each target platform.
<a id='1to1ish'></a>
One-to-One(-ish) Mapping to C
========================================================================
This JNI binding aims to provide as close to a 1-to-1 experience with
the C API as cross-language semantics allow. Interface changes are
necessarily made where cross-language semantics do not allow a 1-to-1,
and judiciously made where a 1-to-1 mapping would be unduly cumbersome
to use in Java. In all cases, this binding makes every effort to
provide semantics compatible with the C API documentation even if the
interface to those semantics is slightly different. Any cases which
deviate from those semantics (either removing or adding semantics) are
clearly documented.
Where it makes sense to do so for usability, Java-side overloads are
provided which accept or return data in alternative forms or provide
sensible default argument values. In all such cases they are thin
proxies around the corresponding C APIs and do not introduce new
semantics.
In a few cases, Java-specific capabilities have been added in
new APIs, all of which have "_java" somewhere in their names.
Examples include:
- `sqlite3_result_java_object()`
- `sqlite3_column_java_object()`
- `sqlite3_value_java_object()`
which, as one might surmise, collectively enable the passing of
arbitrary Java objects from user-defined SQL functions through to the
caller.
Golden Rule: Garbage Collection Cannot Free SQLite Resources
------------------------------------------------------------------------
It is important that all databases and prepared statement handles get
cleaned up by client code. A database cannot be closed if it has open
statement handles. `sqlite3_close()` fails if the db cannot be closed
whereas `sqlite3_close_v2()` recognizes that case and marks the db as
a "zombie," pending finalization when the library detects that all
pending statements have been closed. Be aware that Java garbage
collection _cannot_ close a database or finalize a prepared statement.
Those things require explicit API calls.
Classes for which it is sensible support Java's `AutoCloseable`
interface so can be used with try-with-resources constructs.
Golden Rule #2: _Never_ Throw from Callbacks (Unless...)
------------------------------------------------------------------------
All routines in this API, barring explicitly documented exceptions,
retain C-like semantics. For example, they are not permitted to throw
or propagate exceptions and must return error information (if any) via
result codes or `null`. The only cases where the C-style APIs may
throw is through client-side misuse, e.g. passing in a null where it
may cause a `NullPointerException`. The APIs clearly mark function
parameters which should not be null, but does not generally actively
defend itself against such misuse. Some C-style APIs explicitly accept
`null` as a no-op for usability's sake, and some of the JNI APIs
deliberately return an error code, instead of segfaulting, when passed
a `null`.
Client-defined callbacks _must never throw exceptions_ unless _very
explitly documented_ as being throw-safe. Exceptions are generally
reserved for higher-level bindings which are constructed to
specifically deal with them and ensure that they do not leak C-level
resources. In some cases, callback handlers are permitted to throw, in
which cases they get translated to C-level result codes and/or
messages. If a callback which is not permitted to throw throws, its
exception may trigger debug output but will otherwise be suppressed.
The reason some callbacks are permitted to throw and others not is
because all such callbacks act as proxies for C function callback
interfaces and some of those interfaces have no error-reporting
mechanism. Those which are capable of propagating errors back through
the library convert exceptions from callbacks into corresponding
C-level error information. Those which cannot propagate errors
necessarily suppress any exceptions in order to maintain the C-style
semantics of the APIs.
Unwieldy Constructs are Re-mapped
------------------------------------------------------------------------
Some constructs, when modelled 1-to-1 from C to Java, are unduly
clumsy to work with in Java because they try to shoehorn C's way of
doing certain things into Java's wildly different ways. The following
subsections cover those, starting with a verbose explanation and
demonstration of where such changes are "really necessary"...
### Custom Collations
A prime example of where interface changes for Java are necessary for
usability is [registration of a custom
collation](https://sqlite.org/c3ref/create_collation.html):
```c
// C:
int sqlite3_create_collation(sqlite3 * db, const char * name, int eTextRep,
void *pUserData,
int (*xCompare)(void*,int,void const *,int,void const *));
int sqlite3_create_collation_v2(sqlite3 * db, const char * name, int eTextRep,
void *pUserData,
int (*xCompare)(void*,int,void const *,int,void const *),
void (*xDestroy)(void*));
```
The `pUserData` object is optional client-defined state for the
`xCompare()` and/or `xDestroy()` callback functions, both of which are
passed that object as their first argument. That data is passed around
"externally" in C because that's how C models the world. If we were to
bind that part as-is to Java, the result would be awkward to use (^Yes,
we tried this.):
```java
// Java:
int sqlite3_create_collation(sqlite3 db, String name, int eTextRep,
Object pUserData, xCompareType xCompare);
int sqlite3_create_collation_v2(sqlite3 db, String name, int eTextRep,
Object pUserData,
xCompareType xCompare, xDestroyType xDestroy);
```
The awkwardness comes from (A) having two distinctly different objects
for callbacks and (B) having their internal state provided separately,
which is ill-fitting in Java. For the sake of usability, C APIs which
follow that pattern use a slightly different Java interface:
```java
int sqlite3_create_collation(sqlite3 db, String name, int eTextRep,
SomeCallbackType collation);
```
Where the `Collation` class has an abstract `call()` method and
no-op `xDestroy()` method which can be overridden if needed, leading to
a much more Java-esque usage:
```java
int rc = sqlite3_create_collation(db, "mycollation", SQLITE_UTF8, new SomeCallbackType(){
// Required comparison function:
@Override public int call(byte[] lhs, byte[] rhs){ ... }
// Optional finalizer function:
@Override public void xDestroy(){ ... }
// Optional local state:
private String localState1 =
"This is local state. There are many like it, but this one is mine.";
private MyStateType localState2 = new MyStateType();
...
});
```
Noting that:
- It is possible to bind in call-scope-local state via closures, if
desired, as opposed to packing it into the Collation object.
- No capabilities of the C API are lost or unduly obscured via the
above API reshaping, so power users need not make any compromises.
- In the specific example above, `sqlite3_create_collation_v2()`
becomes superfluous because the provided interface effectively
provides both the v1 and v2 interfaces, the difference being that
overriding the `xDestroy()` method effectively gives it v2
semantics.
### User-defined SQL Functions (a.k.a. UDFs)
The [`sqlite3_create_function()`](https://sqlite.org/c3ref/create_function.html)
family of APIs make heavy use of function pointers to provide
client-defined callbacks, necessitating interface changes in the JNI
binding. The Java API has only one core function-registration function:
```java
int sqlite3_create_function(sqlite3 db, String funcName, int nArgs,
int encoding, SQLFunction func);
```
> Design question: does the encoding argument serve any purpose in
Java? That's as-yet undetermined. If not, it will be removed.
`SQLFunction` is not used directly, but is instead instantiated via
one of its three subclasses:
- `ScalarFunction` implements simple scalar functions using but a
single callback.
- `AggregateFunction` implements aggregate functions using two
callbacks.
- `WindowFunction` implements window functions using four
callbacks.
Search [`Tester1.java`](/file/ext/jni/src/org/sqlite/jni/capi/Tester1.java) for
`SQLFunction` for how it's used.
Reminder: see the disclaimer at the top of this document regarding the
in-flux nature of this API.
### And so on...
Various APIs which accept callbacks, e.g. `sqlite3_trace_v2()` and
`sqlite3_update_hook()`, use interfaces similar to those shown above.
Despite the changes in signature, the JNI layer makes every effort to
provide the same semantics as the C API documentation suggests.
+60
View File
@@ -0,0 +1,60 @@
#!/this/is/make
#^^^^ help emacs out
#
# This is a POSIX-make-compatible makefile for building the sqlite3
# JNI library from "dist" zip file. It must be edited to set the
# proper top-level JDK directory and, depending on the platform, add a
# platform-specific -I directory. It should build as-is with any
# 2020s-era version of gcc or clang. It requires JDK version 8 or
# higher and that JAVA_HOME points to the top-most installation
# directory of that JDK. On Ubuntu-style systems the JDK is typically
# installed under /usr/lib/jvm/java-VERSION-PLATFORM.
default: all
JAVA_HOME = /usr/lib/jvm/java-1.8.0-openjdk-amd64
CFLAGS = \
-fPIC \
-Isrc \
-I$(JAVA_HOME)/include \
-I$(JAVA_HOME)/include/linux \
-I$(JAVA_HOME)/include/apple \
-I$(JAVA_HOME)/include/bsd \
-Wall
SQLITE_OPT = \
-DSQLITE_ENABLE_RTREE \
-DSQLITE_ENABLE_EXPLAIN_COMMENTS \
-DSQLITE_ENABLE_STMTVTAB \
-DSQLITE_ENABLE_DBPAGE_VTAB \
-DSQLITE_ENABLE_DBSTAT_VTAB \
-DSQLITE_ENABLE_BYTECODE_VTAB \
-DSQLITE_ENABLE_OFFSET_SQL_FUNC \
-DSQLITE_OMIT_LOAD_EXTENSION \
-DSQLITE_OMIT_DEPRECATED \
-DSQLITE_OMIT_SHARED_CACHE \
-DSQLITE_THREADSAFE=1 \
-DSQLITE_TEMP_STORE=2 \
-DSQLITE_USE_URI=1 \
-DSQLITE_ENABLE_FTS5 \
-DSQLITE_DEBUG
sqlite3-jni.dll = libsqlite3-jni.so
$(sqlite3-jni.dll):
@echo "************************************************************************"; \
echo "*** If this fails to build, be sure to edit this makefile ***"; \
echo "*** to configure it for your system. ***"; \
echo "************************************************************************"
$(CC) $(CFLAGS) $(SQLITE_OPT) \
src/sqlite3-jni.c -shared -o $@
@echo "Now try running it with: make test"
test.flags = -Djava.library.path=. sqlite3-jni-*.jar
test: $(sqlite3-jni.dll)
java -jar $(test.flags)
java -jar $(test.flags) -t 7 -r 10 -shuffle
clean:
-rm -f $(sqlite3-jni.dll)
all: $(sqlite3-jni.dll)
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,30 @@
/*
** 2023-09-27
**
** The author disclaims copyright to this source code. In place of
** a legal notice, here is a blessing:
**
** May you do good and not evil.
** May you find forgiveness for yourself and forgive others.
** May you share freely, never taking more than you give.
**
*************************************************************************
** This file houses the Experimental annotation for the sqlite3 C API.
*/
package org.sqlite.jni.annotation;
import java.lang.annotation.*;
/**
This annotation is for flagging methods, constructors, and types
which are expressly experimental and subject to any amount of
change or outright removal. Client code should not rely on such
features.
*/
@Documented
@Retention(RetentionPolicy.SOURCE)
@Target({
ElementType.METHOD,
ElementType.CONSTRUCTOR,
ElementType.TYPE
})
public @interface Experimental{}
@@ -0,0 +1,71 @@
/*
** 2023-09-27
**
** The author disclaims copyright to this source code. In place of
** a legal notice, here is a blessing:
**
** May you do good and not evil.
** May you find forgiveness for yourself and forgive others.
** May you share freely, never taking more than you give.
**
*************************************************************************
** This file houses the NotNull annotation for the sqlite3 C API.
*/
package org.sqlite.jni.annotation;
import java.lang.annotation.*;
/**
This annotation is for flagging parameters which may not legally be
null or point to closed/finalized C-side resources.
<p>In the case of Java types which map directly to C struct types
(e.g. {@link org.sqlite.jni.capi.sqlite3}, {@link
org.sqlite.jni.capi.sqlite3_stmt}, and {@link
org.sqlite.jni.capi.sqlite3_context}), a closed/finalized resource
is also considered to be null for purposes this annotation because
the C-side effect of passing such a handle is the same as if null
is passed.</p>
<p>When used in the context of Java interfaces which are called
from the C APIs, this annotation communicates that the C API will
never pass a null value to the callback for that parameter.</p>
<p>Passing a null, for this annotation's definition of null, for
any parameter marked with this annoation specifically invokes
undefined behavior (see below).</p>
<p>Passing 0 (i.e. C NULL) or a negative value for any long-type
parameter marked with this annoation specifically invokes undefined
behavior (see below). Such values are treated as C pointers in the
JNI layer.</p>
<p><b>Undefined behaviour:</b> the JNI build uses the {@code
SQLITE_ENABLE_API_ARMOR} build flag, meaning that the C code
invoked with invalid NULL pointers and the like will not invoke
undefined behavior in the conventional C sense, but may, for
example, return result codes which are not documented for the
affected APIs or may otherwise behave unpredictably. In no known
cases will such arguments result in C-level code dereferencing a
NULL pointer or accessing out-of-bounds (or otherwise invalid)
memory. In other words, they may cause unexpected behavior but
should never cause an outright crash or security issue.</p>
<p>Note that the C-style API does not throw any exceptions on its
own because it has a no-throw policy in order to retain its C-style
semantics, but it may trigger NullPointerExceptions (or similar) if
passed a null for a parameter flagged with this annotation.</p>
<p>This annotation is informational only. No policy is in place to
programmatically ensure that NotNull is conformed to in client
code.</p>
<p>This annotation is solely for the use by the classes in the
org.sqlite.jni package and subpackages, but is made public so that
javadoc will link to it from the annotated functions. It is not
part of the public API and client-level code must not rely on
it.</p>
*/
@Documented
@Retention(RetentionPolicy.SOURCE)
@Target(ElementType.PARAMETER)
public @interface NotNull{}
@@ -0,0 +1,33 @@
/*
** 2023-09-27
**
** The author disclaims copyright to this source code. In place of
** a legal notice, here is a blessing:
**
** May you do good and not evil.
** May you find forgiveness for yourself and forgive others.
** May you share freely, never taking more than you give.
**
*************************************************************************
** This file houses the Nullable annotation for the sqlite3 C API.
*/
package org.sqlite.jni.annotation;
import java.lang.annotation.*;
/**
This annotation is for flagging parameters which may legally be
null, noting that they may behave differently if passed null but
are prepared to expect null as a value. When used in the context of
callback methods which are called into from the C APIs, this
annotation communicates that the C API may pass a null value to the
callback.
<p>This annotation is solely for the use by the classes in this
package but is made public so that javadoc will link to it from the
annotated functions. It is not part of the public API and
client-level code must not rely on it.
*/
@Documented
@Retention(RetentionPolicy.SOURCE)
@Target(ElementType.PARAMETER)
public @interface Nullable{}
@@ -0,0 +1,17 @@
/*
** 2023-09-27
**
** The author disclaims copyright to this source code. In place of
** a legal notice, here is a blessing:
**
** May you do good and not evil.
** May you find forgiveness for yourself and forgive others.
** May you share freely, never taking more than you give.
**
*************************************************************************
*/
/**
This package houses annotations specific to the JNI bindings of the
SQLite3 C API.
*/
package org.sqlite.jni.annotation;
@@ -0,0 +1,34 @@
/*
** 2023-08-25
**
** The author disclaims copyright to this source code. In place of
** a legal notice, here is a blessing:
**
** May you do good and not evil.
** May you find forgiveness for yourself and forgive others.
** May you share freely, never taking more than you give.
**
*************************************************************************
** This file is part of the JNI bindings for the sqlite3 C API.
*/
package org.sqlite.jni.capi;
import org.sqlite.jni.annotation.NotNull;
/**
An implementation of {@link CollationCallback} which provides a
no-op xDestroy() method.
*/
public abstract class AbstractCollationCallback
implements CollationCallback, XDestroyCallback {
/**
Must compare the given byte arrays and return the result using
{@code memcmp()} semantics.
*/
public abstract int call(@NotNull byte[] lhs, @NotNull byte[] rhs);
/**
Optionally override to be notified when the UDF is finalized by
SQLite. This implementation does nothing.
*/
public void xDestroy(){}
}
@@ -0,0 +1,138 @@
/*
** 2023-08-25
**
** The author disclaims copyright to this source code. In place of
** a legal notice, here is a blessing:
**
** May you do good and not evil.
** May you find forgiveness for yourself and forgive others.
** May you share freely, never taking more than you give.
**
*************************************************************************
** This file is part of the JNI bindings for the sqlite3 C API.
*/
package org.sqlite.jni.capi;
/**
A SQLFunction implementation for aggregate functions. Its T is the
data type of its "accumulator" state, an instance of which is
intended to be be managed using the getAggregateState() and
takeAggregateState() methods.
*/
public abstract class AggregateFunction<T> implements SQLFunction {
/**
As for the xStep() argument of the C API's
sqlite3_create_function(). If this function throws, the
exception is not propagated and a warning might be emitted to a
debugging channel.
*/
public abstract void xStep(sqlite3_context cx, sqlite3_value[] args);
/**
As for the xFinal() argument of the C API's sqlite3_create_function().
If this function throws, it is translated into an sqlite3_result_error().
*/
public abstract void xFinal(sqlite3_context cx);
/**
Optionally override to be notified when the UDF is finalized by
SQLite.
*/
public void xDestroy() {}
/**
PerContextState assists aggregate and window functions in
managing their accumulator state across calls to the UDF's
callbacks.
<p>T must be of a type which can be legally stored as a value in
java.util.HashMap<KeyType,T>.
<p>If a given aggregate or window function is called multiple times
in a single SQL statement, e.g. SELECT MYFUNC(A), MYFUNC(B)...,
then the clients need some way of knowing which call is which so
that they can map their state between their various UDF callbacks
and reset it via xFinal(). This class takes care of such
mappings.
<p>This class works by mapping
sqlite3_context.getAggregateContext() to a single piece of
state, of a client-defined type (the T part of this class), which
persists across a "matching set" of the UDF's callbacks.
<p>This class is a helper providing commonly-needed functionality
- it is not required for use with aggregate or window functions.
Client UDFs are free to perform such mappings using custom
approaches. The provided {@link AggregateFunction} and {@link
WindowFunction} classes use this.
*/
public static final class PerContextState<T> {
private final java.util.Map<Long,ValueHolder<T>> map
= new java.util.HashMap<>();
/**
Should be called from a UDF's xStep(), xValue(), and xInverse()
methods, passing it that method's first argument and an initial
value for the persistent state. If there is currently no
mapping for the given context within the map, one is created
using the given initial value, else the existing one is used
and the 2nd argument is ignored. It returns a ValueHolder<T>
which can be used to modify that state directly without
requiring that the client update the underlying map's entry.
<p>The caller is obligated to eventually call
takeAggregateState() to clear the mapping.
*/
public ValueHolder<T> getAggregateState(sqlite3_context cx, T initialValue){
final Long key = cx.getAggregateContext(true);
ValueHolder<T> rc = null==key ? null : map.get(key);
if( null==rc ){
map.put(key, rc = new ValueHolder<>(initialValue));
}
return rc;
}
/**
Should be called from a UDF's xFinal() method and passed that
method's first argument. This function removes the value
associated with cx.getAggregateContext() from the map and
returns it, returning null if no other UDF method has been
called to set up such a mapping. The latter condition will be
the case if a UDF is used in a statement which has no result
rows.
*/
public T takeAggregateState(sqlite3_context cx){
final ValueHolder<T> h = map.remove(cx.getAggregateContext(false));
return null==h ? null : h.value;
}
}
/** Per-invocation state for the UDF. */
private final PerContextState<T> map = new PerContextState<>();
/**
To be called from the implementation's xStep() method, as well
as the xValue() and xInverse() methods of the {@link WindowFunction}
subclass, to fetch the current per-call UDF state. On the
first call to this method for any given sqlite3_context
argument, the context is set to the given initial value. On all other
calls, the 2nd argument is ignored.
@see SQLFunction.PerContextState#getAggregateState
*/
protected final ValueHolder<T> getAggregateState(sqlite3_context cx, T initialValue){
return map.getAggregateState(cx, initialValue);
}
/**
To be called from the implementation's xFinal() method to fetch
the final state of the UDF and remove its mapping.
see SQLFunction.PerContextState#takeAggregateState
*/
protected final T takeAggregateState(sqlite3_context cx){
return map.takeAggregateState(cx);
}
}
@@ -0,0 +1,29 @@
/*
** 2023-08-25
**
** The author disclaims copyright to this source code. In place of
** a legal notice, here is a blessing:
**
** May you do good and not evil.
** May you find forgiveness for yourself and forgive others.
** May you share freely, never taking more than you give.
**
*************************************************************************
** This file is part of the JNI bindings for the sqlite3 C API.
*/
package org.sqlite.jni.capi;
import org.sqlite.jni.annotation.*;
/**
Callback for use with {@link CApi#sqlite3_set_authorizer}.
*/
public interface AuthorizerCallback extends CallbackProxy {
/**
Must function as described for the C-level
sqlite3_set_authorizer() callback. If it throws, the error is
converted to a db-level error and the exception is suppressed.
*/
int call(int opId, @Nullable String s1, @Nullable String s2,
@Nullable String s3, @Nullable String s4);
}
@@ -0,0 +1,40 @@
/*
** 2023-08-25
**
** The author disclaims copyright to this source code. In place of
** a legal notice, here is a blessing:
**
** May you do good and not evil.
** May you find forgiveness for yourself and forgive others.
** May you share freely, never taking more than you give.
**
*************************************************************************
** This file is part of the JNI bindings for the sqlite3 C API.
*/
package org.sqlite.jni.capi;
/**
Callback for use with the {@link CApi#sqlite3_auto_extension}
family of APIs.
*/
public interface AutoExtensionCallback extends CallbackProxy {
/**
Must function as described for a C-level
sqlite3_auto_extension() callback.
<p>This callback may throw and the exception's error message will
be set as the db's error string.
<p>Tips for implementations:
<p>- Opening a database from an auto-extension handler will lead to
an endless recursion of the auto-handler triggering itself
indirectly for each newly-opened database.
<p>- If this routine is stateful, it may be useful to make the
overridden method synchronized.
<p>- Results are undefined if the given db is closed by an auto-extension.
*/
int call(sqlite3 db);
}
@@ -0,0 +1,26 @@
/*
** 2023-08-25
**
** The author disclaims copyright to this source code. In place of
** a legal notice, here is a blessing:
**
** May you do good and not evil.
** May you find forgiveness for yourself and forgive others.
** May you share freely, never taking more than you give.
**
*************************************************************************
** This file is part of the JNI bindings for the sqlite3 C API.
*/
package org.sqlite.jni.capi;
/**
Callback for use with {@link CApi#sqlite3_busy_handler}.
*/
public interface BusyHandlerCallback extends CallbackProxy {
/**
Must function as documented for the C-level
sqlite3_busy_handler() callback argument, minus the (void*)
argument the C-level function requires.
*/
int call(int n);
}
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,45 @@
/*
** 2023-08-25
**
** The author disclaims copyright to this source code. In place of
** a legal notice, here is a blessing:
**
** May you do good and not evil.
** May you find forgiveness for yourself and forgive others.
** May you share freely, never taking more than you give.
**
*************************************************************************
** This file is part of the JNI bindings for the sqlite3 C API.
*/
package org.sqlite.jni.capi;
/**
This marker interface exists soley for use as a documentation and
class-grouping tool. It should be applied to interfaces or
classes which have a call() method implementing some specific
callback interface on behalf of the C library.
<p>Unless very explicitely documented otherwise, callbacks must
never throw. Any which do throw but should not might trigger debug
output regarding the error, but the exception will not be
propagated. For callback interfaces which support returning error
info to the core, the JNI binding will convert any exceptions to
C-level error information. For callback interfaces which do not
support returning error information, all exceptions will
necessarily be suppressed in order to retain the C-style no-throw
semantics and avoid invoking undefined behavior in the C layer.
<p>Callbacks of this style follow a common naming convention:
<p>1) They use the UpperCamelCase form of the C function they're
proxying for, minus the {@code sqlite3_} prefix, plus a {@code
Callback} suffix. e.g. {@code sqlite3_busy_handler()}'s callback is
named {@code BusyHandlerCallback}. Exceptions are made where that
would potentially be ambiguous, e.g. {@link ConfigSqllogCallback}
instead of {@code ConfigCallback} because the {@code
sqlite3_config()} interface may need to support more callback types
in the future.
<p>2) They all have a {@code call()} method but its signature is
callback-specific.
*/
public interface CallbackProxy {}
@@ -0,0 +1,35 @@
/*
** 2023-08-25
**
** The author disclaims copyright to this source code. In place of
** a legal notice, here is a blessing:
**
** May you do good and not evil.
** May you find forgiveness for yourself and forgive others.
** May you share freely, never taking more than you give.
**
*************************************************************************
** This file is part of the JNI bindings for the sqlite3 C API.
*/
package org.sqlite.jni.capi;
import org.sqlite.jni.annotation.NotNull;
/**
Callback for use with {@link CApi#sqlite3_create_collation}.
@see AbstractCollationCallback
*/
public interface CollationCallback
extends CallbackProxy, XDestroyCallback {
/**
Must compare the given byte arrays and return the result using
{@code memcmp()} semantics.
*/
int call(@NotNull byte[] lhs, @NotNull byte[] rhs);
/**
Called by SQLite when the collation is destroyed. If a collation
requires custom cleanup, override this method.
*/
void xDestroy();
}
@@ -0,0 +1,29 @@
/*
** 2023-08-25
**
** The author disclaims copyright to this source code. In place of
** a legal notice, here is a blessing:
**
** May you do good and not evil.
** May you find forgiveness for yourself and forgive others.
** May you share freely, never taking more than you give.
**
*************************************************************************
** This file is part of the JNI bindings for the sqlite3 C API.
*/
package org.sqlite.jni.capi;
/**
Callback for use with {@link CApi#sqlite3_collation_needed}.
*/
public interface CollationNeededCallback extends CallbackProxy {
/**
Has the same semantics as the C-level sqlite3_create_collation()
callback.
<p>Because the C API has no mechanism for reporting errors
from this callbacks, any exceptions thrown by this callback
are suppressed.
*/
void call(sqlite3 db, int eTextRep, String collationName);
}
@@ -0,0 +1,26 @@
/*
** 2023-08-25
**
** The author disclaims copyright to this source code. In place of
** a legal notice, here is a blessing:
**
** May you do good and not evil.
** May you find forgiveness for yourself and forgive others.
** May you share freely, never taking more than you give.
**
*************************************************************************
** This file is part of the JNI bindings for the sqlite3 C API.
*/
package org.sqlite.jni.capi;
/**
Callback for use with {@link CApi#sqlite3_commit_hook}.
*/
public interface CommitHookCallback extends CallbackProxy {
/**
Works as documented for the C-level sqlite3_commit_hook()
callback. If it throws, the exception is translated into
a db-level error.
*/
int call();
}
@@ -0,0 +1,25 @@
/*
** 2023-08-23
**
** The author disclaims copyright to this source code. In place of
** a legal notice, here is a blessing:
**
** May you do good and not evil.
** May you find forgiveness for yourself and forgive others.
** May you share freely, never taking more than you give.
**
*************************************************************************
** This file is part of the JNI bindings for the sqlite3 C API.
*/
package org.sqlite.jni.capi;
/**
A callback for use with sqlite3_config().
*/
public interface ConfigLogCallback {
/**
Must function as described for a C-level callback for
{@link CApi#sqlite3_config(ConfigLogCallback)}, with the slight signature change.
*/
void call(int errCode, String msg);
}
@@ -0,0 +1,25 @@
/*
** 2023-08-23
**
** The author disclaims copyright to this source code. In place of
** a legal notice, here is a blessing:
**
** May you do good and not evil.
** May you find forgiveness for yourself and forgive others.
** May you share freely, never taking more than you give.
**
*************************************************************************
** This file is part of the JNI bindings for the sqlite3 C API.
*/
package org.sqlite.jni.capi;
/**
A callback for use with sqlite3_config().
*/
public interface ConfigSqlLogCallback {
/**
Must function as described for a C-level callback for
{@link CApi#sqlite3_config(ConfigSqlLogCallback)}, with the slight signature change.
*/
void call(sqlite3 db, String msg, int msgType );
}
@@ -0,0 +1,46 @@
/*
** 2023-07-21
**
** The author disclaims copyright to this source code. In place of
** a legal notice, here is a blessing:
**
** May you do good and not evil.
** May you find forgiveness for yourself and forgive others.
** May you share freely, never taking more than you give.
**
*************************************************************************
** This file is part of the JNI bindings for the sqlite3 C API.
*/
package org.sqlite.jni.capi;
/**
A helper for passing pointers between JNI C code and Java, in
particular for output pointers of high-level object types in the
sqlite3 C API, e.g. (sqlite3**) and (sqlite3_stmt**). This is
intended to be subclassed and the ContextType is intended to be the
class which is doing the subclassing. The intent of the ContextType
is strictly to provide some level of type safety by avoiding that
NativePointerHolder is not inadvertently passed to an incompatible
function signature.
These objects do not own the pointer they refer to. They are
intended simply to communicate that pointer between C and Java.
*/
public class NativePointerHolder<ContextType> {
//! Only set from JNI, where access permissions don't matter.
private volatile long nativePointer = 0;
/**
For use ONLY by package-level APIs which act as proxies for
close/finalize operations. Such ops must call this to zero out
the pointer so that this object is not carrying a stale
pointer. This function returns the prior value of the pointer and
sets it to 0.
*/
final long clearNativePointer() {
final long rv = nativePointer;
nativePointer= 0;
return rv;
}
public final long getNativePointer(){ return nativePointer; }
}
@@ -0,0 +1,253 @@
/*
** 2023-07-21
**
** The author disclaims copyright to this source code. In place of
** a legal notice, here is a blessing:
**
** May you do good and not evil.
** May you find forgiveness for yourself and forgive others.
** May you share freely, never taking more than you give.
**
*************************************************************************
** This file is part of the JNI bindings for the sqlite3 C API.
*/
package org.sqlite.jni.capi;
/**
Helper classes for handling JNI output pointers.
<p>We do not use a generic OutputPointer<T> because working with those
from the native JNI code is unduly quirky due to a lack of
autoboxing at that level.
<p>The usage is similar for all of thes types:
<pre>{@code
OutputPointer.sqlite3 out = new OutputPointer.sqlite3();
assert( null==out.get() );
int rc = sqlite3_open(":memory:", out);
if( 0!=rc ) ... error;
assert( null!=out.get() );
sqlite3 db = out.take();
assert( null==out.get() );
}</pre>
<p>With the minor exception that the primitive types permit direct
access to the object's value via the `value` property, whereas the
JNI-level opaque types do not permit client-level code to set that
property.
<p>Warning: do not share instances of these classes across
threads. Doing so may lead to corrupting sqlite3-internal state.
*/
public final class OutputPointer {
/**
Output pointer for use with routines, such as sqlite3_open(),
which return a database handle via an output pointer. These
pointers can only be set by the JNI layer, not by client-level
code.
*/
public static final class sqlite3 {
private org.sqlite.jni.capi.sqlite3 value;
/** Initializes with a null value. */
public sqlite3(){value = null;}
/** Sets the current value to null. */
public void clear(){value = null;}
/** Returns the current value. */
public final org.sqlite.jni.capi.sqlite3 get(){return value;}
/** Equivalent to calling get() then clear(). */
public final org.sqlite.jni.capi.sqlite3 take(){
final org.sqlite.jni.capi.sqlite3 v = value;
value = null;
return v;
}
}
/**
Output pointer for sqlite3_blob_open(). These
pointers can only be set by the JNI layer, not by client-level
code.
*/
public static final class sqlite3_blob {
private org.sqlite.jni.capi.sqlite3_blob value;
/** Initializes with a null value. */
public sqlite3_blob(){value = null;}
/** Sets the current value to null. */
public void clear(){value = null;}
/** Returns the current value. */
public final org.sqlite.jni.capi.sqlite3_blob get(){return value;}
/** Equivalent to calling get() then clear(). */
public final org.sqlite.jni.capi.sqlite3_blob take(){
final org.sqlite.jni.capi.sqlite3_blob v = value;
value = null;
return v;
}
}
/**
Output pointer for use with routines, such as sqlite3_prepare(),
which return a statement handle via an output pointer. These
pointers can only be set by the JNI layer, not by client-level
code.
*/
public static final class sqlite3_stmt {
private org.sqlite.jni.capi.sqlite3_stmt value;
/** Initializes with a null value. */
public sqlite3_stmt(){value = null;}
/** Sets the current value to null. */
public void clear(){value = null;}
/** Returns the current value. */
public final org.sqlite.jni.capi.sqlite3_stmt get(){return value;}
/** Equivalent to calling get() then clear(). */
public final org.sqlite.jni.capi.sqlite3_stmt take(){
final org.sqlite.jni.capi.sqlite3_stmt v = value;
value = null;
return v;
}
}
/**
Output pointer for use with routines, such as sqlite3_prepupdate_new(),
which return a sqlite3_value handle via an output pointer. These
pointers can only be set by the JNI layer, not by client-level
code.
*/
public static final class sqlite3_value {
private org.sqlite.jni.capi.sqlite3_value value;
/** Initializes with a null value. */
public sqlite3_value(){value = null;}
/** Sets the current value to null. */
public void clear(){value = null;}
/** Returns the current value. */
public final org.sqlite.jni.capi.sqlite3_value get(){return value;}
/** Equivalent to calling get() then clear(). */
public final org.sqlite.jni.capi.sqlite3_value take(){
final org.sqlite.jni.capi.sqlite3_value v = value;
value = null;
return v;
}
}
/**
Output pointer for use with native routines which return booleans
via integer output pointers.
*/
public static final class Bool {
/**
This is public for ease of use. Accessors are provided for
consistency with the higher-level types.
*/
public boolean value;
/** Initializes with the value 0. */
public Bool(){this(false);}
/** Initializes with the value v. */
public Bool(boolean v){value = v;}
/** Returns the current value. */
public final boolean get(){return value;}
/** Sets the current value to v. */
public final void set(boolean v){value = v;}
}
/**
Output pointer for use with native routines which return integers via
output pointers.
*/
public static final class Int32 {
/**
This is public for ease of use. Accessors are provided for
consistency with the higher-level types.
*/
public int value;
/** Initializes with the value 0. */
public Int32(){this(0);}
/** Initializes with the value v. */
public Int32(int v){value = v;}
/** Returns the current value. */
public final int get(){return value;}
/** Sets the current value to v. */
public final void set(int v){value = v;}
}
/**
Output pointer for use with native routines which return 64-bit integers
via output pointers.
*/
public static final class Int64 {
/**
This is public for ease of use. Accessors are provided for
consistency with the higher-level types.
*/
public long value;
/** Initializes with the value 0. */
public Int64(){this(0);}
/** Initializes with the value v. */
public Int64(long v){value = v;}
/** Returns the current value. */
public final long get(){return value;}
/** Sets the current value. */
public final void set(long v){value = v;}
}
/**
Output pointer for use with native routines which return strings via
output pointers.
*/
public static final class String {
/**
This is public for ease of use. Accessors are provided for
consistency with the higher-level types.
*/
public java.lang.String value;
/** Initializes with a null value. */
public String(){this(null);}
/** Initializes with the value v. */
public String(java.lang.String v){value = v;}
/** Returns the current value. */
public final java.lang.String get(){return value;}
/** Sets the current value. */
public final void set(java.lang.String v){value = v;}
}
/**
Output pointer for use with native routines which return byte
arrays via output pointers.
*/
public static final class ByteArray {
/**
This is public for ease of use. Accessors are provided for
consistency with the higher-level types.
*/
public byte[] value;
/** Initializes with the value null. */
public ByteArray(){this(null);}
/** Initializes with the value v. */
public ByteArray(byte[] v){value = v;}
/** Returns the current value. */
public final byte[] get(){return value;}
/** Sets the current value. */
public final void set(byte[] v){value = v;}
}
/**
Output pointer for use with native routines which return
blobs via java.nio.ByteBuffer.
See {@link org.sqlite.jni.capi.CApi#sqlite3_jni_supports_nio}
*/
public static final class ByteBuffer {
/**
This is public for ease of use. Accessors are provided for
consistency with the higher-level types.
*/
public java.nio.ByteBuffer value;
/** Initializes with the value null. */
public ByteBuffer(){this(null);}
/** Initializes with the value v. */
public ByteBuffer(java.nio.ByteBuffer v){value = v;}
/** Returns the current value. */
public final java.nio.ByteBuffer get(){return value;}
/** Sets the current value. */
public final void set(java.nio.ByteBuffer v){value = v;}
}
}
@@ -0,0 +1,81 @@
/*
** 2023-09-13
**
** The author disclaims copyright to this source code. In place of
** a legal notice, here is a blessing:
**
** May you do good and not evil.
** May you find forgiveness for yourself and forgive others.
** May you share freely, never taking more than you give.
**
*************************************************************************
** This file is part of the JNI bindings for the sqlite3 C API.
*/
package org.sqlite.jni.capi;
/**
Callback for use with {@link CApi#sqlite3_prepare_multi}.
*/
public interface PrepareMultiCallback extends CallbackProxy {
/**
Gets passed a sqlite3_stmt which it may handle in arbitrary ways,
transfering ownership of it to this function.
sqlite3_prepare_multi() will _not_ finalize st - it is up
to the call() implementation how st is handled.
Must return 0 on success or an SQLITE_... code on error. If it
throws, sqlite3_prepare_multi() will transform the exception into
a db-level error in order to retain the C-style error semantics
of the API.
See the {@link Finalize} class for a wrapper which finalizes the
statement after calling a proxy PrepareMultiCallback.
*/
int call(sqlite3_stmt st);
/**
A PrepareMultiCallback impl which wraps a separate impl and finalizes
any sqlite3_stmt passed to its callback.
*/
public static final class Finalize implements PrepareMultiCallback {
private final PrepareMultiCallback p;
/**
p is the proxy to call() when this.call() is called.
*/
public Finalize( PrepareMultiCallback p ){
this.p = p;
}
/**
Calls the call() method of the proxied callback and either returns its
result or propagates an exception. Either way, it passes its argument to
sqlite3_finalize() before returning.
*/
@Override public int call(sqlite3_stmt st){
try {
return this.p.call(st);
}finally{
CApi.sqlite3_finalize(st);
}
}
}
/**
A PrepareMultiCallback impl which steps entirely through a result set,
ignoring all non-error results.
*/
public static final class StepAll implements PrepareMultiCallback {
public StepAll(){}
/**
Calls sqlite3_step() on st until it returns something other than
SQLITE_ROW. If the final result is SQLITE_DONE then 0 is returned,
else the result of the final step is returned.
*/
@Override public int call(sqlite3_stmt st){
int rc = CApi.SQLITE_DONE;
while( CApi.SQLITE_ROW == (rc = CApi.sqlite3_step(st)) ){}
return CApi.SQLITE_DONE==rc ? 0 : rc;
}
}
}
@@ -0,0 +1,27 @@
/*
** 2023-08-25
**
** The author disclaims copyright to this source code. In place of
** a legal notice, here is a blessing:
**
** May you do good and not evil.
** May you find forgiveness for yourself and forgive others.
** May you share freely, never taking more than you give.
**
*************************************************************************
** This file is part of the JNI bindings for the sqlite3 C API.
*/
package org.sqlite.jni.capi;
/**
Callback for use with {@link CApi#sqlite3_preupdate_hook}.
*/
public interface PreupdateHookCallback extends CallbackProxy {
/**
Must function as described for the C-level sqlite3_preupdate_hook()
callback. If it throws, the exception is translated to a
db-level error and the exception is suppressed.
*/
void call(sqlite3 db, int op, String dbName, String dbTable,
long iKey1, long iKey2 );
}
@@ -0,0 +1,27 @@
/*
** 2023-08-25
**
** The author disclaims copyright to this source code. In place of
** a legal notice, here is a blessing:
**
** May you do good and not evil.
** May you find forgiveness for yourself and forgive others.
** May you share freely, never taking more than you give.
**
*************************************************************************
** This file is part of the JNI bindings for the sqlite3 C API.
*/
package org.sqlite.jni.capi;
/**
Callback for use with {@link CApi#sqlite3_progress_handler}.
*/
public interface ProgressHandlerCallback extends CallbackProxy {
/**
Works as documented for the C-level sqlite3_progress_handler() callback.
<p>If it throws, the exception message is passed on to the db and
the exception is suppressed.
*/
int call();
}
@@ -0,0 +1,155 @@
/*
** 2023-07-21
**
** The author disclaims copyright to this source code. In place of
** a legal notice, here is a blessing:
**
** May you do good and not evil.
** May you find forgiveness for yourself and forgive others.
** May you share freely, never taking more than you give.
**
*************************************************************************
** This file is part of the JNI bindings for the sqlite3 C API.
*/
package org.sqlite.jni.capi;
/**
This enum contains all of the core and "extended" result codes used
by the sqlite3 library. It is provided not for use with the C-style
API (with which it won't work) but for higher-level code which may
find it useful to map SQLite result codes to human-readable names.
*/
public enum ResultCode {
SQLITE_OK(CApi.SQLITE_OK),
SQLITE_ERROR(CApi.SQLITE_ERROR),
SQLITE_INTERNAL(CApi.SQLITE_INTERNAL),
SQLITE_PERM(CApi.SQLITE_PERM),
SQLITE_ABORT(CApi.SQLITE_ABORT),
SQLITE_BUSY(CApi.SQLITE_BUSY),
SQLITE_LOCKED(CApi.SQLITE_LOCKED),
SQLITE_NOMEM(CApi.SQLITE_NOMEM),
SQLITE_READONLY(CApi.SQLITE_READONLY),
SQLITE_INTERRUPT(CApi.SQLITE_INTERRUPT),
SQLITE_IOERR(CApi.SQLITE_IOERR),
SQLITE_CORRUPT(CApi.SQLITE_CORRUPT),
SQLITE_NOTFOUND(CApi.SQLITE_NOTFOUND),
SQLITE_FULL(CApi.SQLITE_FULL),
SQLITE_CANTOPEN(CApi.SQLITE_CANTOPEN),
SQLITE_PROTOCOL(CApi.SQLITE_PROTOCOL),
SQLITE_EMPTY(CApi.SQLITE_EMPTY),
SQLITE_SCHEMA(CApi.SQLITE_SCHEMA),
SQLITE_TOOBIG(CApi.SQLITE_TOOBIG),
SQLITE_CONSTRAINT(CApi.SQLITE_CONSTRAINT),
SQLITE_MISMATCH(CApi.SQLITE_MISMATCH),
SQLITE_MISUSE(CApi.SQLITE_MISUSE),
SQLITE_NOLFS(CApi.SQLITE_NOLFS),
SQLITE_AUTH(CApi.SQLITE_AUTH),
SQLITE_FORMAT(CApi.SQLITE_FORMAT),
SQLITE_RANGE(CApi.SQLITE_RANGE),
SQLITE_NOTADB(CApi.SQLITE_NOTADB),
SQLITE_NOTICE(CApi.SQLITE_NOTICE),
SQLITE_WARNING(CApi.SQLITE_WARNING),
SQLITE_ROW(CApi.SQLITE_ROW),
SQLITE_DONE(CApi.SQLITE_DONE),
SQLITE_ERROR_MISSING_COLLSEQ(CApi.SQLITE_ERROR_MISSING_COLLSEQ),
SQLITE_ERROR_RETRY(CApi.SQLITE_ERROR_RETRY),
SQLITE_ERROR_SNAPSHOT(CApi.SQLITE_ERROR_SNAPSHOT),
SQLITE_IOERR_READ(CApi.SQLITE_IOERR_READ),
SQLITE_IOERR_SHORT_READ(CApi.SQLITE_IOERR_SHORT_READ),
SQLITE_IOERR_WRITE(CApi.SQLITE_IOERR_WRITE),
SQLITE_IOERR_FSYNC(CApi.SQLITE_IOERR_FSYNC),
SQLITE_IOERR_DIR_FSYNC(CApi.SQLITE_IOERR_DIR_FSYNC),
SQLITE_IOERR_TRUNCATE(CApi.SQLITE_IOERR_TRUNCATE),
SQLITE_IOERR_FSTAT(CApi.SQLITE_IOERR_FSTAT),
SQLITE_IOERR_UNLOCK(CApi.SQLITE_IOERR_UNLOCK),
SQLITE_IOERR_RDLOCK(CApi.SQLITE_IOERR_RDLOCK),
SQLITE_IOERR_DELETE(CApi.SQLITE_IOERR_DELETE),
SQLITE_IOERR_BLOCKED(CApi.SQLITE_IOERR_BLOCKED),
SQLITE_IOERR_NOMEM(CApi.SQLITE_IOERR_NOMEM),
SQLITE_IOERR_ACCESS(CApi.SQLITE_IOERR_ACCESS),
SQLITE_IOERR_CHECKRESERVEDLOCK(CApi.SQLITE_IOERR_CHECKRESERVEDLOCK),
SQLITE_IOERR_LOCK(CApi.SQLITE_IOERR_LOCK),
SQLITE_IOERR_CLOSE(CApi.SQLITE_IOERR_CLOSE),
SQLITE_IOERR_DIR_CLOSE(CApi.SQLITE_IOERR_DIR_CLOSE),
SQLITE_IOERR_SHMOPEN(CApi.SQLITE_IOERR_SHMOPEN),
SQLITE_IOERR_SHMSIZE(CApi.SQLITE_IOERR_SHMSIZE),
SQLITE_IOERR_SHMLOCK(CApi.SQLITE_IOERR_SHMLOCK),
SQLITE_IOERR_SHMMAP(CApi.SQLITE_IOERR_SHMMAP),
SQLITE_IOERR_SEEK(CApi.SQLITE_IOERR_SEEK),
SQLITE_IOERR_DELETE_NOENT(CApi.SQLITE_IOERR_DELETE_NOENT),
SQLITE_IOERR_MMAP(CApi.SQLITE_IOERR_MMAP),
SQLITE_IOERR_GETTEMPPATH(CApi.SQLITE_IOERR_GETTEMPPATH),
SQLITE_IOERR_CONVPATH(CApi.SQLITE_IOERR_CONVPATH),
SQLITE_IOERR_VNODE(CApi.SQLITE_IOERR_VNODE),
SQLITE_IOERR_AUTH(CApi.SQLITE_IOERR_AUTH),
SQLITE_IOERR_BEGIN_ATOMIC(CApi.SQLITE_IOERR_BEGIN_ATOMIC),
SQLITE_IOERR_COMMIT_ATOMIC(CApi.SQLITE_IOERR_COMMIT_ATOMIC),
SQLITE_IOERR_ROLLBACK_ATOMIC(CApi.SQLITE_IOERR_ROLLBACK_ATOMIC),
SQLITE_IOERR_DATA(CApi.SQLITE_IOERR_DATA),
SQLITE_IOERR_CORRUPTFS(CApi.SQLITE_IOERR_CORRUPTFS),
SQLITE_LOCKED_SHAREDCACHE(CApi.SQLITE_LOCKED_SHAREDCACHE),
SQLITE_LOCKED_VTAB(CApi.SQLITE_LOCKED_VTAB),
SQLITE_BUSY_RECOVERY(CApi.SQLITE_BUSY_RECOVERY),
SQLITE_BUSY_SNAPSHOT(CApi.SQLITE_BUSY_SNAPSHOT),
SQLITE_BUSY_TIMEOUT(CApi.SQLITE_BUSY_TIMEOUT),
SQLITE_CANTOPEN_NOTEMPDIR(CApi.SQLITE_CANTOPEN_NOTEMPDIR),
SQLITE_CANTOPEN_ISDIR(CApi.SQLITE_CANTOPEN_ISDIR),
SQLITE_CANTOPEN_FULLPATH(CApi.SQLITE_CANTOPEN_FULLPATH),
SQLITE_CANTOPEN_CONVPATH(CApi.SQLITE_CANTOPEN_CONVPATH),
SQLITE_CANTOPEN_SYMLINK(CApi.SQLITE_CANTOPEN_SYMLINK),
SQLITE_CORRUPT_VTAB(CApi.SQLITE_CORRUPT_VTAB),
SQLITE_CORRUPT_SEQUENCE(CApi.SQLITE_CORRUPT_SEQUENCE),
SQLITE_CORRUPT_INDEX(CApi.SQLITE_CORRUPT_INDEX),
SQLITE_READONLY_RECOVERY(CApi.SQLITE_READONLY_RECOVERY),
SQLITE_READONLY_CANTLOCK(CApi.SQLITE_READONLY_CANTLOCK),
SQLITE_READONLY_ROLLBACK(CApi.SQLITE_READONLY_ROLLBACK),
SQLITE_READONLY_DBMOVED(CApi.SQLITE_READONLY_DBMOVED),
SQLITE_READONLY_CANTINIT(CApi.SQLITE_READONLY_CANTINIT),
SQLITE_READONLY_DIRECTORY(CApi.SQLITE_READONLY_DIRECTORY),
SQLITE_ABORT_ROLLBACK(CApi.SQLITE_ABORT_ROLLBACK),
SQLITE_CONSTRAINT_CHECK(CApi.SQLITE_CONSTRAINT_CHECK),
SQLITE_CONSTRAINT_COMMITHOOK(CApi.SQLITE_CONSTRAINT_COMMITHOOK),
SQLITE_CONSTRAINT_FOREIGNKEY(CApi.SQLITE_CONSTRAINT_FOREIGNKEY),
SQLITE_CONSTRAINT_FUNCTION(CApi.SQLITE_CONSTRAINT_FUNCTION),
SQLITE_CONSTRAINT_NOTNULL(CApi.SQLITE_CONSTRAINT_NOTNULL),
SQLITE_CONSTRAINT_PRIMARYKEY(CApi.SQLITE_CONSTRAINT_PRIMARYKEY),
SQLITE_CONSTRAINT_TRIGGER(CApi.SQLITE_CONSTRAINT_TRIGGER),
SQLITE_CONSTRAINT_UNIQUE(CApi.SQLITE_CONSTRAINT_UNIQUE),
SQLITE_CONSTRAINT_VTAB(CApi.SQLITE_CONSTRAINT_VTAB),
SQLITE_CONSTRAINT_ROWID(CApi.SQLITE_CONSTRAINT_ROWID),
SQLITE_CONSTRAINT_PINNED(CApi.SQLITE_CONSTRAINT_PINNED),
SQLITE_CONSTRAINT_DATATYPE(CApi.SQLITE_CONSTRAINT_DATATYPE),
SQLITE_NOTICE_RECOVER_WAL(CApi.SQLITE_NOTICE_RECOVER_WAL),
SQLITE_NOTICE_RECOVER_ROLLBACK(CApi.SQLITE_NOTICE_RECOVER_ROLLBACK),
SQLITE_WARNING_AUTOINDEX(CApi.SQLITE_WARNING_AUTOINDEX),
SQLITE_AUTH_USER(CApi.SQLITE_AUTH_USER),
SQLITE_OK_LOAD_PERMANENTLY(CApi.SQLITE_OK_LOAD_PERMANENTLY);
public final int value;
ResultCode(int rc){
value = rc;
ResultCodeMap.set(rc, this);
}
/**
Returns the entry from this enum for the given result code, or
null if no match is found.
*/
public static ResultCode getEntryForInt(int rc){
return ResultCodeMap.get(rc);
}
/**
Internal level of indirection required because we cannot initialize
static enum members in an enum before the enum constructor is
invoked.
*/
private static final class ResultCodeMap {
private static final java.util.Map<Integer,ResultCode> i2e
= new java.util.HashMap<>();
private static void set(int rc, ResultCode e){ i2e.put(rc, e); }
private static ResultCode get(int rc){ return i2e.get(rc); }
}
}
@@ -0,0 +1,26 @@
/*
** 2023-08-25
**
** The author disclaims copyright to this source code. In place of
** a legal notice, here is a blessing:
**
** May you do good and not evil.
** May you find forgiveness for yourself and forgive others.
** May you share freely, never taking more than you give.
**
*************************************************************************
** This file is part of the JNI bindings for the sqlite3 C API.
*/
package org.sqlite.jni.capi;
/**
Callback for use with {@link CApi#sqlite3_rollback_hook}.
*/
public interface RollbackHookCallback extends CallbackProxy {
/**
Must function as documented for the C-level sqlite3_rollback_hook()
callback. If it throws, the exception is translated into
a db-level error.
*/
void call();
}
@@ -0,0 +1,36 @@
/*
** 2023-07-22
**
** The author disclaims copyright to this source code. In place of
** a legal notice, here is a blessing:
**
** May you do good and not evil.
** May you find forgiveness for yourself and forgive others.
** May you share freely, never taking more than you give.
**
*************************************************************************
** This file is part of the JNI bindings for the sqlite3 C API.
*/
package org.sqlite.jni.capi;
/**
SQLFunction is used in conjunction with the
sqlite3_create_function() JNI-bound API to give that native code
access to the callback functions needed in order to implement SQL
functions in Java.
<p>
This class is not used by itself, but is a marker base class. The
three UDF types are modelled by the inner classes Scalar,
Aggregate<T>, and Window<T>. Most simply, clients may subclass
those, or create anonymous classes from them, to implement
UDFs. Clients are free to create their own classes for use with
UDFs, so long as they conform to the public interfaces defined by
those three classes. The JNI layer only actively relies on the
SQLFunction base class and the method names and signatures used by
the UDF callback interfaces.
*/
public interface SQLFunction {
}
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,33 @@
/*
** 2023-08-25
**
** The author disclaims copyright to this source code. In place of
** a legal notice, here is a blessing:
**
** May you do good and not evil.
** May you find forgiveness for yourself and forgive others.
** May you share freely, never taking more than you give.
**
*************************************************************************
** This file is part of the JNI bindings for the sqlite3 C API.
*/
package org.sqlite.jni.capi;
/**
A SQLFunction implementation for scalar functions.
*/
public abstract class ScalarFunction implements SQLFunction {
/**
As for the xFunc() argument of the C API's
sqlite3_create_function(). If this function throws, it is
translated into an sqlite3_result_error().
*/
public abstract void xFunc(sqlite3_context cx, sqlite3_value[] args);
/**
Optionally override to be notified when the UDF is finalized by
SQLite. This default implementation does nothing.
*/
public void xDestroy() {}
}
@@ -0,0 +1,35 @@
/*
** 2023-07-21
**
** The author disclaims copyright to this source code. In place of
** a legal notice, here is a blessing:
**
** May you do good and not evil.
** May you find forgiveness for yourself and forgive others.
** May you share freely, never taking more than you give.
**
*************************************************************************
** This file is part of the JNI bindings for the sqlite3 C API.
*/
package org.sqlite.jni.capi;
/**
A wrapper object for use with sqlite3_table_column_metadata().
They are populated only via that interface.
*/
public final class TableColumnMetadata {
OutputPointer.Bool pNotNull = new OutputPointer.Bool();
OutputPointer.Bool pPrimaryKey = new OutputPointer.Bool();
OutputPointer.Bool pAutoinc = new OutputPointer.Bool();
OutputPointer.String pzCollSeq = new OutputPointer.String();
OutputPointer.String pzDataType = new OutputPointer.String();
public TableColumnMetadata(){
}
public String getDataType(){ return pzDataType.value; }
public String getCollation(){ return pzCollSeq.value; }
public boolean isNotNull(){ return pNotNull.value; }
public boolean isPrimaryKey(){ return pPrimaryKey.value; }
public boolean isAutoincrement(){ return pAutoinc.value; }
}
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,50 @@
/*
** 2023-08-25
**
** The author disclaims copyright to this source code. In place of
** a legal notice, here is a blessing:
**
** May you do good and not evil.
** May you find forgiveness for yourself and forgive others.
** May you share freely, never taking more than you give.
**
*************************************************************************
** This file is part of the JNI bindings for the sqlite3 C API.
*/
package org.sqlite.jni.capi;
import org.sqlite.jni.annotation.Nullable;
/**
Callback for use with {@link CApi#sqlite3_trace_v2}.
*/
public interface TraceV2Callback extends CallbackProxy {
/**
Called by sqlite3 for various tracing operations, as per
sqlite3_trace_v2(). Note that this interface elides the 2nd
argument to the native trace callback, as that role is better
filled by instance-local state.
<p>These callbacks may throw, in which case their exceptions are
converted to C-level error information.
<p>The 2nd argument to this function, if non-null, will be a an
sqlite3 or sqlite3_stmt object, depending on the first argument
(see below).
<p>The final argument to this function is the "X" argument
documented for sqlite3_trace() and sqlite3_trace_v2(). Its type
depends on value of the first argument:
<p>- SQLITE_TRACE_STMT: pNative is a sqlite3_stmt. pX is a String
containing the prepared SQL.
<p>- SQLITE_TRACE_PROFILE: pNative is a sqlite3_stmt. pX is a Long
holding an approximate number of nanoseconds the statement took
to run.
<p>- SQLITE_TRACE_ROW: pNative is a sqlite3_stmt. pX is null.
<p>- SQLITE_TRACE_CLOSE: pNative is a sqlite3. pX is null.
*/
int call(int traceFlag, Object pNative, @Nullable Object pX);
}
@@ -0,0 +1,26 @@
/*
** 2023-08-25
**
** The author disclaims copyright to this source code. In place of
** a legal notice, here is a blessing:
**
** May you do good and not evil.
** May you find forgiveness for yourself and forgive others.
** May you share freely, never taking more than you give.
**
*************************************************************************
** This file is part of the JNI bindings for the sqlite3 C API.
*/
package org.sqlite.jni.capi;
/**
Callback for use with {@link CApi#sqlite3_update_hook}.
*/
public interface UpdateHookCallback extends CallbackProxy {
/**
Must function as described for the C-level sqlite3_update_hook()
callback. If it throws, the exception is translated into
a db-level error.
*/
void call(int opId, String dbName, String tableName, long rowId);
}

Some files were not shown because too many files have changed in this diff Show More