Compare commits

...

4811 Commits

Author SHA1 Message Date
dan 9e4d355fd0 Ensure that logging is only performed for COMMIT operations that take 2 seconds or longer.
FossilOrigin-Name: 14186d1e8d11176d8e3f05139c7ea5988a9d7ed99384eea6083ce570e2d1cbcc
2024-10-04 14:49:08 +00:00
dan 104ad9aee1 Add further logging intended for use in analyzing COMMIT performance.
FossilOrigin-Name: bbbd3d9be88194039bb5e2862cb14733403d37ec6c8f21fccc8447172c999343
2024-10-04 14:11:12 +00:00
dan 023b035251 Add logging to help analyze COMMIT performance.
FossilOrigin-Name: 72404dfc0d692bc21b3263d04c9c45dda9e61eb13dc87d2014416703bdf537de
2024-10-03 21:00:36 +00:00
dan 855e3792bd Merge the latest trunk enhancements, via the wal2 branch, into the bedrock branch.
FossilOrigin-Name: fd6ff55141be82ba36513ba6273a60c6d8b3c496ed7890ab47cccb05bec7f7c2
2024-09-30 17:57:58 +00:00
dan aa45304859 Fix a usan warning in test_hexio.c on this branch.
FossilOrigin-Name: 59fba0288d8fb68f9b6e4936c09ae9169bc002bfe6548354504563a674000c20
2024-09-30 17:53:47 +00:00
dan e9ef82f592 Merge the latest trunk enhancements, including fix for assert() failure in fts5, into the wal2 branch.
FossilOrigin-Name: ab6356181274016571124989bc10e16256666567bb284612a0a689cf18a50adb
2024-09-30 17:32:08 +00:00
dan d5838eaa42 In fts5, avoid starting a new merge of level L if there exists already an ongoing merge of a level less than L.
FossilOrigin-Name: 350c6e75ce3c1e81458d1baa73045df489284206e8b279ab3c2f5e3d011c262a
2024-09-30 17:28:45 +00:00
drh 5bd2d02612 Merge the latest trunk enhancements into the wal2 branch.
FossilOrigin-Name: 0f9e2fceecdb7fad0a2528a4244a786d17274d11b83f336d6012420675a2e4cd
2024-09-30 14:41:43 +00:00
drh 54fd01c4ab Fix the CLI so that the --bom option only outputs a single BOM, not two.
FossilOrigin-Name: 76b6331e6a705a420a64820a18214f07cf4c1d5151e7158d6fff09964e63f352
2024-09-28 19:52:38 +00:00
stephan 33669ef289 Add another missing mkdir to the wasm build process.
FossilOrigin-Name: e815055b321085deda8607ac3279ef1a1c890fe3bf9d9b9c0a74028e87857a7d
2024-09-28 15:13:49 +00:00
stephan 5ec9ed7fe9 Complete a line of documentation which was interrupted mid-sentence.
FossilOrigin-Name: c9cbbeda3d1ec4215396aaaf94428b829c4f53329431fa61251914c195f8a9a1
2024-09-28 13:51:47 +00:00
stephan 6a8336987a wasm: move the makefile-eval-generated fiddle rules into mkwasmbuilds.c. Squelch some warnings from grep when running 'make clean' on a clean tree.
FossilOrigin-Name: 0485d1360b0dbf9987d16ab1df47b6b7a25595881581799f87889f6d4eeb60f4
2024-09-28 12:01:10 +00:00
stephan d4c735d491 Ensure that the temporary build dir is mkdir'd by wasm deps which depend on it.
FossilOrigin-Name: cabdf9797281090ab3f95c9f4c2a79878d1c64d1f2a09e433e2309abe5e19e59
2024-09-28 11:23:09 +00:00
stephan 51fb37db61 For wasm builds: automatically use higher optimization (but slower build) levels when a target related to deliverables (as opposed to dev mode) is explicitly invoked.
FossilOrigin-Name: 45361ab9744ccac3419d97fe8951838f14bd28220e3f7f747b39e46cfc3b5a1d
2024-09-28 11:03:00 +00:00
stephan fabae7a134 Minor makefile doc fixes. No functional changes.
FossilOrigin-Name: b3ad58e1fc73941f2d32bab0e0fac0b95849ae69f84788a431e65bb072e42db5
2024-09-28 00:39:19 +00:00
stephan 3c58039e9e Merge wasm-build-rework branch into trunk. Summary: 1) add optional bare-bones build, stripped of many optional library features, 2) replace much of the hyper-spaghetti wasm-specific makefile code with a more legible static code generator.
FossilOrigin-Name: 208c27714646c9bc26eef11266086a71da04bc24e87078de0955e7beb68a821e
2024-09-28 00:02:52 +00:00
stephan add19478c1 Fix fiddle build broken by recent build-level reworks. Remove a stale reference to an old CSS file.
FossilOrigin-Name: f904b3b7842da036c3e8423bae02cdf207573c92dc379a601eae289eb3c5f547
2024-09-27 23:45:40 +00:00
stephan c707b2eb6d Squash sign-comparison warnings reported in [forum:5e605a763a65c3f8 | forum post 5e605a763a65c3f8].
FossilOrigin-Name: e74fce93c518296bdb0a4273cd5fd3f785d37d27750ca456b61a3502135775f9
2024-09-27 16:20:03 +00:00
drh c94e4a7b2f Fix a harmless compiler warning in the CLI.
FossilOrigin-Name: 27ef1909bb0c4d9470c6074b40500632c68341127a079a3eb3b6a19dbfb2aeac
2024-09-26 22:25:13 +00:00
drh 74194f14ec Add the ext/misc/sqlite3_stdio.c portability interface and incorporate it
into the CLI, sqldiff, and sqlite3_analyzer.  Enhance the CLI with the new
".www" dot-command and related options on .once and .output.

FossilOrigin-Name: f97f9944b829a49da12786f934da0a5ad51591afd6d8a19a4a0835f51bbdbff2
2024-09-26 19:38:34 +00:00
drh 2b041fb97a Provide SQLITE_U8TEXT_ONLY and SQLITE_U8TEXT_STDIO compile-time options
to the sqlite3_stdio.c module.

FossilOrigin-Name: f31588520e3f45b50dcaa9eecab17f52ebb56bb53d0f9bdb88cc596d1a156353
2024-09-26 19:16:20 +00:00
stephan 2c77d51dfe Resolve a harmless compiler warning in QNX builds.
FossilOrigin-Name: 2916460179c6089375188c6e4e3cff1fca5bbbbb7280a10c919e09e75d1f11f8
2024-09-26 18:13:10 +00:00
drh 2fb055366d Clarification of the meaning of the nByte parameter to sqlite3_prepare().
Comment and documentation change only - no changes to the code.

FossilOrigin-Name: 92d71eee4f3a5edb3877c108d14972d80654982b0de3e635d9d008e9d3b6591f
2024-09-26 13:12:19 +00:00
drh b7ceffdadf Fix shell test cases to account for recent additions to the ".help" output.
FossilOrigin-Name: 54c22bc6165affbea0d1166a0912d19939300ccc4ae33a6b2ed8c510ab61c574
2024-09-26 01:29:22 +00:00
drh 66b3ed318b Always show HTML table headers in ".www" output mode.
FossilOrigin-Name: 660ca5ce6600d897cc2b00b9d39e5d993c1c0e71ec0d5dc706246c053a163281
2024-09-25 16:11:27 +00:00
drh 878db7b099 Add the --plain option to the ".www" dot-command.
FossilOrigin-Name: a9209519f612e66cfe11c89e70efd8285a0185ac0d3e5795846aafbd05d7a21f
2024-09-25 15:55:52 +00:00
drh 7119a6c16b Have the zipfile and fileio extensions use sqlite3_stdio.c when it is
available - such as when those extensions are preloaded into the CLI.

FossilOrigin-Name: 74bbb2b2b4507d9acbd91209a2ce341968e9ff64f3aebe9e817bfe488d39ae03
2024-09-25 15:26:37 +00:00
drh 95f35b64da Redirect timer output just like any other text.
FossilOrigin-Name: 3b5ae21074958788b23ccf449e52fbbad1f81779e07a6ca62ad8395f88a37286
2024-09-25 14:09:47 +00:00
drh 62d96919f6 Improvements to ".www" and ".output -w" so that text that is not part of
query output is shown using <pre>.

FossilOrigin-Name: f8ef65c52305b2180ec56760f5762ac5638584cd504c79d57e86f61736901aa4
2024-09-25 13:56:05 +00:00
drh 2a9254cf2c Add the "www" output mode that include <table> in the HTML output.
Add the ".www" command and the "-w" option to ".once".

FossilOrigin-Name: b06fd9e6bcce09f12c994dc34f329a8d267ea0601bb07c9b00903c5017d55d42
2024-09-25 13:29:03 +00:00
drh 9e59c06fb1 Always include a UTF-8 BOM at the beginning of the output CSV when using
the ".excel" command on Windows, as the actual Excel program requires the
BOM in order to work correctly.

FossilOrigin-Name: 04727fc00207325a76a5d5f20549c00232810ac727dedb70bc9e8112e60b3f56
2024-09-25 12:56:44 +00:00
drh 853520ddbf Fix stray fputs() calls in the CLI.
FossilOrigin-Name: 1ee3fa0f0e87a61b5700abd56547c4acefe999317b0d53d66d890d9e09d8b7d9
2024-09-25 11:38:00 +00:00
drh 38b4158aba Use sqlite3_fgets() instead of fgetc() to end the startup debugging pause
in the CLI.

FossilOrigin-Name: 869b3c05e852a797b5801a81d30a1f7955c31afad0274327f7af2284b555f23a
2024-09-25 11:27:59 +00:00
drh 8f1f9ef09f Merge trunk enhancements, and especially the zero- and double-width character
processing for columnar outputs into the cli-stdlib branch.

FossilOrigin-Name: d6262a4bcd7b0334b23ae4d194ca2bce7c6554ee6c678666b24c830e4c14a467
2024-09-25 09:43:05 +00:00
drh 8cd30e3f5b In the CLI, for columnar output formats, try to account for the presence of
zero-width and double-width characters in the output and adjust column widths
accordingly.

FossilOrigin-Name: 9592b9ba3ad7a842cdd4c4010da278485a6fdec7e811bda01ebe640162a8c3b6
2024-09-25 09:39:11 +00:00
drh f4382b2415 Improvements to comments on data structures and subroutines.
FossilOrigin-Name: b217e3004b58af0e777726bdd652b999ad41815261299ef4ce8f8d2f6b0afe8d
2024-09-25 09:33:14 +00:00
stephan 613c2ee21f Minor wasmfs speedtest1 build tweaks. Disable one wasmfs demo which was broken by Emscripten-side changes.
FossilOrigin-Name: 5ccc85d8f5809d961602dff557dca1ed34f5db61203912387272034e4da02614
2024-09-24 21:54:37 +00:00
drh 18e2ace468 Fix over-length lines in the shell source code.
FossilOrigin-Name: f3fdf76a0d3abeb5202f2b6f26318815396da7df680073955bde60b13f84e797
2024-09-24 19:47:12 +00:00
drh 697c3fb25b Fix error messages in the shell.
FossilOrigin-Name: aeef82e974c24071e3211588001a92c7f122fbaf02d9dcf7fa5aaf30d2729f64
2024-09-24 19:19:53 +00:00
drh 63558eae54 Get fiddle working again on this branch.
FossilOrigin-Name: 0f228317c6ea1388d82df46f44c166b2aa9be63361ff9b81569ebee63c930f2e
2024-09-24 18:01:50 +00:00
drh 6ba175f0c8 Port sqlite3_analyzer.exe to use sqlite3_stdio.
FossilOrigin-Name: 7c1c1226d803dbaf8e75085b1cc2aad88aef4c3168288e587c8c736633d09054
2024-09-24 17:49:06 +00:00
drh a55901a27c Port sqldiff over to use sqlite3_stdio.
FossilOrigin-Name: 18f784c47d4252bc3696a7e084a1afb9f51f006cf2021292f2103531b8235226
2024-09-24 17:40:54 +00:00
drh 0be2dd9a1b Get output redirection working again in the CLI.
FossilOrigin-Name: 086034c3508d95e4f620c5e0580fae770e85410b0c8bd94f600fc0fd25088947
2024-09-24 16:09:50 +00:00
dan 29057f3d18 Fix a memory leak in sqlite3expert.c introduced by [f1d76c86].
FossilOrigin-Name: 42bb941584a1ac922ee6b0b6ecadce71c9259555563cf49913a6f820f3f9b887
2024-09-24 15:11:34 +00:00
drh 57b32ef924 Add the sqlite3_stdio.h library for Windows console I/O.
FossilOrigin-Name: fcd0ecffc9889f8c855ea340f075ec42cdca482df82d6e67dc9c32613e8d5846
2024-09-24 13:46:32 +00:00
stephan d2917d9f69 Minor tweaks in mkwasmbuilds.c.
FossilOrigin-Name: cee43280727435440b9e01b302a790a1a102aea930bc821ebc737ea519458cef
2024-09-24 13:00:21 +00:00
stephan 88c43ce1f6 Wasm build cleanups. Fix the wasm speedtest1 builds broken by yesterday's refactoring.
FossilOrigin-Name: 5d4a090230d1d7609e70b9c28d662dc8b0f8dd4d70a254f3ce6e449c68a25897
2024-09-24 12:16:05 +00:00
dan 3c90b86372 Avoid an error in sqlite3expert if the database has existing indexes on expressions.
FossilOrigin-Name: f1d76c8636866424fe0b57d8814b8bb1a0ed4e2ca1b75d8e308d4b962ad8dc3b
2024-09-24 11:26:21 +00:00
drh 4e34558f1b Use _wfopen() instead of fopen() on Windows in the CLI.
FossilOrigin-Name: 21a8cac5e9a0d5ead29ca1114be7520d182348f7e2e2e2416852b827d7e09f21
2024-09-24 10:30:07 +00:00
drh 118ac6a7ad Always use fputws() for output to a Windows command-line prompt.
FossilOrigin-Name: 33950a8c3f3e48e5107fe56647da05147aa84f9c3eccbe7c8671f5b502ebb70b
2024-09-24 09:51:53 +00:00
drh 7d24e6b080 Remove an ALWAYS() that can sometimes be false, with wildly incorrect SQL
inputs.  dbsqlfuzz 707afcf73930de2624cdeca0ad1f0deea982dfea

FossilOrigin-Name: 766cf5535b9f51decd88c1736b988def929a90f68eb6a52bcaa21bb321586a08
2024-09-24 00:53:27 +00:00
stephan 405ae974ee Further wasm build cleanups and tweaks. No functional changes.
FossilOrigin-Name: 3b72a143431537ec275aefae739e56330c1d445c9ea80885882a2b9aa2201179
2024-09-24 00:31:08 +00:00
drh 415acd09d7 In the CLI, when displaying results in a columnar format, take into account
zero-width and double-width Unicode characters.

FossilOrigin-Name: 47cfad71eddc6fc4414ff853ee172da72527620eee743721806072c6e0a80caa
2024-09-24 00:01:47 +00:00
stephan 6ed21e8fd7 Typo fix in makefile comments.
FossilOrigin-Name: a4a1287fe8205d7c24556ec2e7ae09cda37a36dbe405ea70cd756cf488549036
2024-09-23 22:56:54 +00:00
stephan 69cce89425 wasm: replace much of the eval makefile spaghetti with equivalent code generated from a small C app. It turns out that's easier to read and write than doing the same thing from shell or tcl code, due entirely to C's lack of need for escaping dollar signs.
FossilOrigin-Name: 5440de48903e94f91090e2df65702ec0c504e33dd5cbd50f684cf30988f20b02
2024-09-23 21:22:09 +00:00
drh 1d9db01065 CLI uses only lib-c for I/O on Windows. No calls to Win32. Works on Win11,
at least.  Reads and writes unicode to/from the console and UTF-8 to/from files.
Prototype code only - must testing and additional work required.

FossilOrigin-Name: 5c54530d5a0a4125a1ba44f22537c4f63d5e5708f347c43cbac3e1832c4335da
2024-09-23 20:23:43 +00:00
dan 253695caf3 Fix a couple of problems with the sessions streaming interfaces.
FossilOrigin-Name: 9a1fc8d36b5b262f9d17a4743b4b9bb54510cf257d2d9778ebc42d5224856b1e
2024-09-23 19:32:06 +00:00
stephan d69b4249ff Merge current trunk into this branch.
FossilOrigin-Name: ed47d7f9a44b2af8ab7dd956495e71ea9a159cec438d1909f7022254a779e068
2024-09-23 17:06:06 +00:00
drh 5e419c1048 Add the run-fuzzcheck target to the MSVC makefile.
FossilOrigin-Name: 2e5194407a1b34dd0659c350ea8098bfef7b3f11aa5b2a07ecd2bce5582655a2
2024-09-21 17:27:47 +00:00
drh 38f2c5f81c If the database filename pointer passed to sqlite3_open(), change it
into ":memory:".

FossilOrigin-Name: 62e11a3a78edf9853b74d6495ccd8ae9ac1966c7d78eb3682cf2d5885e3740ec
2024-09-21 15:57:06 +00:00
drh 8a894cd804 Fix comment typo in the fileio.c extension. No changes to code.
FossilOrigin-Name: bf2c5b368ac094e048ca33a5e9175fe6a1fb5e2b1e392935eb29388e8999c40b
2024-09-20 17:41:05 +00:00
drh 7f0e0c7dbf New assert()s to help verify union access. No logic changes. Testing and
validation code only.

FossilOrigin-Name: b7b64c53f5d7f6b405ee3f1f1be1d84230aa79db5a502c8ad78d3e524f4384fb
2024-09-20 12:58:15 +00:00
drh db467f5777 Fix harmless compiler warning in the vfstrace extension.
FossilOrigin-Name: c8b4d092a22d87f1231424c8e3fa6a02f17fb965cb2a931621e1b51f4d215de7
2024-09-19 15:44:41 +00:00
drh f8305e4616 Improved rebustness to malformed UTF-16 inputs to sqlite3_prepare16_v2().
FossilOrigin-Name: 7b3a517b3e16ea487ca77a2c88a0c11d737de366524fc911aa1bdd6bfb7ad148
2024-09-19 13:39:06 +00:00
mistachkin 8513eb6ba8 Remove a stray newline.
FossilOrigin-Name: 77f587dc3bbc784e8884c1b56b53fa90bbd76f4bd121bd572ab529a2b9796e57
2024-09-19 00:05:18 +00:00
drh 0e8cbd5e78 Improvements to the scope of valueFromFunction().
FossilOrigin-Name: a0f39419cb5bdfa42ab2978cf3819e3d7821212996571f8251d2efbeaa26c603
2024-09-18 16:33:07 +00:00
dan 704561f346 Fix the preupdate hook so that it works when the "old.*" row has a column with a non-NULL default value that was added by ALTER TABLE ADD COLUMN after the current record was created.
FossilOrigin-Name: 6a8c687904e92f00c1a5f768947545d2920ab9025bf8649adb0ea8053f5aec4e
2024-09-18 15:52:05 +00:00
drh ae2a4acf8d Fix sqlite3-rsync so that it recognizes drive-letters on the front of pathnames
in Windows, and does not misinterpret them as hostnames.

FossilOrigin-Name: 54a3bbd5781f76bd73f5276b632d1107312f7a789134910793262317edd8359c
2024-09-18 15:38:05 +00:00
dan 38b31a93b3 Fix the preupdate hook so that it works when the "old.*" row has a column with a non-NULL default value that was added by ALTER TABLE ADD COLUMN after the current record was created.
FossilOrigin-Name: 00a398cf900179aa5a8aab09fe4a671d99e7a31583282848ef39390f2ef246eb
2024-09-18 15:02:27 +00:00
drh eaa50b8660 Fix harmless static analyzer warning in sqlite3-rsync.
FossilOrigin-Name: a63e412b6b2939422ecfa99d91fccb7a9c61e1533bb0db20ff12f3815ef41a2c
2024-09-17 22:57:08 +00:00
drh 87dce45f96 Add assert() statements to some of the sqlite3_bind() APIs that help human
readers and static analyzers, both, reason about the code and verify that it
is memory safe.

FossilOrigin-Name: 97528788145b83a1486dbaf09326ebedbc07bd0b47a57cdff773885b0b984604
2024-09-17 21:42:04 +00:00
drh ff60709d68 In sqlite3-rsync, do not allow the '/' character to appear anywhere in the
USER@HOST: prefix to one of the argument databases.

FossilOrigin-Name: 6089a90463dcb3ba8e1584cfc5e2528fbc131311c6df7834fb41a5614a8ca9e8
2024-09-17 10:36:33 +00:00
drh 3b204ec8bb Fix a typo in an error message in sqlite3-rsync.
FossilOrigin-Name: 3f25d6c8eac25de4afff486b134a339bc06404ddaed15b46db48c6770535b1e6
2024-09-16 20:53:52 +00:00
drh 61e3067529 Cleanup the sqlite3-rsync executable as part of "make clean"
FossilOrigin-Name: 8c5e481b49d86d97fefa345d0c1b96c050846ef9fc51d8b9dd203ee35afd6f6d
2024-09-16 14:11:14 +00:00
drh d70fbffc1a In sqlite3-rsync,
open the replica database using ATTACH since the sqlite3_dbpage virtual table
is technically a part of "main".  This avoids locking problems.

FossilOrigin-Name: 39c56c836a8ae52c5b42cc0d04b92f7c0e6889a03c4fdffd301209383b7c1f87
2024-09-16 10:58:11 +00:00
drh f8cafd7b11 Improved error message in sqlite3-rsync if the SQL statement for ORIGIN_TXN
fails.

FossilOrigin-Name: 73bde71ed10f52b851e220de35519d5fb7e65a932437ab82a194faff490b7930
2024-09-16 09:12:11 +00:00
drh a95620c141 Provide more opportunity for the remote side in sqlite3-rsync to send error
messages back to the local side.  Be sure to invoke pclose2() when done.

FossilOrigin-Name: 23a63505152ff995e4ce0963dc4ab9f76ed55da7e7f5e1b85a2577473642e3ad
2024-09-15 18:37:34 +00:00
drh d87a4f8bdd Merge subsequent changes to sqlite-rsync that were accidentally missed by
the prior check-in.

FossilOrigin-Name: 50762ba0783a04e0dcd9456a1ae17d875b0a9272f2f09854a23d9d5253761e9f
2024-09-14 16:52:45 +00:00
drh dbff02abc5 Enhancements to sqlite_dbpage() so that it accepts INSERT statements that can
extend or truncate the database.  Add the sqlite3-rsync utility program that
make a copy of a live database over SSH.

FossilOrigin-Name: b7a8ce4c8c5fc6a3b4744d412d96f99d2452eb4086ad84472511da3b4d6afec6
2024-09-14 16:39:02 +00:00
stephan 11e1defc8f CLI arg handling robustness improvements.
FossilOrigin-Name: 129aca54f6b791c222b51f3eb01569e1e569269860e153b005140eb65af378b9
2024-09-14 16:30:01 +00:00
drh 3d56d59adc Add the --version option.
FossilOrigin-Name: 9961334c8007e7cb6ae55885075b74acddc4fa701b359cf67e0f3c237d7eba4a
2024-09-14 16:28:38 +00:00
drh 7154803d4c New debugging options: --logfile, --errorfile, --remote-errorfile.
If page 1 changes, always send it last.

FossilOrigin-Name: 2d8cd76691554578e987ce682cf0c42c083711dd1511a178148978182ef43ba2
2024-09-14 16:03:16 +00:00
drh df9b488799 Always show the -v traffic message even if there are multiple -v options.
FossilOrigin-Name: fc05a5b7f77cdbfcc659d49eb09569a64a172362cb90199e2861028085178f10
2024-09-14 11:38:46 +00:00
drh 15e05eab32 Add the --logfile FILE option for debugging.
FossilOrigin-Name: 30e1b92d5663e24d2f325f2bab35f81b55848ef39d15688e40b9005269626303
2024-09-14 11:23:57 +00:00
drh 9d3047bd0d Improved error messages coming out of sqlite3-rsync.
FossilOrigin-Name: 452fb6de3984c3cb10d30b51dcdb2574578ca128a0c519b2bd43df0bdd343083
2024-09-14 10:59:32 +00:00
drh 2b30518804 Make the output from sqlite3-rsync with a single -v option the same as it
is with regular rsync.  Only show the SSH command with two or more -v options,
or if there is an error in popen2().

FossilOrigin-Name: 105ec44b470318fc9ff1773027c4064343f224068c9b6e71c5618f18f7dfcc3f
2024-09-14 10:48:05 +00:00
drh 33f0a3ed13 Merge sqlite_dbpage fixes into the sqlite3-rsync branch.
FossilOrigin-Name: dff76b7a3436031ea5a61b8a44ddfa1d40ea20c983f3d34a8501cd7074db68b8
2024-09-13 23:41:16 +00:00
drh 762946b236 Bug fix in the enhanced sqlite_dbpage for when truncating two or more
ATTACH-ed databases within the same transaction.

FossilOrigin-Name: 6aa9c8e79b440c6419e65990d9ceba8f00a6f975455138cf2aa82b113daec825
2024-09-13 21:47:57 +00:00
drh 8550e4a9f8 Fix harmless compiler warning on Windows.
FossilOrigin-Name: 86e0219d977c493ac19d00c3ddcf560eb317d506c7cf6e4ef17e92daa91e1762
2024-09-13 16:12:54 +00:00
larrybr 99f50dd219 Close the db as sqlite3_analyzer exits. (See [forum:af384c2315d9718a|forum post, "sqlite3_analyzer not closing WAL-mode db cleanly" ].)
FossilOrigin-Name: 94ceac98845e31249b656dcdb8a58f456b9212dc83968ea333852a66d72a0dae
2024-09-13 16:10:04 +00:00
drh 7316b6c072 Improved informational output from sqlite3-rsync.
FossilOrigin-Name: e55e3e8ec2fe3a9190872d999cee55c85bde92667040cc166233faaa2fa34266
2024-09-13 13:53:20 +00:00
drh d61d934f9b Preveious check-in was actually doing an 8-round KeccakF1600. This one
corrects that to just 6 rounds.

FossilOrigin-Name: 3c36f5814f25483586c4fd49ef2fe5c7c0ff8c59672b1622c92061ec0ba8547a
2024-09-13 12:35:04 +00:00
drh d0d3182fa8 Change the hash algorithm from SHA1 to a 6-round KeccakF1600 with a rate
of 160.  This uses about 1/3rd fewer CPU cycles.

FossilOrigin-Name: 96c7f47a8f59e5078bd296979421c1b57fbcb7be261f8a7a0b1d22a4b5914db0
2024-09-13 12:28:11 +00:00
drh e92f8e565b Further enhancement to PTRMAP display in showdb: Show the details of invalid
entries that are within the range of the database file.  Continue to ignore
invalid entries beyond the end of the database file.

FossilOrigin-Name: 4cad385b90eaca2d90e3375e473472145af4134160b81097a8535d06638c2e4a
2024-09-13 11:14:10 +00:00
drh 5c6dc45a10 Better initialization of new database files in sqlite3-rsync.
FossilOrigin-Name: 75d5a8eb3d4ece06900109ad4022ba2a3e82de2f0acb012e3a02bfb4326bfa6d
2024-09-12 23:30:29 +00:00
drh 74c8e234cd Enhancement to the "showdb" utility such that the "ptrmap" command shows
PTRMAP entries that extend off the end of the database, as long as they
appear to be well-formatted.

FossilOrigin-Name: a9f95fe5ce90ab9864165e603f3a34013c3c98d03f1db689996f4a32086e2ed6
2024-09-12 21:58:31 +00:00
stephan 5735921b0f Add RSYNC_OPT to the sqlite3-rsync build flags.
FossilOrigin-Name: b2a3497e5525dd33faf70961107a0529f476735fef756953c66e105747271c6d
2024-09-12 21:03:11 +00:00
stephan 6677cfeda7 Port some of the shell's CLI flag-handling code into sqlite3-rsync to make the arg handling more robust.
FossilOrigin-Name: 53fb9b11807ff7accd8cd41f9cb6516d2503f161ea976940437a1d3aae868665
2024-09-12 18:15:28 +00:00
drh 40b831851e Replica must be in writable_schema mode.
FossilOrigin-Name: e385525793c7d74ce8ee139c9d6cfc1248834754598f3fd45b22b9426ff106ee
2024-09-12 17:06:02 +00:00
drh 6df1de1c73 Improved --commcheck. Add the infoMsg() function which is useful for
debugging.

FossilOrigin-Name: b979d02ffd1370d8840328bce06c76c224f0fc1fb54b47d6c904547580a820a1
2024-09-12 16:54:34 +00:00
drh 0b8c8c861b Pass the names of both the origin and the replica databases to the remote
side, so that if the remote is the replica, it will have access to the
origin database name in case the replica name is really a directory.

FossilOrigin-Name: 435c30171d3c6073b7aaf5cc11cc4813f6a2d225ae6dce1b0e478f0cd5a0b532
2024-09-12 15:51:56 +00:00
drh f7c96ee98b Add a protocol version number to the first message, and give the two sides
an opportunity to negotiate a suitable protocol number, for future
compatibility.  Send the page size as a power-of-two.

FossilOrigin-Name: df0623aae1154281157409f62d6d3fb3ce41829281d53bc55868ce44b3d36883
2024-09-12 15:36:34 +00:00
drh 165daef043 Add sha1() functions to the CLI. Fix sha1b() such that it actually returns
a BLOB.

FossilOrigin-Name: fe65821a3b912f061026e6fd7174be26897010e6b474e2780350cac60faebaad
2024-09-12 14:43:05 +00:00
drh f1fb5ce10f Improved debugging output.
FossilOrigin-Name: 80461e0d724963aaf2646005298f1194c5f1c4c9ae41c1085d4d137ed485bd9f
2024-09-12 12:04:53 +00:00
drh dc3bec34a6 Progress on the sqlite3-rsync utility. This is an incremental check-in. It
does compile, but it does not work.

FossilOrigin-Name: fa06977b6db7fa745720561ec0b10570cf7e71598dc7a7c5ee650640e5bdf6f5
2024-09-11 17:02:44 +00:00
drh 2a4a4ec408 Disable the debug-use-only functions sqlite3_mutex_held() and
sqlite3_mutex_notheld() when compiling with TSAN in as much as those
routines cause TSAN to complain.  Response to
[https://issues.chromium.org/issues/41427446].

FossilOrigin-Name: db702dd78500a0839b0b2810a580d3634df49275470787b170973a86b73826d3
2024-09-11 12:17:26 +00:00
drh 79254dc363 Improved SSH infrastructure. The foundation is now in place to begin working
on the actual sync protocol.  Still experimental.  Still a work in progress.

FossilOrigin-Name: 9a1a95f523a96303aad57e2422c2b51ea7e125f5490f32f7a2929d49b6c69ef8
2024-09-10 22:14:18 +00:00
dan 0c8144ab4f Alternative implementation of fts5 locale=1 feature that allows blobs to be stored in indexed columns of fts5 locale=1 tables.
FossilOrigin-Name: 198305de92ebba7045d8ec7d2de98511f3b00924f808a3811f061dca47b01ec7
2024-09-10 20:32:36 +00:00
dan 27119c56bd Remove a stray "breakpoint" from a test script.
FossilOrigin-Name: 7d87a27a01311153ddee122cedecedc3bcc331618dc2ab1da397a3b257dc21cf
2024-09-10 18:38:47 +00:00
drh a9c8f7cf34 Initial infrastructure for the sqlite3-rsync utility. Prototype only.
Does not work.

FossilOrigin-Name: 397b2d37b7a6619b0c1eee201065585d03496f94786b21540f613e4716d56612
2024-09-10 17:05:12 +00:00
dan 923423c16f Fix a test in fts3corrupt4.test that would fail if SQLITE_ENABLE_FTS5 was not defined.
FossilOrigin-Name: 437849c80851da842b5c4fd37d5c147f821abc541e9b4d6f9000c12983548844
2024-09-10 16:40:08 +00:00
dan 7a41b48cf9 Alternative implementation of fts5 locale=1 feature that allows blobs to be stored in indexed columns of fts5 locale=1 tables.
FossilOrigin-Name: 55c5c119a0a77fac2c9f46d718ef78c0f33ed3520e10c240cf5bf1801e0586ee
2024-09-10 16:19:31 +00:00
drh 38b26d82e6 Add the ability for sqlite_dbpage to truncate the database file by writing
a NULL page.  Experimental.

FossilOrigin-Name: eb3c89ee2e4c5425be75deaf46a06a9cd8b210c695b918dd63a78f930c6e6b63
2024-09-10 12:09:03 +00:00
dan cd889c7a88 Fix an OOM-handling problem affecting locale=1 fts5 tables.
FossilOrigin-Name: d8103684f660ff9b3186d0f89afb113ca580bd16f0bf413ed8a9434236b54426
2024-09-09 19:12:57 +00:00
drh 882aba4090 Generalize the sqlite3_dbpage virtual table so that it is able to write
new pages onto the end of the database file using INSERT.

FossilOrigin-Name: fe0d67e72d4228661c021f227bfc0d5ddb1b726db0f36c7221ead8dd8bd1dc73
2024-09-09 18:45:58 +00:00
drh 9c2f068ebd Move the vfstrace extension out of src/ and into ext/misc/. Make it a
standard part of the CLI.  Improve its output.  Also fix some unrelated
comment typos.

FossilOrigin-Name: 123cb1f579daec3ed092fe9dd1bc0d3250f2b56d4cda1efa92af139029e112e2
2024-09-09 15:39:40 +00:00
drh dbab768d07 Fix dependencies in makefiles to include ext/misc/vfstrace.c as necessary.
Improved xFileControl() output from vfstrace.c.

FossilOrigin-Name: e8f2d6313075c92fdeebcdfd8b50f43e9d45225890b2ef6b77148a766a42e940
2024-09-09 15:33:37 +00:00
drh a40f6eb80e Fix harmless compiler warnings in the vfstrace.c extension.
FossilOrigin-Name: f23954e604bf4da45f07194b54a4fe1c83002ab65d6c6f0ac095e88baba18547
2024-09-09 15:19:26 +00:00
drh a8cf79e322 Fix harmless compiler warnings in FTS5.
FossilOrigin-Name: aa75e701de61fe63ec15c35d70e53e950ff73b0dcb0d871dd8721412f3af297a
2024-09-09 15:12:21 +00:00
drh 5600adf1b1 Move the vfstrace extension out of src/ over into ext/misc/ where it belongs.
Make it part of the standard build for the CLI.  Bring some of the vfstrace
output up-to-date.

FossilOrigin-Name: 055b97de8d2397987d72dbab1cde78ece2d1c066e95042b4ed6b7b36b2cf9006
2024-09-09 14:50:23 +00:00
dan cf25c16a54 Fix a problem with fts5 locale=1 tables and UPDATE statements that may affect more than one row.
FossilOrigin-Name: 70e42f941c0778a04b82655409c7caf4c1039589f7e43a8ec1e736ea8f931b26
2024-09-07 16:22:22 +00:00
drh 8755e695c5 Fix an off-by-one error in the routines that bind the special $test_TTT and
$int_NNN parameters for fuzz testing.  Fix to testing logic only - no changes
to the SQLite core.

FossilOrigin-Name: 6206b90a4ec3f05e3bbb4844e71569bbde7df237550569e6419ff7c3146505dc
2024-09-07 16:04:04 +00:00
dan f1c750e4ca Add tests for an fts5 NEAR() expression with a single argument phrase.
FossilOrigin-Name: e319d43bfd5ee4ed92b93531b239af4d1be0a8215b2a06c3532122ff2c7b6a7c
2024-09-06 20:12:59 +00:00
drh 8bd361e9e6 Update the bedrock branch to include all the latest enhancements from trunk.
FossilOrigin-Name: 4a3386ccb80e974538a96b830f0bb3c7802f5ef5a4f2f26d01fa0732fccae9e0
2024-09-06 15:45:23 +00:00
drh 7d322c14cf Merge the latest trunk enhancement into the wal2 branch.
FossilOrigin-Name: f3e3ad6945647c88f41748b5e8ce71c06d6e77ac67c607898af63ea14a91910b
2024-09-06 15:42:31 +00:00
stephan 6946a7fa50 Resolve the "No rule to make target '0'" error when building with --disable-tcl.
FossilOrigin-Name: d94541ae76b5d8b69f5524f10dcccc0814283f438a03f553848ed631a1983633
2024-09-06 15:01:00 +00:00
drh bc97630313 Fix a harmless stray carriage-return character.
FossilOrigin-Name: 84a6c5f975de36ba93b888e01cc5188717b5644ab8384beb8fd8906bdfc8e227
2024-09-06 14:01:48 +00:00
drh c43d16d69a In testrunner.tcl, correctly capture and display information about the
system under test:  hostname, OS, pointer size, byte-order.

FossilOrigin-Name: ec75cfc5d4b69e4aed64d17748cac15cd62a759a1cbe7feaa4580ad8346b0b95
2024-09-06 13:13:25 +00:00
drh deb6a60a84 Fix harmless compiler warnings.
FossilOrigin-Name: 60fb8ee153ec293b6b3a4170dafa305e4c16af575aced72daef46116d8dc2bb6
2024-09-06 11:21:53 +00:00
drh c4e84b7295 A testcase() macro added to help ensure that the fix to the omit-noop-join
optimization from yesterday does not regress.

FossilOrigin-Name: 224628b1039b996499e0d806fc0215f175da6f1059eb1b9ac491ac11126971ab
2024-09-06 10:35:36 +00:00
drh 56a0480540 Clarification and simplification to the README.md file and to the instructions
for building on Windows.

FossilOrigin-Name: f69ef1a37b2778bdf73ee7e3b3edd74f7344ab8e5eedbedc22203c782e521f5b
2024-09-06 09:49:13 +00:00
drh dc181d41d5 Fix a testrunner status reporting problem introduced by [aa5f10f21dbfb24e].
FossilOrigin-Name: d20c65c3b4256a662ebf7ed024b7b7adaceca90358f58111dc645da322000794
2024-09-05 23:40:13 +00:00
drh 934e796e43 Ensure that the WhereInfo.revMask bitmap is adjusted when tables are removed
from the FROM clause by the Omit-Noop-Join optimization of
[0cd82ee9a8413cf1].  Fix for the issue described by
[forum:/forum/8a1e467e905b8d27|format post 8a1e467e905b8d27].

FossilOrigin-Name: 22ca5a2ffb89ccb5f337993b5a95e27c449c39014284156eabc33da012a8759c
2024-09-05 23:22:55 +00:00
drh 6ffcdb1856 Update notes on build procedures in the README.md file.
FossilOrigin-Name: 1d12744fe7cc294994e53e1ffe72799d0321dd5c1ef031213f083e200918fee5
2024-09-05 15:55:04 +00:00
drh 4987c5c3c2 Testrunner.tcl enhancements: (1) Attempt to build the SQLite tcl extension
if it is not already available.  (2) testrunner target "devtest" is added as
an alias for "mdevtest".  (3) Try to keep summary information at the end of
a test below 80-characters per line.  (4) Update the Makefile.in so that
the "clean" target removes the tcl extension built by item 1 above.

FossilOrigin-Name: aa5f10f21dbfb24ee54ca96bfb7b013ae29e26fec05b80681f19cc63d9face49
2024-09-05 15:28:15 +00:00
drh 21e7816e65 Enhance testrunner.tcl to keep track of the platform and SQLite version
as reported by individual test cases and to report that information in the
summary at the end of each test run.

FossilOrigin-Name: 80ebb7c7e686cd936ac834f2258f585a7004762593e0bc859ecd75d6fb0badfd
2024-09-05 12:06:45 +00:00
drh 6aedd6d9fc Faster implementation of the aggregate_test_counts procedure inside of
testrunner.tcl.

FossilOrigin-Name: a01d869520329fb9e786cdc65f359785a95d19d289e4c6b844c758d6e5385aaf
2024-09-05 11:46:43 +00:00
drh 7bd230cf4d Make the default TCLDIR be C:\Tcl in the Makefile.msc.
FossilOrigin-Name: c2c0a9176ae7291a0b72f3d048a2ef5716c325b9f693cdc5eada552e0c881a9d
2024-09-04 18:51:59 +00:00
drh 854c8b2b2d Update the bedrock branch to include all of the latest trunk enhancements.
FossilOrigin-Name: cf8f155280100c1a5b79b118875a93686a554d8be55114277f4f7b20cfd7835a
2024-09-04 16:54:56 +00:00
drh 81d50a0880 Merge all the latest trunk enhancements into the wal2 branch.
FossilOrigin-Name: 9f530343717a891d57013c5fb58f30e422da391815a7cc79ed8beb1196c89faa
2024-09-04 16:46:09 +00:00
drh bb4d2edf10 Fix a bug in the parsing of some corner-case JSON PATH strings that contain
escaped double-quotes.

FossilOrigin-Name: 60ac55c4b76355aaf7cbde38bf1f6082ff5612bf4ffc49ab69d00fd4e3d64e64
2024-09-04 16:01:44 +00:00
drh 109025c0a4 Put a proper shebang at the start of the testrunner.tcl script and make the
script executable.

FossilOrigin-Name: c0c6e9abebf76358625f30a179658319b260baba6eded2a4c5ad356143e36f97
2024-09-04 13:42:19 +00:00
drh 3c72072a6f Make extra efforts to delete the over-size directory names created by
the win32longpath.test module.

FossilOrigin-Name: 3c8035ed3f023fb31f65f298fb9ce19d4fa7a5511bd90877b27969f5f0892472
2024-09-04 13:09:58 +00:00
drh c81ab76cd9 Add the "joblist" command to testrunner.tcl
FossilOrigin-Name: f64469f4806d4d5d7103c171a37a542c7aab9db09a226bccd8411e9ccd55353d
2024-09-04 11:22:47 +00:00
drh fd486258a0 Enhance the "errors" command in testrunner.tcl so that it accepts the
"-s" or "--summary" argument to see a list of failed jobs, and so that
an additional argument is a GLOB pattern that restricts the output to
jobs whose names match that pattern.

FossilOrigin-Name: dcbebe30f594a99e23b5ccd8d199b92118204a3e52e75c78d98c394601252e81
2024-09-03 16:04:34 +00:00
drh 940cdc94f3 Fix ext/consio so that it works correctly with SQLITE_USE_ONLY_WIN32.
FossilOrigin-Name: efc6f3d7e92a25f440fb8d392daf325af5ca7dca104a5bb4bd59f7df93af53b0
2024-09-03 14:15:57 +00:00
drh 23a16c8387 In testrunner.tcl, compile the Windows Default configuration using
SQLITE_ENABLE_STMT_SCANSTATUS.

FossilOrigin-Name: 49e8b1635f29d9fd0dc2ef6e312fd4129f2283f68e9423d95ff9b55077688ad7
2024-09-03 14:00:36 +00:00
drh e6e49bfef2 Fix harmless compiler warnings in the percentile extension.
FossilOrigin-Name: c5557f281c6294b6db6682349d245feac7c6ce7f4f61356f486afdf186c566c4
2024-09-03 12:41:21 +00:00
drh 06638117bb In testrunner.tcl, allow setting njob to zero, which causes no new jobs
to be launched and for the process to shut down once all current jobs are
completed.

FossilOrigin-Name: 0ef65fd4ba17def4c13986365b3af300c4024725af4bc314845d1af8be568ab4
2024-09-03 10:53:32 +00:00
larrybr 22cd1eec5a Cause cfGets() (under SQLITE_USE_ONLY_WIN32) to better emulate fgets().
FossilOrigin-Name: 2d783524d1671d988ebb1b21c83de99c9f335963b6c20cf4df612f58c13da913
2024-09-03 02:09:13 +00:00
drh 7c6f2880aa Fix possible NULL pointer dereference following OOM in the new error reporting
logic of the percentile extension.

FossilOrigin-Name: 7891a266c4425722ae8b9231397ef9e42e2432be9e6b70632dfaf9ff15300d2c
2024-09-02 21:59:31 +00:00
dan 70989b6f59 Fix a technically undefined signed integer overflow in fts5.
FossilOrigin-Name: e6bec37ea1ca51e1d048941ce4c5211d8fc5c5e3556a1441f9c79b036843f9e3
2024-09-02 18:41:59 +00:00
drh ef7c692baf Make the percential extension easier to incorporate as a built-in on Windows
applications.

FossilOrigin-Name: 831e2be4117dc7f5113f66d3e484ca34e4405444ac7b0def12ee9a00608d40da
2024-09-02 17:27:10 +00:00
drh da52f11250 Remove some debug/test code from percentile. Make the extension easier to
statically link.

FossilOrigin-Name: 6e5f146e274b210749dd8923a2da56905eb098b5a66699ff9130e3e51b1ff662
2024-09-02 14:46:56 +00:00
drh 7a29505e57 Enhance the percentile extension to support percentile_cont() and
percentile_disc().  Add support for ordered-set aggregate notation in 
the parser when compiling with SQLITE_ENABLE_ORDERED_SET_AGGREGATES.  A
new function property SQLITE_SELFORDER1 must appear on any aggregate function
that wants to take advantage of the ordered-set aggregate notation.

FossilOrigin-Name: 557bd245ef24af15da1374762bbeb9b8b0241da06e604efdf00d567a9ed09216
2024-09-02 12:07:37 +00:00
drh d281889ac8 Improved error messages on percentile functions. More tests cases for
percentile and for ordered-set aggregates.

FossilOrigin-Name: e1bca168e70335fa2f9537632fd16e374c566a564fab4974c0b8f61cb63d08ce
2024-09-02 11:17:04 +00:00
drh a955ec806a Change the name of the enabling compile-time macro to
SQLITE_ENABLE_ORDERED_SET_AGGREGATES.

FossilOrigin-Name: 3b1cdddf8339cc339ec74cd8be2bfa42e62b500048a444eb9e5d9817bc4702ae
2024-09-02 09:40:37 +00:00
drh b5df31d15f More internal documentation improvements.
FossilOrigin-Name: 8b91b74931c36e1955ef933a07d8ec40c8b54c882efe7084d179168867c5244f
2024-09-01 23:56:14 +00:00
drh b9a6e42a55 Improved documentation of recent enhancements to the percentile extension.
FossilOrigin-Name: 51e7b4c9cf19a5986432a76c5fd30cef715c170a403d7b4304a8c5888c445a91
2024-09-01 23:47:20 +00:00
drh 1c30df3598 The WITHIN GROUP (ORDER BY ...) syntax is now a compile-time option:
SQLITE_ENABLE_ORDERED_SET_FUNCS.  There is no increment in the code size
if the option is omitted.

FossilOrigin-Name: c9367e2532d653738af27c4d92810eb648a9e12f72d7223017c87cb0dddc6804
2024-09-01 23:27:34 +00:00
drh ca447020ae Implement percentile_cont() and percentile_disc().
FossilOrigin-Name: b1a93f67d6b21df6fe3247c9333fead61dd425574f66ea3eb06b80c2b06f616a
2024-09-01 20:24:24 +00:00
drh 7aa01a0ffc No prepare-time penality for ordered-set aggregates for applications that
do not use them.

FossilOrigin-Name: e070c16d2183312e416ff6af770346041e4d3836c4db2c9ea6049f63fb0eaa07
2024-09-01 19:19:26 +00:00
drh 5d5d6194c9 Merge fixes from trunk into the ordere-set-agg branch.
FossilOrigin-Name: 7528ddcfdf155116353266e00e6408c526e29f04cb00ca2d9e84b105329b17c0
2024-09-01 18:57:52 +00:00
drh c97ad9b4e1 Improved sort function in the percentile extension.
FossilOrigin-Name: 9f84e8d59bcda642e732565e840f6a880a01b2fc65af2651248f6a8a6e1cb65a
2024-09-01 18:54:31 +00:00
drh 41155d8885 Fix to the quicksort algorithm in the percentile extension.
FossilOrigin-Name: 54313f74b24c46f1afa1ff082ef569fa46938527095be358015e7047e17702ac
2024-09-01 10:52:20 +00:00
drh df2d14b86e Add error checking: Do not allow functions other than those in the
percentile extension to use the ordered-set aggregate notation.

FossilOrigin-Name: 317d901429303340290334dbe7680a36339df0a50b586e3f71b6c5e5eba6d411
2024-08-31 22:22:24 +00:00
drh a239ece378 Demonstration of how ordered-set aggregates might be parsed and integrated
into the existing parse tree, should we decide to support them.

FossilOrigin-Name: ef9777890001ba0122d3add799795e118fd87243f42731ab84d4748b67605647
2024-08-31 20:09:37 +00:00
drh c1042cd133 Include percentile() and merge() in the fuzzcheck test utility.
FossilOrigin-Name: d1d0942a947803d45a1fd9068f3518cf412178b6b9bafcb82db44c52d5820c11
2024-08-31 19:24:17 +00:00
drh 65b1244781 Omit the percentile_cont() function added by [095c22e62248f8ef] (and not yet
released) since its usage conflicts with the PG percentile_cont() function.

FossilOrigin-Name: 3fe0a852978f3f1218e37a58f0d3b54016d4116a3301aa32efa7c4c12c767755
2024-08-31 18:35:10 +00:00
drh 49af3dfbf4 Allow percentile() and median() to act as window functions.
FossilOrigin-Name: 94cf96af8fee55449080655bddf81cbf5c078a02d7bb5dd7e4903b36f83a8c07
2024-08-31 18:08:31 +00:00
drh a5315a6ece Still more test cases.
FossilOrigin-Name: f09904608195dac38172b0dd4dcab3190f33c116d468beff27f913a7433b400e
2024-08-31 17:50:06 +00:00
drh f138110dfd Add more test cases.
FossilOrigin-Name: 0d0e5456793b4bef673ebc7fcc1c393a6d3c817363d948ddfe06d60eab56dc24
2024-08-31 17:27:55 +00:00
drh ad8ec9db63 Avoid unnecessary sort operations when running one of the percentile
aggregates as a window function.

FossilOrigin-Name: 5d311536211eb1e3c887ceb7e6516d3900e6eebbccc8c445dd43cdd556182728
2024-08-31 16:55:14 +00:00
drh 129767958a Test cases added.
FossilOrigin-Name: 25e68229843cc84978955817285550085d1306ba4ce3b0517dd00e5d05b9ae0a
2024-08-31 15:02:07 +00:00
drh 90fa4c7b2c Enhance the percentile() and median() extension functions so that they can be
window functions.

FossilOrigin-Name: 4d0e3df4b9c609755977b8a462126242d2be1310c0122a8d4ba76d98d32a7230
2024-08-31 14:31:17 +00:00
drh 3196c349d1 Do not fail the omittest if ICU libraries are not installed.
FossilOrigin-Name: ddc55efd2d59df3f20743b0533550436da945453c069025a3f871d28d40e13d4
2024-08-30 17:33:25 +00:00
drh 0337b9a628 New makefile target: "tidy". The "tidy" target removes all build products
be leaves behind test results.  The "clean" target has been enhanced to do a
better job.  The tool/omittest.tcl script is rewritten to work on the
configure-generated Makefile and to require no arguments.

FossilOrigin-Name: e48add02695a41b26a04e7942b5333e2bf4dc5598e363367aea3a4690982667d
2024-08-30 16:51:41 +00:00
drh 6cb12337b9 A couple of #defines so that the build work with SQLITE_OMIT_FLOATING_POINT.
FossilOrigin-Name: be768f80c703ac1645251ccbac83e07b3ea4cc231c8ea7e66374a4f78361f10b
2024-08-30 16:43:36 +00:00
drh c589d5ab06 Rework the tool/omittest.tcl script so that it works with the
configure-generated Makefile, and requires no arguments.
Update the configure-generated Makefile with a new "tidy" target which
is like "clean" except that is preserves test logs.  Use "make tidy" to
force everything to be recompiled from scratch without destroying logs.

FossilOrigin-Name: 99491df19d68df1f0515bd5ec9240ef3dd6e0fa38a6c837a38a16a873f8e2cbb
2024-08-30 16:24:49 +00:00
drh 28db1a68ba Improvements to "make clean" on unix.
FossilOrigin-Name: a1d775f07528192b09566ff1546b067ffbee0db5812e424e6b7f3cc33a67dc83
2024-08-30 12:10:28 +00:00
drh 11397ce104 In the testrunner.tcl status report, try to provide a rough estimate of
the time remaining.

FossilOrigin-Name: 90bc616d20e8c247691c45de2a28d41c1632b21152dc34253eefc179a90ab31f
2024-08-30 01:11:19 +00:00
drh 920662eaf3 Merge the latest trunk enhancement into the bedrock branch through the wal2
intermediary.

FossilOrigin-Name: ff94464cec871bd90634e7e1e4252954eb68e77037dd290b0fbf7747c9b19308
2024-08-29 23:43:55 +00:00
drh cb4a48e249 Merge all the latest trunk enhancements into the wal2 branch.
FossilOrigin-Name: dfecc79c8fa4b39e5211f1a7051783c750bb7e1132f3e3a60f8ae0837dbbe486
2024-08-29 23:33:17 +00:00
drh 494ba30c9b Sync the autoconf/Makefile.msc with the main ./Makefile.msc
FossilOrigin-Name: 9de47c3611ca05e03a661807a06c6c5c0b5c6548ea7a8d73cb63832d2dff9d0e
2024-08-29 23:32:16 +00:00
drh 6a18c9e06a All makefiles are responsive to OPTIONS=... command-line arguments, and add
the RHS value of OPTIONS= to builds and to lemon and other build steps that
might respond to -D or -U options.

FossilOrigin-Name: 854b3776ee1fcaa5931e3a0ed104978ca350d218e553586d1c40c2420e1be498
2024-08-29 18:32:56 +00:00
dan ccd34894e1 Fix a case where the BTCF_AtLast flag was being left set incorrectly. This was causing obscure window function queries to fail.
FossilOrigin-Name: e042eb024738a83eed92cd2dfac3d2a50d2589715a3d81b1a494564dd7d8e7fa
2024-08-29 16:27:57 +00:00
drh c1547d1f7a Do not allow sqlite3_blob_open() to work on a any table that contains
generated columns, even columns of the table which are not generated themselves,
because such columns might be part of the expression of a STORED column.
This restriction could be relaxed some, but that would be a lot of code
for something that nobody ever uses.

FossilOrigin-Name: 6e84947123339f0ea2b7dabc6134a621eed09317f81647f15870ed50a0f7d1bc
2024-08-28 19:13:16 +00:00
dan 50ca8af6f2 Fix a dropped error code in fts5.
FossilOrigin-Name: df55502e4f412e5b1daccf82f11fa4eb932047d9972dcd16e36be00cf09f78e1
2024-08-28 15:54:46 +00:00
drh 8db3fc5488 Update the README.md file to recommend running "devtest" instead of "mdevtest".
FossilOrigin-Name: eb016f66ea2c7ebacf8c57495843db3414602d062e4a430f2cc603c88b10c3f9
2024-08-28 13:55:46 +00:00
drh ba5994b181 Maybe [7420b13238da4e6b] was not such a good idea. Restore "make test" so
that it works as before - running a single-threaded test over the source tree
with options specified by ./configure and similar.  Instead, change the
"make devtest" target so that runs the automated multi-threaded tests on 
a standard configuration.

FossilOrigin-Name: dc9fee980d4f4b5a569628702e52cef9e5d8e6c69c2b84546dce47e3e514775e
2024-08-28 13:51:51 +00:00
drh 3341f5ce85 Minor wording change on testrunner output.
FossilOrigin-Name: 115e68597957a9ee80de7f554ee3768c992051687c8f07eaf3a9fcaab7efd44e
2024-08-28 13:34:20 +00:00
drh d6b56d65a8 Update the "make test" target so that it is really just "mdevtest" with
"srctree-check" and "sourcetest".

FossilOrigin-Name: 7420b13238da4e6bfb1589eaf3cfd842fe3c4b0f3b6a3b49c24cd70589bf4809
2024-08-28 12:10:51 +00:00
drh 4f2f91bf08 Further improvements to the status display in order to maximum the amount of
information shown on the limited screen realestate available when running
via "watch" or in continuous monitoring mode.  All details are still shown
when doing a stand-alone "status".

FossilOrigin-Name: 557a2c65f5ae1d8cb1858c17da0fe64fbfba83d787cfb7f2f21582960a8b9eac
2024-08-28 11:03:58 +00:00
drh 242e064978 Only limit the number of failures reported by testrunner status if using
VT100 cursor movement.  For a straight-up "status" command, show everything.

FossilOrigin-Name: 791237e0f4b67f0197ae11e966554edeade0c443289d2ade00470d23bf5e8ec8
2024-08-28 10:36:18 +00:00
drh 4a90f81a6d In the testrunner status display, limit the number of reported failed jobs
to avoid overflowing the terminal when there are many failures.

FossilOrigin-Name: ffeaa4d5d73871cbdf1ef70b9845d921ebdb96e964d232661a5048cab7d744ed
2024-08-28 10:25:44 +00:00
drh a88d61824d Disable shell test cases that require virtual tables when testfixture is
built using SQLITE_OMIT_VIRTUALTABLE.

FossilOrigin-Name: 8c73d54fd1e250fcd7f30741cfbd169af9aaecc2096c0c8a9486abaa064d69af
2024-08-28 09:47:29 +00:00
drh af43eddc23 Reenable SQLITE_OMIT_VIRTUALTABLE on the Device-One alignment for testrunner.
FossilOrigin-Name: 8f2cb357634ec0b5aef14f3d967e76db236f8899f3201efc80c264f548cc1b0b
2024-08-28 09:31:17 +00:00
drh b5b4ca2a11 Changes to the way "release" tests were constructed in [d03d35eebaf82709]
were incorrect.  Fixed here.  Also enhance --explain to chose permutation
configurations and so that PATTERNS on the the command line can match
against permutation configurations.

FossilOrigin-Name: 1f962ffc9e956a9d6311ed9c02fea13c0fe056283ee2c650a2312a258808b3b3
2024-08-27 22:28:03 +00:00
drh c3c645ef46 Improvements to --status overwrite in testrunner.
FossilOrigin-Name: 26372762ab1451eb95d29367f5b2534bde616a37e5e757e72ba2cfe42571d180
2024-08-27 21:44:45 +00:00
dan f5a9b58c8b Avoid reading the structure record from within the fts5 xConnect method.
FossilOrigin-Name: 6a6ce343b249e269229867b2c2f107a49ac8a8f24ad66801718ef01159655319
2024-08-27 20:37:30 +00:00
drh ce018f5bb3 Further minor improvements and tweaks to the testrunner --status display.
FossilOrigin-Name: 7d1ccc6dfc9abe053baaf39f68e9fb14001f5032a52b85fe1e4b752e48317536
2024-08-27 19:49:51 +00:00
drh 1a7cfbebb0 Fix typo in the previous check-in. Ensure that --status header lines do not
overflow and wrap.

FossilOrigin-Name: 2eaea67495fd9961b7145d829789889bf1d8fa7a834d40d1be43e7c74c233bc2
2024-08-27 19:48:19 +00:00
drh db05c66a16 In testrunner, when doing the --status updates to the screen, be sure to
overwrite dead text at the end of each line with spaces.

FossilOrigin-Name: 7c195f132cb48ee44124eb4af3532937493429f7c45f6ac611f7a9b128799ad7
2024-08-27 19:43:21 +00:00
drh 71475478a1 Further improvement to the status display in testrunner.tcl. Show the
number of errors and the number of test cases on the status summary.

FossilOrigin-Name: dbc1ac8692feccde271eaca78f2833c8bb7bf620088c8f975692d6ae2c8ba53a
2024-08-27 19:28:21 +00:00
dan c8edf07ca7 Modify the implementation of "wrapper" tokenizers to make them more robust in the case where the database connection is closed before the tokenizers are deleted.
FossilOrigin-Name: 7c0001d6eb43f89144eb84d9e30f575a3feed401d685a0d1f260692e419b2df2
2024-08-27 19:27:40 +00:00
drh d9bc2bb638 Improvements to testrunner.tcl: Show elapse time in MM:SS or HH:MM:SS.
Keep track of the total number of tests and the total number of errors
and report those values on a summary line at the end.

FossilOrigin-Name: 40b232924c973f8d94605946fff75acbe120d90634a2eaf38bd31649e9f1390b
2024-08-27 19:17:29 +00:00
dan 3c39f1f5d6 Add test code to drop an fts5 table with corrupt records in its shadow tables.
FossilOrigin-Name: ca21c942c30a3dbff0e7d118e105b847d80b5388c74d19c2eeea71581f8f40b8
2024-08-27 18:25:04 +00:00
drh dac0d0b310 Updates to testrunner: (1) Omit all testing of User-Auth.
(2) Automatically add the "*" wildcard before and after all pattern
arguments.  (3) Build the sqlite3 CLI for release tests.

FossilOrigin-Name: d03d35eebaf82709414c87cfa6abc9d2baf8d7e64c2627bad0fd5bbda3e78d60
2024-08-27 17:38:26 +00:00
drh 4a972bc1b8 Merge the SQLITE_USE_ONLY_WIN32 flag for consio into trunk.
FossilOrigin-Name: e9b03b082dcf141695140a6c2c50bf0dc577c3d64ab76f81dd22f0084eca26d6
2024-08-27 14:35:54 +00:00
drh 8aa9135f2c Only avoid all C-runtime I/O if the SQLITE_USE_ONLY_WIN32 flag is defined.
FossilOrigin-Name: 164b1e1962aa1e16bdf52e9e86d4cf9c9e09220c0821932ac8e390e82074185f
2024-08-27 14:25:52 +00:00
drh fcd65efcf1 Merge all the latests trunk enhancements into the win-dupe-crt-fio branch.
FossilOrigin-Name: 2d52db98f47fbcda0622c034b21c2fb19bf4345b88c5c565ae9e6f7128642e43
2024-08-27 13:57:43 +00:00
dan c013fa50db Changes to documentation comments in fts5.h.
FossilOrigin-Name: 886545a8d0e9cbf4911ecf747da0ea8aad702d79cb065dad01a04aa4f51da12b
2024-08-26 19:08:54 +00:00
drh 1321ea5010 Randomize the name of the attached database used to do VACUUM.
FossilOrigin-Name: 0f4e65d73a2b95932f36768c0d9925a2d265d9b395b886d8e492d1d519b2093c
2024-08-26 17:35:32 +00:00
stephan 8ade1d2f97 Merge trunk into the wasm-build-rework branch.
FossilOrigin-Name: a1cf3095a87e101ff83ad64f20ae2e61858c7049181abfb18aece6f0a8451c15
2024-08-25 12:04:05 +00:00
stephan 87ebadbf4e shell.c.in: use eputz/oputz() instead of eputf/oputf() where appropriate to avoid compilation errors in -std=c99 mode (namely wasm builds).
FossilOrigin-Name: 9ef8317faebc29d016bdf2e8c678fb21ca2cfa95272f1f18b461fcaf0e220ac5
2024-08-25 11:59:29 +00:00
drh cc902fd25e Fix harmless compiler warnings in fts5V2toV1Tokenize().
FossilOrigin-Name: df65d00f104f31741056686f2ba41ecb192c552012bffb17c2a8b5d4db058328
2024-08-24 20:05:59 +00:00
drh 306a8dcf3e Fix compiler warnings in the CLI detected by MSVC.
FossilOrigin-Name: 23ae505cbfde6dfd1dbb2216cf76c1e316b49f2ca84491a1aae28ad33f7777a9
2024-08-24 20:01:05 +00:00
drh 838d2d3078 Improved help message for the --enable-test-status configuration option.
Fix the build for when that option is omitted.

FossilOrigin-Name: b404a5fe3f0532f6d56b5e65c44a59379c17d175c21ac69a1f2ba8aadf453313
2024-08-24 19:06:52 +00:00
drh 1d5a12e6a5 New ./configure option --enable-test-status causes testrunner.tcl to be
run with the --status option for targets like mdevtest and releasetest.

FossilOrigin-Name: 9ca8513c26e54c97f2d3c6105c42144100044727f6fb4c28048f7cd7270dd35e
2024-08-24 19:02:33 +00:00
drh f9eff9052d Remove a term from an assert() that is no longer relevant.
FossilOrigin-Name: 8b4766252b80075354a6b1406c5430fe999a6490aa3e079893fcc682d88734f7
2024-08-24 18:42:39 +00:00
dan 58b08d7702 Avoid a stack overflow that could be caused by a recursively defined WINDOW() with a strategically embedded error.
FossilOrigin-Name: bada54bd6bf54190e40aa721b77081015957d204c7b6a9fdbe8c67bcf20798f8
2024-08-24 15:54:15 +00:00
drh 27095fc4a5 Ensure that the WhereLoop.u.btree.pOrderBy field added by the
order-by-subquery optimization [7a0cdc7edb704a88] is correctly
initialized.  dbsqlfuzz 437be171a98687082a31ea6efc2fd5c5d64481c8.

FossilOrigin-Name: b75f8b2252aefe7bb96fa237205a1a9dc88c46d9ec09bca7407e6c636b203e7e
2024-08-24 11:48:19 +00:00
stephan 7766c9943e Eliminate use of strcpy() in shell.c.in to squelch an unwarranted link-time warning on OpenBSD.
FossilOrigin-Name: 32e9bf7f972144eacbb2580151220bc4fc1cbcc8bcf586ffd8e10762abf75c7e
2024-08-23 21:20:50 +00:00
dan 789da35848 Fix a problem in the ext/fts5/extract_api_docs.tcl script.
FossilOrigin-Name: 9a9d0f6301faefe324261f03543023ffb6a90823349c6946abb0df2f69b31f96
2024-08-23 17:40:29 +00:00
dan 8d4edd0d67 Add fts5 auxiliary function fts5_get_locale(). For querying the locale of a stored value.
FossilOrigin-Name: 396f720f36a937145ac727c2750acfd26e4eda350b46d648d82a7e9985545a9f
2024-08-23 15:18:02 +00:00
drh 5804da0218 Enhance the generate_series() table-valued function such that it is able to
recognize equality and inequality constraints on the "value" column and
optimize its operating accordingly.

FossilOrigin-Name: d50b784807333c5461a2d027778c746c799285b95bb1952f142b317ea2846460
2024-08-22 18:12:10 +00:00
drh d55ab84451 Add the SQLITE_INDEX_SCAN_HEX bit to the sqlite3_index_info.idxFlags bitmask.
When set, this bit causes the EXPLAIN QUERY PLAN output to show the idxNum
value in hex rather than in decimal.  This is purely a debugging aid.

FossilOrigin-Name: 6c00e88ebdb41d6317bb8758825521614dedc2e6e6289ff415c5f0406eed815b
2024-08-22 16:22:08 +00:00
drh 889ce8ee90 Add SQLITE_TESTCTRL_GETOPT. Use it to implement improvements to the
".testctrl opt" dot-command in the CLI, and similar enhancements to TH3.

FossilOrigin-Name: 6500baa9914b08ce19c361a69f19a80e5d800fcb82579d1c1e03601c7a664338
2024-08-21 20:25:31 +00:00
drh be1fdbd272 Add the new SQLITE_TESTCTRL_OPTGET that retrieves the current optimization
setting.  Use this in the CLI to provide symbolic fine-grain control
over optimization settings using the ".testctrl opt" dot-command.

FossilOrigin-Name: 0ea4e1e8fc689cb1af3a8d21dd6af9d483115412b414e85e8a42480f50d65af2
2024-08-21 18:57:55 +00:00
drh 9904d824b8 Improved comments on byte-code subroutine calls to materialize subqueries.
Should not affect production builds.

FossilOrigin-Name: 6b024c85ef856e26cddac8ad862e16909b5149c3e3efb4e73ba54c97291f1cc4
2024-08-21 17:38:50 +00:00
stephan 5b5625f679 README.md: a markup link fix and use the canonical URL for Fossil.
FossilOrigin-Name: c49bfee6b878bd96adfc852fdc76e98bf81e2fb82be8325672fbc7af04e7dffa
2024-08-21 13:57:30 +00:00
drh 59b171172f Updates and enhancements to the README.md file. No code changes.
FossilOrigin-Name: 3161b8028916bff9a13a04bfb1996bce7e1274a3c403fc58f4d798afd30c528f
2024-08-21 13:44:40 +00:00
drh 4cc86b6e92 Add code comments for a "table-of-contents" and various milestone marks in
the 1300+ line long sqlite3Select() function, to help improve readiability
and maintainability.  Comment changes only - no functional changes.

FossilOrigin-Name: 00cfbb9fa5136e6a7847da4e4ab30d320ca79c22acda9db2030e558d59b8c744
2024-08-21 12:01:46 +00:00
drh 91937a6961 Fix a minor typo in a comment.
FossilOrigin-Name: 92c80af1129051c9eded8df170730ad6366b4f7715dead34c4364c8149d0dce9
2024-08-21 10:37:59 +00:00
drh 01972f55fe Improvements to treetrace output for subqueries. Debug code only.
FossilOrigin-Name: 0a835d18d9063b17a1d506fd989ebcf7f0d76593978a3bd5981be59618335f21
2024-08-21 10:32:00 +00:00
stephan fb7d0c3c01 De-duplicate some error-output code in shell.c.in as part of tracking down the cause of [forum:5647ca2af1|forum post 5647ca2af1].
FossilOrigin-Name: fd5904495e0c775174f9380bc6877fb9def3d30dec1c9315979ad4503a18d70b
2024-08-21 09:56:45 +00:00
drh 8c58fd78c3 Remove incorrect assert() statements in the SrcItem invariant validator.
dbsqlfuzz 7f908865436ce531b0ace55c97a1c71d05c4e76c.

FossilOrigin-Name: 81a1ede38823b8d50e221ecb6ce52a05d82a856099002a71d9f1ac561a587f05
2024-08-21 07:38:14 +00:00
drh 5648d717b1 Refactor the SrcItem object to move fields associated with subqueries out
into a separate object named Subquery.  This reduces the size of the SrcItem
object by about 1/3rd and provides improved performance.

FossilOrigin-Name: 484bcd75bc95491d8540c791c1c4d40d996cb465839564662e14f98739699bf1
2024-08-20 23:11:28 +00:00
drh a0651b3707 Fix a name resolution issue with CTEs.
FossilOrigin-Name: 4fa8235dd59cd683d6c6c97bfe181a9637be7c054d435323c903b9dbd74aff02
2024-08-20 22:44:40 +00:00
drh bb36d55e63 Fix minor comment issues.
FossilOrigin-Name: c1bbed19f4348ec31c3cb4723eaa8f1554bb58ef4de7feb3a0f39612785f13d5
2024-08-20 22:05:01 +00:00
drh 0766cbf511 Remove unreachable code.
FossilOrigin-Name: 17699329aa8e234c6793a38db5ca05def9395b287bcc8bc05493b824327edb59
2024-08-20 20:01:21 +00:00
drh 692c160360 Tighter checking of access constraints on union members in SrcItem.
Improved invariant checking.

FossilOrigin-Name: fd72d3400a8fe5747f494eee81654698acee350bb95b9db269e87d857af03492
2024-08-20 19:09:59 +00:00
drh ff176231da Fix some stray SrcItem field name fixes.
FossilOrigin-Name: bc5f5ce59e1e4323422bda7d002310923f927a03b4fd42749bf04f6c9853956b
2024-08-20 17:41:44 +00:00
drh 27c680cc69 Fix a memory leak so that now all dev tests (including TH3) are passing.
FossilOrigin-Name: 3995c45fff0e4bd10bcf49cc16eb1050216426723c63ba56d3ae5ba738c34019
2024-08-20 17:24:42 +00:00
drh ff4ad29d05 Memory issues fixes so that mdevtest now passes.
FossilOrigin-Name: b249516f6a6650808e035ec219e762bf24dcb7c32a6235984b995d08d07d35d8
2024-08-20 16:50:21 +00:00
drh 781163aa9d When the database encoding is UTF-16LE and the GLOB optimization is used,
it is ok to use the range search over an index, but it is not ok to
disable the actual GLOB function call.

FossilOrigin-Name: 3399698376761ab8c422f8ea02bfa2759afb606f08bedbd1cf7eee834229a9aa
2024-08-20 14:12:16 +00:00
drh 6ad4e9fd2a Fix for the UTF-16LE problem reporte by
[forum:/forumpost/dc7e1b7527e84343|forum post dc7e1b7527e84343].  Because of
the unexpected ordering of characters using the default collation (memcmp()
order) in UTF-16LE, the LIKE/GLOB optimization restricts its attention to
the pattern prefix that is all ASCII, which is the common case.

FossilOrigin-Name: a5797ebdea423afc3d2d3bd8adaf1575d33a01f594c0c315afcb1499f1718e9b
2024-08-20 12:09:55 +00:00
drh 1c8ed5fa43 Earlier error detection while processing complex aggregate
queries.  dbsqlfuzz 5242c2f07f4aa031aa3c80461f18e9b7619ede9b

FossilOrigin-Name: 70f4973078ffc72f4ff247234e6f8e695b40803c3e7c9ed12050d97195728352
2024-08-19 23:43:08 +00:00
drh 1521ca4c20 Refactor the SrcItem object so that information about subqueries is stored
in a separately allocated Subquery object.  This reduces the memory requirements
for SrcItem and makes the code run faster.  It also provides an expansion path
for subquery processing that does not burden simple queries.  The current
checking mostly works, but there are still issues that need to be tracked
down and fixed.

FossilOrigin-Name: 8ff5dda8448d7e1a533d7f27db2573ce68fa9956b9d9847ced45e83c1f06bab0
2024-08-19 22:48:30 +00:00
dan 127812e645 Update some comments that are processed into the fts5.html webpage.
FossilOrigin-Name: 3e06ab218bbd1ed75a24afb44e8df9ce84e9fc24701428cb8b3459760f44006d
2024-08-19 20:35:30 +00:00
dan 18557fdeb5 Add the fts5_locale() function and related functionality to fts5.
FossilOrigin-Name: b9632900100bdbc913f83bfb03b32585cf07a192b9a7f26b9bebc7d91e63a610
2024-08-19 14:33:39 +00:00
drh 922cf3e2e4 Fix leap-year handling for dates prior to 0400-03-01.
[forum:/forumpost/eaa0a09786c6368b|Forum post eaa0a09786c6368b].

FossilOrigin-Name: 474b7e34b83bc5e85854bc3b386f31ff39b390549d89b94921f33bbc5b658d1d
2024-08-19 14:20:18 +00:00
drh 5a18c1f9cd New date/time test cases to validate ancient dates.
FossilOrigin-Name: 82719074f090d9d0a0b9baea0dee7b1dd40a272e3f3e45043d4a640c07989d5d
2024-08-19 13:53:49 +00:00
drh aa2258e35b Attempt to fix date/time calculations for days earlier than 0400-03-01.
See [forum:/forumpost/eaa0a09786c6368b|forum thread eaa0a09786c6368b].

FossilOrigin-Name: 00cae11fffaf50e2133915e851d41098d00b6b1458e1438bcb6664e1411d8498
2024-08-19 12:54:12 +00:00
dan f5b1c3da36 Merge trunk changes into this branch.
FossilOrigin-Name: 6787603bd0d8117e41e6b572c0d7cada0f6d7266a4ff2ad79f9471b9beb28a9e
2024-08-19 10:40:08 +00:00
dan dd9a7e4540 Fix a problem with detail=none, content='' tables.
FossilOrigin-Name: 4b8070816ada3f269d53c9cee20669f59e19a23c31c8509fc397c21dd097ec9f
2024-08-18 14:46:13 +00:00
drh 901b671455 Automatically disable directory fsync when compiling for AIX.
FossilOrigin-Name: 8d170e07e20c21a5bb97fdc8c2d01d92178f5fe740e60d203e4621747cfce33e
2024-08-18 09:53:37 +00:00
drh b204b6aa7b Give unique names to fields in the SrcItem object, to facilitate analysis of
how those fields are used using "grep".

FossilOrigin-Name: 9f5aeef3cbc2c95267c8f7bf60d5c66971a76789669fb0e8f853273ff6f616f2
2024-08-17 23:23:23 +00:00
drh 8797bd695f Reduce the size of the SrcItem object by combining fields into a union.
FossilOrigin-Name: a4c59ac3c6ec979c25b544d29e47b8e39f6439c098eed8f84b3bd506c9adf047
2024-08-17 19:46:49 +00:00
dan 6c796631fc Fix a broken assert() in fts5_storage.c.
FossilOrigin-Name: 01116e32797e1b127ba79059d890fbe5075eee2b6e3ce4468e82862b0fc3aa46
2024-08-17 19:40:15 +00:00
dan 408a34925d Merge trunk changes into this branch.
FossilOrigin-Name: 6a8ff9ba5e71d817489093d8dff0a8d77365b4222773b941accbd58558d24379
2024-08-17 19:11:09 +00:00
dan ef2401f669 Tests to improve coverage of fts5_expr.c.
FossilOrigin-Name: f4b839e5265700b1a89066d1b6e0d0d010852a69c5da3d75d2c41624dbf3c0af
2024-08-17 19:07:13 +00:00
dan b651084713 Add tests to restore coverage of fts5_tokenizer.c.
FossilOrigin-Name: 8f9257361b05e368bf433e56d0698923b0f97d12e7c0ad7760aaab6746c0e467
2024-08-17 17:22:49 +00:00
drh 21363ac78d Adjust Makefile.in so that it outputs a slightly better error message on an
attempt to build something that requires TCL libraries when HAVE_TCL is false.

FossilOrigin-Name: a4043cbeb8a08fca2fdd2ea703e030d3a5574cc6002292ecc6f0e88c116472a3
2024-08-17 16:17:16 +00:00
drh f87a54ab41 Adjust the "configure" script so that it just disables the ability to
run tests rather than erroring out if tclsh is unable to recommend a
suitable tclConfig.sh.

FossilOrigin-Name: 793ff83d09bea165c8cedc978afd3fcfb1a4cee894c5914aa1d92416afce963d
2024-08-17 15:52:21 +00:00
dan 52d0310256 Fix a typo in an "ifdef" that broke SQLITE_OMIT_ANALYZE builds.
FossilOrigin-Name: c09da26cb6f772a18c595ff0eb55d8345a377c3ef5ed02a5e539ffdbe9d07f85
2024-08-17 14:59:26 +00:00
dan 342984075b Add tests to restore coverage of fts5_config.c.
FossilOrigin-Name: 9d971b31df7ad4d68eb348f95d8f996071cf87d41c47033bde3fcc4dba732e06
2024-08-16 19:05:47 +00:00
drh e11232a37a Merge all the latest trunk enhancements, and especially the
order-by-subquery optimization, into the bedrock branch, via wal2.

FossilOrigin-Name: d0591db3bb84c8f6325d3d5a44467f602cf6d8da6aef6456ef0bd3a19fca3d90
2024-08-16 19:04:36 +00:00
drh f5528a1168 Merge the latest trunk enhancements into the wal2 branch.
FossilOrigin-Name: a78208b597ee34b1121dd8014b3c1376b46baddff41448c96a37723cc9ede921
2024-08-16 18:58:50 +00:00
drh 65a46af7ae If a subquery has an ORDER BY clause and that ordering is helpfile in
satisfying the ORDER BY or GROUP BY of the outer query without doing an
extra sort, then omit or reduce the sort in the outer query.  Call
this the "order-by-subquery optimization".

FossilOrigin-Name: 7a0cdc7edb704a88a77b748cd28f6e00c49849cc2c1af838b95b34232ecc21f9
2024-08-16 18:51:46 +00:00
dan f0f240c0d8 Add tests to restore coverage of fts5_storage.c.
FossilOrigin-Name: 38841881a08f4d75c2dfcbc1b2b45e311d7dda1287780f8ba1c8388f7adc1629
2024-08-16 17:54:00 +00:00
dan db79dd1f88 Add tests to restore coverage of fts5_main.c.
FossilOrigin-Name: 681edd08ab7024ddcd6447a64d8aad581b47b039481b697d91d26e6c6660f373
2024-08-16 16:20:43 +00:00
drh b8123415b2 Ensure that memory allocations in ANALYZE are always 8-byte aligned in order
to satisfy a new assert() added by [539e4f661767ef90].

FossilOrigin-Name: a1915cbdeccac2ed49fb8a9733abf2c6239396e3174fed2ff275f66b606c48b6
2024-08-16 15:35:20 +00:00
drh 2ed4f5016a Fix handling of COLLATE. Add test cases for the same. Code cleanup for
improved understandability and maintainability.

FossilOrigin-Name: 41a41c173a9d15d94f23d73a5c04bfb1616cb9223bc81d41808f9b4d00817fbf
2024-08-16 11:26:21 +00:00
drh 42f997d768 Bug fix in the subquery ORDER BY propagator.
FossilOrigin-Name: 5a9a3b8af7ac0aa1c04ad2d735e341c92d67952acb9a1d30217c0471e92cd468
2024-08-16 02:19:17 +00:00
drh 743188642d Improvements to the subquery ORDER BY decision algorithm. Fewer false-negatives.
FossilOrigin-Name: b82421e3f5811946e9d60b845fc882b6ea5c53c648695654c6900754427bf9bf
2024-08-16 00:37:54 +00:00
drh 235b5d0ac5 If a subquery is materialized due to an ORDER BY and that ordering is useful
in helping to satisfy the ORDER BY or GROUP BY in the order query without
doing an extra sort, then omit the extra sort.

FossilOrigin-Name: 2fbb4dc2327ee435cb2b7a4adcddf5a9cee6dff7de96e2ecb761166427b5ddea
2024-08-15 23:38:52 +00:00
dan 41ebf965cd Extra tests.
FossilOrigin-Name: fb257540d32ced4a5fe0759d3ad9c1b36fd2b7d64d93cb0a0ecdb48355a68b7a
2024-08-15 20:33:05 +00:00
dan 189c41221d Further tests and fixes for this branch.
FossilOrigin-Name: d27985245a0e8c0d6b04323c98b26b6a8fb4e489fa8f5f3234252c7c198f23c8
2024-08-15 18:50:13 +00:00
drh 7c5a829025 Improved help message for --disable-tcl on the configure script.
FossilOrigin-Name: 96e6cfb211f7f1aab50f997f4bc61dfb9701beb720c59bc413a4285c1ae5e20d
2024-08-15 15:31:46 +00:00
drh 3f138e1d8f Reinstate the --disable-tcl option on ./configure.
[forum:/forumpost/845d4a57becc7f3c|Forum thread 845d4a57becc7f3c].

FossilOrigin-Name: de927016aadd7ee55d947134e3540907a3ea5ab4015034c5d088e3a84905d98a
2024-08-15 15:23:33 +00:00
stephan ff7200e75b Doc typo correction reported in the forum.
FossilOrigin-Name: 48900a867cc4b6d733d2a3533a553af3dfadf96d3ef7ddf4c72bd432e7bf0f76
2024-08-14 20:00:03 +00:00
drh 78f9ba9774 Improved documentation for the sha3() SQL functions in the shathree.c
extension in ext/misc.

FossilOrigin-Name: 0ce609103eb35cfa26d9b6fa1769679be7e2b9032c37cc8c00a194fe355b06a4
2024-08-14 17:58:46 +00:00
dan 8ff952773d Fix an assert() in fts5 that might fail with an expression like "... NOT ''".
FossilOrigin-Name: 317c91edfc69d1d47e233bb6ac13934241742fda1ba03a88fd51bdf98fcbc463
2024-08-14 14:26:02 +00:00
dan a7384e0de2 More tests for the new code on this branch.
FossilOrigin-Name: 00792e807f1dde750e6ac9f9b0095cf60b9fa6ff7cf4b14440600a21de2ee61a
2024-08-13 21:15:43 +00:00
drh 797fcb8433 Add the sha3_agg() aggregate to the shathree extension.
FossilOrigin-Name: c4d7f9996ce4f5f6d08d83fc3abdb81b6825cfb556318f90c6896736a1da5175
2024-08-13 21:04:00 +00:00
dan 7dad9b4297 Add tests and fix minor issues with the code on this branch.
FossilOrigin-Name: def5243a7b4f1d50427ba607d31332db2d6ac34b181b0cbd2ed4911064f5637a
2024-08-13 15:04:00 +00:00
dan 3e38aa0821 Add extra documentation comments to fts5_main.c.
FossilOrigin-Name: 1a7a9bd3f6a2b9347720f7f0386399e7f14b428500acf07d7c55b9dde1ec233a
2024-08-12 18:39:13 +00:00
dan a4a8e91022 Fix problem with some fts5 aux function APIs with (locale=, detail=none/column) tables.
FossilOrigin-Name: bf116dec6fb20cce69099cb8c031b0de511195482919a99ccfb3c498e9bf8125
2024-08-12 17:28:13 +00:00
dan 10e54e365a Fix further issues to do with fts5 locale support.
FossilOrigin-Name: e626123580065986f7df50b6140f00048944becce179b9391fbf09f97ac55485
2024-08-12 17:03:37 +00:00
stephan 57a0af406e Merge current trunk into this branch.
FossilOrigin-Name: 900a593ddf51eebddb70e2e9ab16abe4911f58f23e574e88ed340a919b8f16b6
2024-08-12 13:31:43 +00:00
dan 29d5e43ad4 Update the porter tokenizer to use locales.
FossilOrigin-Name: 3291ce3a3359a80e51e4546a3d7a187cbe4c7530fca6632f0bb2728525efe212
2024-08-12 11:46:09 +00:00
dan 1286677773 Update the auxiliary function API to include xTokenize_x2() instead of xSetLocale().
FossilOrigin-Name: f7d56a1f2149f0da117167db62e2c28ec337e8da3403873b64cdfc6a951e2e8e
2024-08-12 11:13:56 +00:00
drh 8c411f7fce Resolve parameters Bind $int_N and $test_T in fuzzcheck inputs.
FossilOrigin-Name: 9c10664416274df6f8da53ddd86f6356c9704ad798fd4034d2784ae433c1c056
2024-08-12 09:49:02 +00:00
dan 9b118de9d5 Merge trunk changes into this branch.
FossilOrigin-Name: b243007525a825b3daf8aa9bb2d3088efb853bd8b7c9ea3c0924fde193eb5c44
2024-08-10 20:03:01 +00:00
dan 195ef6baca Prevent the fts5 xPhraseNext() or xPhraseFirst() APIs from returning an out-of-range column number, even if the database is corrupt.
FossilOrigin-Name: d4014c87ba9b011a6a04c2bf85879b668dc762ebcbbfb50a2f8a417ce594ef88
2024-08-10 19:57:28 +00:00
dan 7978910f85 Fix an uninitialized variable in fts5_tcl.c.
FossilOrigin-Name: 924d3467cebd71ac8dfef8225965d71eda82ef1f0b43508af5b7ea78e2099cf3
2024-08-10 19:18:19 +00:00
dan ff6ab9dd2b Update the fts5_tokenizer_v2 API so that the locale is passed as parameter to xTokenize(), instead of via a separate call to xSetLocale().
FossilOrigin-Name: 03e63ed24e7a08817341e59b075ae2d4e3f7a5e5b37e0e6138359d5fd29a5e9e
2024-08-10 18:59:36 +00:00
drh 5af9fd53f4 Revision to check-in [d9f726ade6b258f8] so that OOM and other unrelated failures
are not overridden by a syntax error in the tokenizer spec.

FossilOrigin-Name: 3778b2a9ca1cc12a88ef6c32a1ee7c58a0a829ed9715a3d32a225d377d7527ef
2024-08-10 15:46:57 +00:00
drh 4a9a5cbbe9 Fix harmless compiler warning related to the TCL8-to-TCL9 transition. This
change in testing code only and is not relevante to the core SQLite.

FossilOrigin-Name: c86ef65545485eae6aca23b4582e8115d4edfdd6084afa77858e11b21aa270f6
2024-08-10 15:05:22 +00:00
drh 6010980914 Fix behavior change in the offset() SQL function introduced by
check-in [f0b671183f44d0ae].

FossilOrigin-Name: c2ac17f183082d6388336338b8d5c0b334095e5f77a27bc08419431f37471d22
2024-08-10 14:53:21 +00:00
drh fe4ed96006 Enhancements to covering-index prediction. In particular, avoid the
false-positive prediction described by
[forum:/forumpost/e60e4c295d22f8ce|forum post e60e4c295d22f8ce].
Add early detection of over-prediction of covering-indexes so that
sqlite3_prepare() will return an error rather than just generate bad
bytecode.

FossilOrigin-Name: f0b671183f44d0ae294956e7651a1653f47bd6219f9636872d15993f30f28dfb
2024-08-10 10:05:22 +00:00
drh 4098aa5d19 Add an assert() to fix a harmless compiler warning in the recovery
extension.

FossilOrigin-Name: 2502339339f3c831966479a400c2b1a531fb4d990b1776b4dc4870cf734c3fcd
2024-08-10 09:56:36 +00:00
drh 555aaa432c Fix a few minor TCL9 compatibilities issues.
FossilOrigin-Name: 69eed19914ef52bc0101f4ae1c7b242df4ee28675b3581e1f4d60a35494f9f76
2024-08-09 22:48:11 +00:00
dan 5eba4281b0 Ensure tokenizers registered with xCreateTokenizer_v2() can be accessed using xFindTokenizer(), and that those registered with xCreateTokenizer() work with xFindTokenizer_v2().
FossilOrigin-Name: a3ef7d47b582a66a85ebe802ab40114f77289d6b44a4b375ea7858c60f59b27f
2024-08-09 20:59:50 +00:00
drh c5900bfd59 Import the second ALTER TABLE fix from trunk.
FossilOrigin-Name: 1de3e5883613ff6093203720f3cdcf6b3a3e626c99735284fae95a438e1ce24f
2024-08-09 18:53:02 +00:00
drh c60c928717 Fix another instance where ALTER TABLE was returning SQLITE_INTERNAL instead
of SQLITE_ERROR one malformed input.

FossilOrigin-Name: b416756320dfb46ad72b843a7fbf18807d3fd1966251c320de626280b5238a5c
2024-08-09 18:42:01 +00:00
drh 8f924f0530 Merge the error code fix from trunk.
FossilOrigin-Name: 81dfb8cd5d59ecf974018b03175ddabd11366d2aca0211dc982f7e14ff314d4d
2024-08-09 18:22:20 +00:00
drh 610a5bafe2 Fix the internal routine sqlite3ResolveExprListNames() so that it returns
SQLITE_OK or SQLITE_ERROR instead of WRC_Continue or WRC_Abort.  Without this
fix, some times of obscure syntax errors end up returning SQLITE_INTERNAL
rather than SQLITE_ERROR.

FossilOrigin-Name: 9052187efd1f78abd9b285ac94269a3ba4d65b6aa74c9fa34002ee0628c83919
2024-08-09 18:18:11 +00:00
drh 8bc2647e0c Do not attempt to use a covering index based purely on columns-used
when doing an UPDATE or DELETE of a WITHOUT ROWID table.  Raise an
SQLITE_INTERNAL error if there is ever a situation where a table reference
cannot be converted into an index reference when the query planner thinks
that a covering index is appropriate.

FossilOrigin-Name: fae39ae9320c0f4fcc2e3b3baf0a4170ba2b9a04634a0a1fa41dd4918a74b106
2024-08-09 17:09:18 +00:00
drh 1107b4b9cf Bind the new debug parameters in fuzzinvariants.c.
FossilOrigin-Name: 7e1dc263051cf50db04643d1a2aa9f91559b2b121859b750ce4446012d5f3c3e
2024-08-09 12:35:27 +00:00
stephan caa7a62591 Remove unused static var cBadGroup from ext/consio/console_io.c to resolve a compiler warning reported in the fossil forum. This is a build fix, not a functional change.
FossilOrigin-Name: fa047c3ea7a03d76afd6a11f58ef7b84c653f583ed795e0de5be2a6b986e558e
2024-08-09 12:19:11 +00:00
dan 742f74ec19 Fix some signed/unsigned compiler warnings on this branch.
FossilOrigin-Name: 681d228023efb902b9039acf33600dd91f158e622d4dc0b434ed74f39ce0070a
2024-08-09 10:56:11 +00:00
dan 0632fb359a Fix some signed/unsigned compiler warnings on this branch.
FossilOrigin-Name: d615d5291871ba120d916c4722e6067e083f394c6a7f398059442b36c4500718
2024-08-09 10:50:20 +00:00
stephan e4d39e70c3 Remove two unnecessary int casts, as reported in [forum:84fe63b7b3|forum post 84fe63b7b3].
FossilOrigin-Name: 1de0c1b574d53c9bd2f2c6070b42e973d63d2a74f150836ed54c78614e286f92
2024-08-09 09:49:24 +00:00
dan 0c8c9a64b3 Ensure sqlite3expert.c unregisters any SQL user-functions it registers with the database handle before returning.
FossilOrigin-Name: 123b154ce3b6fee1bbf483704812bd6f8538966f9687520b4470d700f0270719
2024-08-08 15:26:11 +00:00
dan 679f5d458e Add assert() statements and reorganize code slightly in fts3 and fts5 to make it easier to follow.
FossilOrigin-Name: 797b0a13fd7a42b0a48ecbf0cd1961aa932da3e9c9ccffd903a3a4d963d0cc54
2024-08-08 15:07:27 +00:00
drh 1193e4679d Do not let the number of terms on a VALUES clause be limited by
SQLITE_LIMIT_COMPOUND_SELECT, even if the VALUES clause contains elements
that appear to be variables due to the use of double-quoted string literals.
[https://issues.chromium.org/issues/358174302|Chromium issue 358174302].

FossilOrigin-Name: 670beb133eb203065a75022f0c6db7c605a4e0e22c8ef6d6b4724be2663ff3dc
2024-08-08 14:45:50 +00:00
drh 35c6f58dce Back out the --memory option to ".parameter init" in the CLI, as the
capability is no longer needed for testing due to the new "$int_N" and
"$text_T" automatic bindings, and the "init --memory" is hence just
added complication.

FossilOrigin-Name: 3b1b0c141993eb0f8749f54ea40d6014f9bdccbe0fdb3ccccad971a0baea8d3c
2024-08-08 12:14:07 +00:00
drh 6e7a0c2d86 In the CLI, recognize magic parameter names "$int_NNN" and "$text_TTT" and
bind them with integer value NNN and text value TTT respectively.  Intended
for testing and debugging use only.

FossilOrigin-Name: a929cdb00ff74d77cdf55087d32abf53578f7127b36276f89b64ccbef1d4a9a4
2024-08-08 12:08:36 +00:00
stephan 5529ec74da Move a misplaced va_end, as reported in [forum:702c79e9da|forum post 702c79e9da].
FossilOrigin-Name: daa25fb35fd7bdd482d51214439f38b0b9a7df93f689a2b3d30a113daa9f2a1a
2024-08-08 10:51:56 +00:00
drh 6d02d2647b Add the --memory option to the ".parameter init" CLI command. When present,
the --memory option causes the sqlite_parameters table to be created in a
completely independent :memory: database.  This can be done to avoid
parameter binding queries from being affected by debug settings such as
".wheretrace", ".treetrace", "PRAGMA vdbe_addoptrace=on", and similar.

FossilOrigin-Name: 4e69dce2093b75b7db4fbdca4953b664b907be15d991ed352ea1d87c64fbf9d2
2024-08-08 10:10:38 +00:00
dan 5cd01c30b4 Fix a broken assert in fts5_expr.c.
FossilOrigin-Name: 08cc5488404d068e59378b82988460793710df43ec21b4a83a794b497abd035f
2024-08-07 21:20:08 +00:00
drh c76520c5b9 Add a new assert() to help static analyzers understand that a pointer is
never NULL.

FossilOrigin-Name: 433f2b942ee6f79d50ebe9b08fa3ea8162db6a10ce9d80e2bc193124baa1b083
2024-08-07 15:17:37 +00:00
drh 92d317f787 Fix a harmless compiler warning in an assert().
FossilOrigin-Name: 5e4c9a74b2efa74ace67f644fb6bb37e07f5982f8faaca9463e3daa50b0469d7
2024-08-07 14:54:54 +00:00
drh eaa560f3fc Remove unused test cases from FTS5, as they interfere with "grep".
FossilOrigin-Name: 048a71bae48aacdfc91e59330db4d4b3d663f00665f51a7110ea82e0508758ec
2024-08-06 23:09:37 +00:00
drh e9b919d550 Improved robustness of parsing of tokenize= arguments in FTS5.
[forum:/forumpost/171bcc2bcd|Forum post 171bcc2bcd].

FossilOrigin-Name: d9f726ade6b258f8723f90d0b04a4682e885e30939eb29773913e4dfc8e85503
2024-08-06 22:49:01 +00:00
drh 210f6f89be Improvements to ./configure that will hopefully break fewer builds.
FossilOrigin-Name: 769e32a69b7b7c04225afa0371f139b2ed29aaee5a7a4159a30d600ed9f25c57
2024-08-06 20:00:10 +00:00
drh 575a7b85f5 Change the AggInfoColumnReg() and AggInfoFuncReg() macros for
compilers that cannot parse empty elements of a comma expression.
[forum:/forumpost/8fd21998dc|Forum post 8fd21998dc].

FossilOrigin-Name: 533a6251f188805363f0e39613ea03b1bfd758eaea00e0855803238585bdfec7
2024-08-06 10:29:41 +00:00
drh f2b21a5f57 Restore legacy "tcl_install" makefile target, as an interim measure until
I can figure out what is going on.

FossilOrigin-Name: 931ad7d9e3aedb6d466fffc0af1f8d6b20d57a35dec644f28585f3e84f22d1ce
2024-08-04 10:13:01 +00:00
drh 3a799b3e4e Fix to the previous checkin: The colUsed parameter should have high-order bit
set if any of the 64-th or greater columns of the virtual table is used.  The
lower 63 bits of colUsed always show the usage of the first 63 columns of the
virtual table.

FossilOrigin-Name: 90ac8457750ace1d76d7bc957af7877e86e6301bb07361e19beaf5c7b3b6085a
2024-08-03 23:00:17 +00:00
drh 133f930e61 Ensure that the sqlite3_index_info.colUsed mask always includes the PK fields of WITHOUT ROWID virtual tables.
FossilOrigin-Name: ff4ac279b02190cc2107b0b110806dd7abcea47a6589d827b5ee39d4b80fb719
2024-08-03 21:45:46 +00:00
dan 5c85ddc3d7 Fix a typo in [c327c0c0] preventing the shell tool from running ".dump" scripts that feature virtual tables.
FossilOrigin-Name: 10e97abb878debb67410902ea1af4df75a9ced1e38fec710b1796c5143aff48e
2024-08-03 18:58:25 +00:00
dan 0a83dac31c Ensure the sqlite3_index_info.colUsed mask always includes the PK fields of WITHOUT ROWID vtabs in cases where they may be used.
FossilOrigin-Name: c327c0c02cfefdba373cfb15933a9cdfddb578b6582f2ce7c08929203743ffe9
2024-08-03 18:45:48 +00:00
drh 4a36d30493 Add the --status option to the test runs using testrunner.tcl. Only works
on Unix platforms.  The "exec" command appears to interfere with VT100
escape codes on windows.

FossilOrigin-Name: 94015cda4ceb4292ceceadb951fe5d9cb3e4e20403719b7254ad094a5b749ab3
2024-08-03 15:55:25 +00:00
dan e317e7f4b4 Change things so that locale=1 is required to write fts5_locale() values to an fts5 table, and so that blobs may not be stored in indexed (i.e. not UNINDEXED) columns of these tables.
FossilOrigin-Name: c98ccc12169419b8b27ead89ef0665de40320277c5daa748b80869337419e43e
2024-08-02 21:06:13 +00:00
drh 8517b7d2b6 Make sure the input buffer is big enough for an memcpy() when doing the
restore autodetection in the shell.

FossilOrigin-Name: 1637d29d518b3b4534a1b4c0dc0eddcb770f6f71763e4177812a3e79b97d2365
2024-08-02 18:20:10 +00:00
drh b2d590c5f9 Add the "-d SECS" option to the "status" command of testrunner.tcl
FossilOrigin-Name: 2151ea31632f7aa363c7d4b7205e1174a45d710783a8efad8176049d356cfc70
2024-08-02 17:36:34 +00:00
drh dcea6335c7 New MSVC makefile targets: tclextension, tclextension-install,
tclextension-uninstall, and tclextension-list.

FossilOrigin-Name: c38734f57509c50dd28029738dd602600950703b773d9b7b60fdb1dcb06e8fd8
2024-08-02 15:28:40 +00:00
drh 9e88d4bb8a Fix typo in a progress message in the builttclext.tcl script.
FossilOrigin-Name: 5e7c94645632b736e805428ff4c135666ab525fac99c29c12d5127b06f39fc6e
2024-08-02 13:56:55 +00:00
drh eb48fddac7 Fix problems in the configure/Makefile system caused by recent enhancements
and report by [forum:/forumpost/242ebd0c81|forum post 242ebd0c81].

FossilOrigin-Name: a59319fe4adc094381c46e490289b3e0cda8f804edaea6984571c47a68a23eef
2024-08-02 11:32:28 +00:00
drh 698386ca2c New configuration option: --with-tclsh=FILE. New makefile targets:
tclextension, tclextension-install, tclextension-uninstall,
tclextension-list.

FossilOrigin-Name: 888284d1d47269e66e23627e2519cd2cf83ef65a777a679af20c288453f092e2
2024-08-02 00:59:32 +00:00
drh ea9a25e500 Previous Makefile.in change broke the tclextension-install target.
FossilOrigin-Name: 7748367c78a1f480ec9e892ba77652e3c31a4b387b89221ae584891818804851
2024-08-02 00:56:31 +00:00
drh 0c059b598d Continuing refinements of the tclextension-install makefile target.
FossilOrigin-Name: 871c20a848e0e9b7f3d5661bd076fde16e9ec1e445a7ee657ef267978988ad56
2024-08-02 00:51:52 +00:00
drh 490661fd96 Extra robustness for the SQLite Tcl Extension builder, so that it works
on Debian Linux.

FossilOrigin-Name: 2f0738daaff48c51a60f063af6e453443bd7eedf6936daab325b2a36927c7a2b
2024-08-02 00:19:59 +00:00
drh bb99e73ef5 Simplification the ./configure script. Carry Makefile CFLAGS and OPTS through
into builttclext.tcl.  Fix a bug in tclsqlite.c introduced by the previous
check-in.

FossilOrigin-Name: 72a80aa0abb121a23f8cc1eb1af04ba06c733a7a40c06f1fe9e62f2a0a26ca6e
2024-08-01 23:30:29 +00:00
drh 58c5a95c07 Add the tclextension-list makefile target. The the makefile running on Mac.
FossilOrigin-Name: 2f667cc8e54167342e17e63e1a017388365ebf4cd379f9c16c12e4603b34a326
2024-08-01 23:00:37 +00:00
drh 1bed50d424 New makefile targets: tclextension-install and tclextension-uninstall. Seem
to work on Linux.

FossilOrigin-Name: 534f8344abbda78e841edafe84c27ea8b1dfdecb9ab51271772a7f7264ef5b8c
2024-08-01 20:26:22 +00:00
drh 24b1d7a0c8 Work on a TCL script that will compile and install the TCL extension.
Works on Linux only, so far.  This is an incremental check-in.

FossilOrigin-Name: 58babd2aa4b5c9c70e1169cdf09f05eb95d7b13f50bb2ec0e487182288172678
2024-08-01 20:18:10 +00:00
drh 91db27aff5 Clean up and restructure the TCL configuration sections in autoconf.
The new --with-tclsh=FILE option lets you select the specific tclsh
that you want to run.

FossilOrigin-Name: 8fafc679d91e857602fbbe203144ca98242246660ede5a49c5fec8aaff571151
2024-08-01 18:10:00 +00:00
dan f971c15b1f Clarify the role of Fts5Storage.pSavedRow in the new feature on this branch.
FossilOrigin-Name: e8a61d5c48073fdd4d99d0b6fc70469b37af009f281336a44e3789e7eeed820d
2024-08-01 17:15:17 +00:00
drh a51de65758 Include a test case summary in the output of "errors" in testrunner.tcl.
FossilOrigin-Name: 173df1478e89996126e172656e35da8026d4ef145b2341ef56213f00ade14f48
2024-08-01 14:43:27 +00:00
drh 5cc7f4b580 Add the "errors" command to testrunner.tcl.
FossilOrigin-Name: 03b7f99229cfcffde024915e6812d75ae11318db9dc93a00838e4588a79b1243
2024-08-01 14:16:36 +00:00
drh d214af0b4a Merge the latest trunk enhancements into the bedrock branch.
FossilOrigin-Name: c1f616ce802fbb9c3652f8cb43970e4bda18464e765f23fb5f96029721431092
2024-08-01 00:47:59 +00:00
drh 76e67700e0 Merge the latest trunk enhancements into the wal2 branch.
FossilOrigin-Name: eac582056fcf6210bf90576cce30c087503b864dc8d558f2998f04b3d1bfb569
2024-08-01 00:38:59 +00:00
drh a1291e79d3 Fix the tclsqlite.c TCL interface module so that it works with both Tcl8 and
Tcl9.  Make changes across the TCL-based test harness to enable SQLite to
be tested with either Tcl8 or Tcl9.  Get the --with-tcl= argument on the
configure script working. Testing changes only - no changes to the SQLite core.
I believe the previous merge attempt didn't work because of errors in the
merge itself, not because of faults in the code.  Trying again...

FossilOrigin-Name: ea9d88f9ca3399bca83bf03893689a927b73e481604b94527e42de43f103eb46
2024-07-31 22:36:14 +00:00
drh b030bc6980 testrunner.db should be in WAL mode for improved concurrency while running
"watch sqlite3 test/testrunner.tcl status".

FossilOrigin-Name: f23ad0979865f3c11a3dbd0598ee7c4e2eabeaf1863d000fc0242e18890a4a11
2024-07-31 22:26:04 +00:00
drh 5addf9f111 Make the #include section of tclsqlite.c the same as tclsqlite.h.
FossilOrigin-Name: 2afadbeb124d6d7bb81d473379269014ebd8601f98f231e2218f54350a2e5fd3
2024-07-31 22:25:25 +00:00
drh d888e79bb8 Fix typo in testrunner.tcl for the "status" command that was causing it
to fail if any errors were seen.

FossilOrigin-Name: 8327c5ed535c57c4082211a50bfb1c32ea5d5fe1cf71f37798d8b58e88948ac2
2024-07-31 22:17:26 +00:00
drh 3f91c05e5d Use <tcl.h> instead of "tcl.h" to include the TCL interface.
FossilOrigin-Name: 4e32d4aed7c487ee9ed39a7e4486c2989a01781a3edbe9cdce52c6ed357f6366
2024-07-31 21:59:29 +00:00
drh 47ce12d239 Fix the tclsqlite.c TCL interface module so that it works with both Tcl8 and
Tcl9.  Make changes across the TCL-based test harness to enable SQLite to
be tested with either Tcl8 or Tcl9.  Get the --with-tcl= argument on the
configure script working.  Testing changes only - no changes to the SQLite core.

FossilOrigin-Name: 2a04b1bea12b0b4c3387d675f487337533ac0813248f70085398485e4cacd691
2024-07-31 21:46:46 +00:00
dan 5bd8cc7dd5 Fix various problems with the code on this branch.
FossilOrigin-Name: 8bd4ae7e95c7b6ce34db5ea705dc136e742a22f333d0e7370b485ebd736b5ec2
2024-07-31 20:49:00 +00:00
drh ed1b616f4a More tweaks to testfixture-implemented TCL commands to make them safe for Tcl9.
FossilOrigin-Name: 67897ca97d8c98e119a498a3adb39548a579e12ee477e9dd74e9564a6f5e4c2b
2024-07-31 19:57:22 +00:00
drh a7382d3523 Under Tcl9, the %x format letter truncates to 32-bits. Must use %llx in order
to get a 64-bit rendering.

FossilOrigin-Name: a3044e0ae65c099d76bf04536e223e70d89971688627c342252951f6c65f49d1
2024-07-31 19:56:14 +00:00
drh 15fef79a29 Fixes to testfixture routines needed to test snapshots.
FossilOrigin-Name: 66f61c3b35383b2575774d309043a722cc87a53d03518d339d6dbdcf342adc93
2024-07-31 17:15:24 +00:00
drh ff79000f28 Fix a typo in testrunner_data.tcl that prevented --with-tcl specifications
from being carried through into sub-builds.  Adjust test cases to account
for new floating-point differences discovered when --with-tcl was fixed.

FossilOrigin-Name: b8f81351914deac99b14be6bb5e4096c1efed9bdb057750825760f09dae04656
2024-07-31 16:37:02 +00:00
dan 8e3042b565 Add missing documentation for new features to fts5.h.
FossilOrigin-Name: fa0da3b28e411affd45a918d6b7faba49f744ca8b4adf3cd5ce6609bb630499e
2024-07-31 15:46:41 +00:00
drh ad2bcef112 Remove obsolete testfixture TCL commands formerly required by Windows
but now handled by Tcl itself.  Update tests case accordingly.

FossilOrigin-Name: 4f065fb638b24b33283518af8ad55f28a49cfac4086fa5e4568e2f3a6814962d
2024-07-31 15:04:37 +00:00
drh 3f80dbd51e Fix problems uncovered while testing Tcl9 on Windows.
FossilOrigin-Name: d5523c77fd0bcdc8344971a116d6ce9657f2b6daddeb7d936cd7607163a36744
2024-07-31 13:00:18 +00:00
drh 096c22ad5e Minor adjustments to get Tcl9-based testing working on Macs.
FossilOrigin-Name: 066c8672ee85a6ce972de38fcd3793fc2c1ef4f21c3de2379746d0ff18df8dc0
2024-07-31 12:19:16 +00:00
drh d3b58968a9 Adjust the configure script so that the --with-tcl option is used to
find the appropriate tclsh.

FossilOrigin-Name: 6fb9e37c751cd1a411c77f792794e1abafa5c40ad4dda8d433600d44d525a97c
2024-07-31 11:42:47 +00:00
drh f15c80d80f Patch to testrunner.tcl so that it works with Tcl9's pedantic UTF-8 checking.
FossilOrigin-Name: 8d4a23d32dc16b115153e6e100ba3ba15273c3e37310e8a1b4bace885c5383a8
2024-07-31 11:12:13 +00:00
drh e388fc7367 Fixes to tclsqlite.c to facilitate TEA installs.
FossilOrigin-Name: 7157c3c910565c9385a028793a0c0106c5064ab6a8a6efbe29dd1f1cdb2bd48d
2024-07-31 10:59:19 +00:00
dan c97cdbe523 Merge latest changes from trunk into this branch.
FossilOrigin-Name: ff64939ec8399949393f1029fa6d514892fbf2bf1498545300cc5e120b9622a5
2024-07-31 10:52:38 +00:00
drh 4400b0ac45 Fix a problem with strictly conforming UTF8 in shell1.test.
FossilOrigin-Name: 8ee187f98d310e1e7cf6c3fdf000c19e98a9f11e01436c3c407df04da79aa7c0
2024-07-31 02:01:19 +00:00
drh 3bad3d6964 Adjust a few tests that don't work even with the new floating point
comparison routine.

FossilOrigin-Name: 895125a4895099df7c3841ca7c34697ba17fd667bddbc2fd7148f7aa9d6d2b9d
2024-07-31 01:53:48 +00:00
drh f504cc761b Implement a new comparison routine to compare text renderings of
floating point values.  This gets the number of failures down to just 5.
But the routine needs work, and probably refactoring.

FossilOrigin-Name: 71f2ee5db01150707401804b136641170e7ed44760fccec20de19184e4d0a840
2024-07-31 01:45:14 +00:00
dan 9015b8822d Fix cases surrounding blob values being stored in locale=1 fts5 tables.
FossilOrigin-Name: ae435aff1785d5832821c19dc88ccf6c496a7ff55be80276b31e5c9abee723db
2024-07-30 20:39:58 +00:00
drh 262addd9ab All tests in "make test" now pass with Tcl9, except for about 198 cases where
the error involves small differences in the rendering to floating point
values.

FossilOrigin-Name: 8e27f5326c69aa4fb6f3f1f42668ab1b08140ab0a614ac4a9d94679c8fb9734e
2024-07-30 20:39:05 +00:00
drh a28a784ef4 More adjustments to the incrblob channel: Apparently you need to wait
until the close2Proc is called with a flag of 0 before actually shutting
down the channel.

FossilOrigin-Name: fa549a5507b5f805b469b4360c11155aa9d22043f01f6d9428bf44d0f2351eb0
2024-07-30 18:49:20 +00:00
drh 15669d302d Adjustments to test cases for session so that they work with Tcl9.
FossilOrigin-Name: d417415b967e62d41667e14ebf5455932ebf38d218a941f13878f0aba6d19bc8
2024-07-30 18:42:31 +00:00
drh 1c2ad465c6 Tcl_ChannelType implementations for Tcl9 apparently require that
wideSeekProc be implemented.  Also adjust minor test script issues for
fts5 tests so that they can be run sequentially and so that they do not
depend on the specific floating point output formats generated by Tcl.

FossilOrigin-Name: 19fda979c5dc1a385ed3f8ab8df34388c1acfc7ff951fe1b183a79186bd20cdb
2024-07-30 18:15:59 +00:00
drh 780187f090 Get rid of "-encoding binary" across all test cases, as that is no longer
supported in Tcl9.  Use "-translation binary" instead.

FossilOrigin-Name: e0d9670dd98a22d2e22aac8ba8720787a05e280e18ec9b2d4323f947c83718f5
2024-07-30 17:09:51 +00:00
drh 3302bf4dbd Use the SQLite format() function to format floating point values for test
results, to avoid inconsistences in TCL versions.

FossilOrigin-Name: 6a63c472190dafb36582c116c925bbcbe48c3a2f3217fc1ffcb64a733b0151e1
2024-07-30 16:59:33 +00:00
drh 1631c64f78 Fix the incrblob channel so that partial close works with Tcl9.
FossilOrigin-Name: d54ec2de78c2aed12be643e59a30b6827fe4728a0ab6504dd64ee9116e5f885a
2024-07-30 16:51:51 +00:00
dan 8174305e17 Update the highlight() and snippet() functions to use locales when available.
FossilOrigin-Name: 569ae56a673a2e101fac7003a1cb41c7a02a515adf55bd1e1f2a03c19eb2b085
2024-07-30 15:55:51 +00:00
drh 064b681e9b First attempt at getting the build to work with Tcl 9.0.
FossilOrigin-Name: 6e5bb48a74d63fb8c30528f0005d1763cd2dbb882abf86baf1565721e6bfcf84
2024-07-30 15:49:02 +00:00
dan 8f14c45197 Fix xColumnText() and xColumnSize() APIs. Add xColumnLocale() and xTokenizeSetLocale().
FossilOrigin-Name: 14006711d83d098e665c540b978b0e29aa8f458da1c2c8e9c2baf2ad5ebd502c
2024-07-29 20:31:17 +00:00
dan b4afecc626 Fix a couple of asan problems on this branch.
FossilOrigin-Name: 4e3e529313c86b42d5f1ba113d0714d1545bc6641e09e4537d622057f0352989
2024-07-29 15:00:11 +00:00
drh 07f215ad9e Fix documentation typo.
[forum:/forumpost/993cb82402|Forum post 993cb82402]

FossilOrigin-Name: 86de4e755e37dc1cbcbd59018927aa87ff49fc15f706a36187631d8f14075c12
2024-07-27 20:28:13 +00:00
dan 53bef48fbe Ensure that an fts5 UPDATE does not strip the locale from columns that it does not write to.
FossilOrigin-Name: 69205264debd829573b1c777a5a493cfeb6083c4cdec106b1f819989f859ac75
2024-07-27 19:46:10 +00:00
dan 2ec78c0e4b Add the fts5_locale() function, and begin adding the related functionality to fts5.
FossilOrigin-Name: 8839ef7cfb49239e7f1c4812a53a93a672827c88d6921408b1d5062b352c87cc
2024-07-26 20:50:33 +00:00
stephan d35a69a8da Fix a non-bare-bones-mode test case broken by the previous checkin.
FossilOrigin-Name: 3be729493b76a8b51c19d81543dea2dcb1fc718f81d4f2d9c6e1381789217687
2024-07-25 16:28:55 +00:00
stephan e7840ce681 Strip progress handlers and window functions from the wasm bare-bones (formerly 'minimal') JS bits, noting that we can't yet use OMIT_WINDOWFUNC (for the C parts) without a custom amalgamation. Currently at 604kb.
FossilOrigin-Name: ec02e9237e1ef81c4196fa630822cb109eab926143ad09593a24273eb0668601
2024-07-25 16:21:19 +00:00
stephan 520d1a8486 More work on the minimal-mode wasm build (now 603kb uncompressed). Remove the hard-coded feature-enable flags from sqlite3-wasm.c and rely on the build to provide them. Some wasm build cleanup, but attempts to completely overhaul it have been thwarted by my inability to make script-generated makefile code more legible/maintainable than the current eval spaghetti.
FossilOrigin-Name: b029c4067943e366a9b25b8303136fab10822bd771ea4658ac4cd716ff4a0d8f
2024-07-25 14:00:26 +00:00
stephan cc65612e35 Start work on an overhaul of the wasm build process, with an eye towards less over-engineering.
FossilOrigin-Name: ed746b3dd3248b68cb91de50ac5ba5fd3a7c2fcbde76324e86b88edbfecd896b
2024-07-25 10:50:45 +00:00
stephan b16c2980b3 wasm minimal build: strip authorizers and JSON support (saves approx 35kb). Strip vtab support from the JS bits but cannot yet strip it from the C bits because that requires a custom-configured sqlite3.c.
FossilOrigin-Name: eb64d106551718467e0f6c6b53695410bf4c566901008e4cda8580d0f7efa7b0
2024-07-24 23:58:28 +00:00
stephan fe225745fc Restructuring of the wasm build to support an experimental 'minimal' build mode which elides all non-core APIs.
FossilOrigin-Name: ee2191f7302210100fa0b29ace8156531ad995bf61aa2642e526e0901d0c6862
2024-07-24 22:07:18 +00:00
drh 4f2144dac0 Add the percentile extension to the CLI by default.
FossilOrigin-Name: bcc31846964102385d5a21eb5e85d7db153b155e76b4e2847c9453d3d0e1af04
2024-07-24 13:53:51 +00:00
drh b18d851119 Add the percentile_cont(Y,P) variant of percentile() to the percentile
extension.

FossilOrigin-Name: 095c22e62248f8ef50cd8531171827f50a7bdd4fc1128bf0e616a3eb2dce980e
2024-07-24 13:41:09 +00:00
stephan 4eabec5b18 JNI SQLTester: fix --new command to force-replace db if it already exists. Add no-op impls for --stmt-cache and --jsonglob commands.
FossilOrigin-Name: f1185dc349d076a6b8ea22f9b5cd527bfd60e9afd55eaec754c0e2b4d7edd5d4
2024-07-24 12:12:11 +00:00
drh 171c944345 Enhance the percentile() extension function to include the median()
variant.  Update the implementation to implement its own sorting
algorithm, so that the extension no longer depends on qsort().

FossilOrigin-Name: 6e31b1bab1f014933c671a12a5930c1e88d611cfe68d389e9ce888bd8842addd
2024-07-23 16:23:46 +00:00
stephan 8bd5ff4f32 Doc typo fix. No code changes.
FossilOrigin-Name: 8d558ad25bfbdea04de87616d4e3f664b5749a7d23643d1a0238e991b4bb337e
2024-07-22 21:46:55 +00:00
stephan faf10c521f Move a block of JS code which was inadvertently (and harmlessly) moved in the previous checkin. Clarify the semantics of an internal-use-only API. Add another WAL-related JS test.
FossilOrigin-Name: 6cd9f55a975b5237efee8776efce7e7836b41905ca750f82be3b90aa04c1dff2
2024-07-22 21:21:25 +00:00
stephan 71e2bdb2b4 OPFS VFSes: remove the on-open() pragma calls, as those (A) already reflected the build-time default settings and (B) they made it illegal to run locking_mode=exclusive, which is a requirement for WAL mode without shared memory. Modify part of the test suite to demonstrate that the SAHPool VFS can run in WAL mode so long as locking_mode=exclusive is used.
FossilOrigin-Name: 19cd8e2b056d7842ee39afb7160c901c9dc55a5bac8049cb0b5246210f6b920d
2024-07-22 20:58:51 +00:00
stephan 04416ddc51 wasm build: resolve a circular dep and do some minor tidying up.
FossilOrigin-Name: 9df3f1f24c6346dc94695bf533501c54379bb6e3cf492b67dda8a64a6a1eb495
2024-07-22 19:52:02 +00:00
drh 6326e30f86 Fixes to TreeView debugging output for VALUES clauses.
FossilOrigin-Name: b6aed8bcb06edc7f0221fd707d5adc227856fe62dbcaae5ffe1fb4faa5c542e1
2024-07-21 23:34:52 +00:00
drh c855df67a4 Improved debugging output when using PRAGMA vdbe_addoptrace:
Show the range of opcodes being checked for Column-to-Copy conversion.

FossilOrigin-Name: 2c122026db979a099a3d1a14ae1e01a0a64a559905ea414b2d80ed7dc3e04846
2024-07-21 23:10:55 +00:00
dan aecfb0a3b3 Fix an oversized allocation in sqlite3ExprCodeIN().
FossilOrigin-Name: d7be326a80e7b3aa8fd6e5e059c04e6ad3feaffcb20b1e3c251d7195ddc1be8b
2024-07-20 16:11:12 +00:00
stephan f73912694b Minor typo fixes in test-script-interpreter.md.
FossilOrigin-Name: bf54b26092ded2e6acc779acfb960364f05c665c7626c38ffae61caae5636184
2024-07-18 19:17:29 +00:00
drh d109de8f84 Fix a problem in FTS5 where it reports the wrong error code following an OOM.
FossilOrigin-Name: 0c1cc4071edcd3e465779bbb17ea3ee6ddaa7c93c4fde55e5f3e07aa4b45783b
2024-07-18 18:44:44 +00:00
dan 9d82438dcb Fixes for platforms with 32-bit pointers that require 64-bit values to be aligned.
FossilOrigin-Name: 539e4f661767ef90aef8d995f1f75fd5f550703bba1517cd3caff84a4afe2b37
2024-07-17 18:38:34 +00:00
dan a592883d87 Add extra assert() statements to the fixes on this branch.
FossilOrigin-Name: 81b6360050eea95e4367de1b41b5864a640b4d1c5c8fc3bea3b96ed770cb0325
2024-07-17 16:27:36 +00:00
stephan 1ffd7ed54f Slight doc touchup for [af41a1e6fc8b36e9bf65] based on feedback. No code changes.
FossilOrigin-Name: be7b1fc0c5d8cda971b471dfcb4637212bfff4f42b1e074077a381cc493d877a
2024-07-15 10:11:21 +00:00
stephan a0a1fb876f Improve one of the JS test cases.
FossilOrigin-Name: 3767d8bef2dfa661b56c1bfaa4695e32ce26874154e4a322f0d599c257c6fc70
2024-07-13 18:16:40 +00:00
dan 7acf972c59 Fixes for platforms with 32-bit pointers that require 64-bit values to be aligned.
FossilOrigin-Name: 2212d7488ed4ec2839ffa45cb9567056b36519434834634e4ecc441c330694d7
2024-07-13 16:53:56 +00:00
stephan 441993200e Add JS binding for sqlite3_column_decltype().
FossilOrigin-Name: 5252d292b14f6ed99b09f6330b19effd488c2b23ff6ea33311321a065d3f6ce2
2024-07-13 15:42:30 +00:00
stephan 7052c67c36 JS doc touchups. No code changes.
FossilOrigin-Name: a8e240495ff7e0cd38301ef2de215a7cc10fea832311bd19ca2592858ee4e6ff
2024-07-13 15:21:35 +00:00
stephan 55a4bea72a Add JS bindings for sqlite3_stmt_busy() and sqlite3_stmt_explain().
FossilOrigin-Name: b772edfb44143107d4993bde600d0f4c45184f29a4deee403105c64748c36523
2024-07-13 14:07:47 +00:00
stephan 0b9efaffd7 Add missing sqlite3_bind_parameter_name() binding to JS.
FossilOrigin-Name: 6dcfcc7e1c0772b11aec750bb75899a5c8e452735ecf5028c001fbaa7aa6fda0
2024-07-13 13:22:32 +00:00
stephan 6f9b37b35b Add JS bindings and tests for sqlite3_interrupted(), is_interrupted(), and db_readonly().
FossilOrigin-Name: 5589ba5651d80479ae555429340fc74390ff07d3e05ba770bb7d82490211e00d
2024-07-13 12:50:42 +00:00
stephan c636e124df Remove a stray/harmless duplicate wasm function export entry.
FossilOrigin-Name: 5402e4fb1cbd7a2c3dee6ebfb492a076d05daa66f747d18602338874405829df
2024-07-13 11:47:44 +00:00
stephan db8939f481 Add several convenience links to the OPFS VFS concurrency tester app for launching it with various combinations of options.
FossilOrigin-Name: 397d6a18f173a46b60a867fc45611045c76316d6602a3df8b4dc4ba45e1348ce
2024-07-13 01:40:36 +00:00
stephan bdc4ffacec Add an option to the opfs-sahpool VFS which tells it to ignore a previous init failure and retry, as an opt-in workaround for a browser quirk which occasionally denies OPFS access on the first attempt and permits it on subsequent attempts. This resolves [https://github.com/sqlite/sqlite-wasm/issues/79|issue #79 of the npm distribution].
FossilOrigin-Name: fbf3948a4ba27c6ebf55b24e7463b20026439f04d1974bafe78df5c5bc958f59
2024-07-13 01:02:57 +00:00
stephan f7623a0871 Rename the new opfs-sahpool init-retry flag to the suitably descriptive, and suitably onerous, forceReinitIfPreviouslyFailed, based on feedback.
FossilOrigin-Name: c9f0a6090258534a3d81066df5610229183a8457fc60d5034b4b94d9a29796e4
2024-07-13 00:55:55 +00:00
stephan 2e32cbc4a5 sqldiff: fail if the 2nd db argument cannot be opened read-only.
FossilOrigin-Name: e8b33525fc2b4d609a7be9acc43fbc66638effaa7b092b48568ded6c53c4f4db
2024-07-12 14:41:16 +00:00
stephan be3778dee2 Work around a difficult-to-trigger Atomics API message-passing quirk in the OPFS VFS which appears in rare instances in some browsers when running high I/O loads. This resolves [https://github.com/sqlite/sqlite-wasm/issues/12 | issue #12 of the npm distribution].
FossilOrigin-Name: af41a1e6fc8b36e9bf65a5bb0154e1ce7eb99903cb5a3e4779322527c29d8780
2024-07-12 13:49:54 +00:00
stephan 74d5faec93 sqldiff: if the first db argument does not exist, fail instead of creating an empty db. Resolving that for the second argument is trickier, as discussed in [forum:ec2d429e32 | forum post ec2d429e32].
FossilOrigin-Name: 0547ccf776c6054732437bffb8b2fe2ed5194ef817c2593f8cec4a3e2b749720
2024-07-12 13:45:15 +00:00
stephan 8ee747dda5 Reformulate the forceReinitIfFailed handling based on feedback, to correct the type-checking and resolve a potential race condition.
FossilOrigin-Name: c4f468309158f9b951137461213fb269d96d15c331256cf392a56c9bed231460
2024-07-11 19:50:17 +00:00
stephan 20ff50ad10 Remove the timing and counter metrics from the OPFS VFS, as they're long-since unused so are just wasted CPU cycles.
FossilOrigin-Name: 67c035c34fb916e66bfe115a132660771e8fa2921e6d46756975c5df3ebcd73c
2024-07-11 14:07:46 +00:00
stephan 9b183a234c Add some docs explaining [64f4f3cd878b1a72].
FossilOrigin-Name: 4fdf9a322fccf7d68b03d07a42a526f18f951e2e19228b7d41e01121b667cc86
2024-07-11 12:57:58 +00:00
stephan d3bb31b931 Proposed workaround for the "errant op" case in the OPFS VFS, as discussed in [https://github.com/sqlite/sqlite-wasm/issues/12 | issue #12 of the npm distribution].
FossilOrigin-Name: 64f4f3cd878b1a723caa0cd54b21f10fdb27d4e2ecb19fc233cbc8dd9d06f259
2024-07-11 12:36:28 +00:00
stephan 9958c51a5d Add an option to force the opfs-sahpool VFS init to re-run after it fails on a first attempt, as a workaround for flaky environments where initial access to OPFS sync access handles is rejected but then permitted on a second attempt. Reported and discussed in [https://github.com/sqlite/sqlite-wasm/issues/79|issue #79 of the npm distribution].
FossilOrigin-Name: 5286e0f654d91a4ebee51fcabaab696e17ff07bb18990b401a31bd3d1213e695
2024-07-11 11:04:17 +00:00
stephan 64ef4582c2 Rename some JS files for consistency. This affects only the build process, not the deliverables.
FossilOrigin-Name: bcef3f71a2f68768819d9f716f2c29e752fb173df1506469c8669d95ecc2ff50
2024-07-10 08:33:52 +00:00
dan 68d700cac6 Fix a use-after-free that could occur following an OOM introduced by [d8cedbe0].
FossilOrigin-Name: 4fabfacfcf38e129949d3e4e2c3ffde3da3cd40d9d12c97ca29bc7c3604db6ed
2024-07-08 17:39:52 +00:00
dan bee10b987c Improve the error message returned when fts5 finds a row is missing from its content table.
FossilOrigin-Name: 9b1afd4acb8cf9738154e921dcb074d895bbcc4742bf1016ef9f045a0da78bc8
2024-07-08 16:01:49 +00:00
drh 42123a294d Use a mini Bloom filter to help reduce the number of pointless searches for
prior SubrtnSig objects when generating code for IN operators with subqueries
as their right operand.

FossilOrigin-Name: d8cedbe055b40a0ea4e5d47845b535162e9fcb0d0f88c03211797ab64d2d56fb
2024-07-05 13:55:59 +00:00
drh 635c6b8086 Improved reuse of subqueries associated with IN operators, especially when
the IN operator is duplicated due to predicate push-down.

FossilOrigin-Name: c9a3498113074bbcd9a8c8d30286fef6c6a49ad2c84b90ec0f5a148389d6245c
2024-07-05 09:56:11 +00:00
drh 615fc1f083 Small performance optimizations.
FossilOrigin-Name: 99fd34b58a9b09af20f1f1a3c37137ae55159310c8af97806550eeee6adc4fc5
2024-07-05 01:05:34 +00:00
dan a61f11f307 Add test cases to in7.test.
FossilOrigin-Name: 07817efc10a6ab59dcb18ad218a3bb5a5d49a724c51296d5c8d7e386b0e0c789
2024-07-04 18:56:26 +00:00
drh 42305fc126 Disable the reuse of IN-clause subqueries if the subquery is an explicit
"SELECT ALL".  The ALL keyword is almost never used in actual practice (most
developers don't even know it can be used) so this should not interfere with
the optimization, but it does give us a convenient way to turn it off for
testing purposes.

FossilOrigin-Name: a81299be2ce203dcc28e7d0ba24791cbfba80e1ee3e1564469a226cac8adb17d
2024-07-04 18:26:41 +00:00
dan 6c23f1941b Update EXPLAIN output to include P4_SUBRTNSIG.
FossilOrigin-Name: 61e56923c80d72d6467ec703b4946b251abf591649fa9e8e579667d7cbd25682
2024-07-04 17:49:34 +00:00
drh 0cf237c5b0 Be more aggressive about reusing subqueries that appear on the RHS of IN
operators that have been replicated due to the predicate push-down optimization.

FossilOrigin-Name: 2accf32b6e45a396503c29eecc14a103bcc7b4c313cde921b26b489704060177
2024-07-04 16:57:11 +00:00
drh 6357d35da9 Add comment using the name "predicate push-down optimization" to what we have
also called "WHERE-clause push down".  No changes to code.

FossilOrigin-Name: be77fe701c7b7d3407800e1ed6a4c8be14035831790419cc99362bff41c65b60
2024-07-04 11:15:32 +00:00
stephan 9e278d7ec0 For shell completion, use pragma_table_xinfo instead of pragma_table_info, so that generated columns are handled, as reported in [forum:f0735e05d8d7e857|forum post f0735e05d8d7e857].
FossilOrigin-Name: a204ffc06b468c2edf8f11ccf3de639edd4f8282e69a44ceeb68a4d3a43e77ea
2024-07-04 09:45:23 +00:00
drh ba384470c4 When constructing an ephermeral table to use as the right-hand side of
an IN operator, also construct a Bloom filter to speed membership testing.

FossilOrigin-Name: baa83b460c677c210c7fa3f20314d7e05f305aed8a69026bc5fa106a3de4ea38
2024-07-03 20:19:33 +00:00
drh 5b070cfa0f Show the creation of IN-operator Bloom filters in the EXPLAIN QUERY PLAN output.
FossilOrigin-Name: c10a1b99d47a4c93bdb16e646f6a21add570d6dbb34fd91a32f8abacdb28712b
2024-07-03 20:10:42 +00:00
drh c4c2589ff0 Add a new sqlite3FaultSim() call to OP_NotFound to use for testing purposes.
FossilOrigin-Name: 84fd275bfd8afada50d3a5ac491d2b866671f8f0d2496bd240cf4bfdd74e7399
2024-07-03 18:56:56 +00:00
drh 6172e4355f Use a Bloom filter to improve performance of IN operators when the RHS of
the IN operator is a subquery.

FossilOrigin-Name: 1933496539c19cbf429a39d6b0b1c6b1b2af50733a3c4aea4920990ced652f6a
2024-07-03 17:51:48 +00:00
drh 3d24637325 Add assert() statements to FTS5 to hush-up warnings from scan-build.
FossilOrigin-Name: 77a76654e64c05f2c67be120f579fc60924ff51a9d0fa2cf9eb158f40171b5f5
2024-07-02 13:54:46 +00:00
drh 8960c02b91 Reword the deliberate_fall_through macro along the lines suggested by
[forum:/forumpost/7ec11023dd|forum post 7ec11023dd] so that it works better
with LLVM, while preserving compatibility with MSVC and older GCCs.

FossilOrigin-Name: fc248a4a0a232a95a79e24e57faedb5d824c3bf0aa62054b72339257dc9c18b3
2024-07-02 12:16:29 +00:00
drh 2143714d9f In the CLI, if the XDG_CONFIG_HOME environment variable is not set, then also
search in ~/.config/sqlite3/sqliterc for the initialization file.
See [forum:/forumpost/5cc6d059e9e092ed|forum thread 5cc6d059e9e092ed].

FossilOrigin-Name: 33841c9c3cb57beeb3884d4b0715d26199926b7d3e4d3dd1ab6f5603b5a62591
2024-07-02 11:30:10 +00:00
drh f1bf103e4e Fix a typo in a comment in a test case. No changes to code.
FossilOrigin-Name: f501166de4f967b7e2e895f180308630e3c88b12bc51823a0a124ee5089eae4f
2024-06-29 15:57:55 +00:00
drh 6adbdac676 Add a NEVER() around a branch in sqlite3ExprAffinity() that is designed to
prevent an infinite loop, but which can now never be reached due to recent
changes.

FossilOrigin-Name: da0b794852f8ccba5bd30395892044ec71b148075608406867785395e1f50b31
2024-06-29 12:22:32 +00:00
dan 75bdf00e89 Update fts5 comment/docs to include the order matches are visited in by the xPhraseFirst/xPhraseNext API.
FossilOrigin-Name: e41930ef5b883c0f2a6b7eb9606021d14a3df145bc94cb0d508af22b4db7f168
2024-06-28 18:21:30 +00:00
drh f13e2d22fe Additional steps to make sure calls to sqlite3ExprToRegister() are sane.
FossilOrigin-Name: 1c42d7176b228791f99bc9971d92a9ab67b1e2956ca4f8d934bd1a3e596d6c80
2024-06-28 18:07:10 +00:00
drh 076bd5758b Do not convert an expression node that is already TK_REGISTER into a
new TK_REGISTER.  This is a follow-up to [663f5dd32d9db832] that fixes a
bug discovered by dbsqlfuzz.

FossilOrigin-Name: fcb669cfaa8a59ca710504d5aaa1936f65a6da8ff13473ad84458f97a6fd1f49
2024-06-28 17:14:00 +00:00
dan 105c20648e Fix an incorrect tcl comment that appeared in many fts5 test files.
FossilOrigin-Name: d07085e2035b52a7edd27980523225e59c5bf851fb4a6de975f03e653b937c9c
2024-06-24 18:06:15 +00:00
dan fdda1cbe0c Fix some problems with calling fts5 api functions from within non-full-text queries.
FossilOrigin-Name: 56d265f956fe6433b625c6d732e55f387af3f643e705316f5a6f709d63731669
2024-06-24 16:08:01 +00:00
dan 1d8cde9d56 Enhance the output of ".scanstats vm" to include the absolute number of ticks for each VM instruction, instead of just the percentage of the total.
FossilOrigin-Name: 1535828335d2b0eccebae952ed2bc70cc0cd893bd4b079d26b80deff9b5f752b
2024-06-24 14:53:14 +00:00
drh 51b3b402c4 Revert Lemon so that it only shows -D options that are actually used. Though
the change to display the options in sorted order is retained.

FossilOrigin-Name: e54eb217c9508c19aee085b111a1323c9009f014ba4db6019918e27002c4ca8c
2024-06-19 18:54:04 +00:00
drh 8cf7bd5448 In lemon, show all the -D options in the generated header, even if none of them
are used.

FossilOrigin-Name: 2aa009c38bb207ac59b9bbd6f8e0d7315697b3fd6a01f9431f29a3c7ccad53e7
2024-06-19 18:46:44 +00:00
drh 199f091a95 Enhance lemon.c so that when it shows the compile-time options in the header
comment of the generated output file, it shows all options, even those not
used, and it shows them in sorted order.

FossilOrigin-Name: eed76e6698eabe47c6bf9696599ce1c2f7aa428cf60f39d0566fbd0d1f6c4c62
2024-06-19 16:38:17 +00:00
stephan d8295c8674 Remove some extraneous flags from the fiddle-debug build. Internal doc tweaks in the main wasm makefile.
FossilOrigin-Name: afae2c9e6fba0820c9be8d06a2e301a74167e4457bada45fe57b5fd9287d2e8b
2024-06-19 11:08:58 +00:00
stephan 68ba3980d5 Minor cleanups in the new fiddle build code.
FossilOrigin-Name: d356aebdd450bedc1ecbbdd266effe53c7badbd74ed7e6db8d0c31318d99a35f
2024-06-18 22:03:28 +00:00
stephan e042988dc0 Rework the fiddle build to build separate debug and non-debug bundles, with the latter in ./fiddle-debug.
FossilOrigin-Name: a9f1758f1cbe7b7d0c87dfb060f52d03c6ea0e066c079891a8ff354073997f9e
2024-06-18 21:50:30 +00:00
stephan 98526f906e Remove an extraneous CSS file and import its CSS into the one HTML file which uses it.
FossilOrigin-Name: 66085ca42388e8de741712150541a3e0afb06b3e23e738aa9ff687e9fc494c50
2024-06-18 20:47:05 +00:00
stephan bb6d32ae71 Add fiddle-debug make target to ext/wasm. It builds the same target files as a non-debug build, as opposed to a new copy, but adds various debug-related flags.
FossilOrigin-Name: 993143c5c39fb6ee15b0661cec8fed436ff04543b528a3796534f5e3a44799f5
2024-06-18 20:40:11 +00:00
drh c98d42e802 Add instructions for how to do debugging builds of the CLI on Windows.
FossilOrigin-Name: abe81c18614b41c78cf3edeaaf8d85d5d2dbdcf331b580f1c98f80d3726d8189
2024-06-18 19:36:04 +00:00
drh 99623a83aa Update sqlite3_analyzer to report on the number of WITHOUT ROWID tables and
to provide separate aggregate statistics for WITHOUT and WITH ROWID tables.

FossilOrigin-Name: 4152894433c3a292d983b1c196664a0c46e406eccd7e644a64fa443fdb204825
2024-06-18 12:19:13 +00:00
stephan e9540e2881 Internal doc typo fixes, as reported in the forum. No code changes.
FossilOrigin-Name: ea16b286b4344c1833ca3d17b6d315fb4ddcb99bcc232ff6444debeb634881f2
2024-06-18 09:58:39 +00:00
stephan c7cbcfbbbd Remove some dead JS code.
FossilOrigin-Name: 754e6e059b740435475c1869feeaca39585b262d5c27210e0fbddbbcc202c4d5
2024-06-16 17:06:50 +00:00
drh 9175b18226 Do not omit the ORDER BY clause from a recursive CTE just because the
query that contains the CTE also contains an ORDER BY clause.  Plus
other changes imported from the recursive-cte-orderby-fix branch.

FossilOrigin-Name: 13242289c5d412b706f50fc7e1553032ea3a52d41a3e34e155432adaf0551481
2024-06-14 23:13:54 +00:00
stephan 709cb313cf Remove some dead JS code. Improve the exception-to-C-result-code conversion to account for the case of a file disappearing while the OPFS VFS is waiting to acquire a lock on it.
FossilOrigin-Name: 5bd9fd5f61e7cd1ed3b44c5cabc759c154c98eb40ad10d29e61f142aaa062ddf
2024-06-12 22:15:25 +00:00
stephan 063a2b338c Remove some JS docs which no longer apply. No code changes.
FossilOrigin-Name: f253cab3359270045a3ae9f5e9eadc1cfc8844181db127165bfdf49d5f99efb1
2024-06-12 21:01:44 +00:00
dan 718c5ff551 Fix a problem with rolling back hot journals using the unix-dotfile VFS.
FossilOrigin-Name: a44703135e8dd3cc67e548e1aa3c42e71df127fb7b593ccfc06025ec42a501fd
2024-06-12 15:50:35 +00:00
dan 9e72ea5ddc Omit tests that depend on sqlite3_release_memory() in lock5.test when running the memsubsys1 or memsubsys2 permutations.
FossilOrigin-Name: 9d63033bf46ddf0f4d9d4ef06d56357fe6165f2370cd19cf81c2f7f61ce224fc
2024-06-12 15:29:37 +00:00
stephan fd70ca40cc OPFS VFS: change the xCheckReservedLock() impl to always return false, as discussed in [forum:a2f573b00cda1372|forum thread a2f573b00cda1372]. This does not impact any current tests, and may have no direct impact at all because of how that VFS handles locking, but is hypothetically a more correct solution than the previous one.
FossilOrigin-Name: c298b8ba2dcd01fa28b79a78bb4986fa0282755a0a36b7f38b93096ac31f521e
2024-06-12 12:36:02 +00:00
stephan 01f07a61e4 Doc updates in JS code. No functional changes.
FossilOrigin-Name: 587ed3a5d283898ad0e67ccee86a0a4ccc432fa292c0a3534e9e6ec70a7b7780
2024-06-12 12:17:03 +00:00
dan f8ed7db472 Do not attempt to run hot journal rollback tests in lock5.test with the "inmemory_journal" permutation, which cannot generate hot journals.
FossilOrigin-Name: 1e6fa95b88cace9e6cac476863d57cfd8de483f83c922ff964b77cbaee6e9ae4
2024-06-12 11:41:18 +00:00
stephan 6975fc56f7 Slight API doc tweak for xCheckReservedLock(), based on forum feedback. No code changes.
FossilOrigin-Name: 2af7a96f6868a4529d689702926b350a4c00082f14da0ded5a8eaea295682d17
2024-06-12 11:39:29 +00:00
dan 41caf1cdfa Handle the case where unix-dotfile is used with URI parameter nolock=1.
FossilOrigin-Name: 3925a5b904e159d54455cfc73fe837a9c6ea3a6d60da63afde3242b4d6f67c90
2024-06-11 20:28:56 +00:00
dan f6ec2b5946 Fix a problem with rolling back hot journals using the unix-dotfile VFS.
FossilOrigin-Name: 4ae3300b79e03381fd7f1033bb7978bb6367369790f17c3bdacac51e205edaf9
2024-06-11 20:03:32 +00:00
stephan e15d5de0a6 Remove some dead JS code and update some JS docs.
FossilOrigin-Name: 6935ac71bad3d36cc519f0325ae4447a674f257309d020cdc0741160fcce0580
2024-06-11 17:04:43 +00:00
stephan cf2710e0b6 When compiling shell.c in SQLITE_SHELL_FIDDLE mode, ensure that the shell_main_exit goto label is reachable.
FossilOrigin-Name: 06e6f64533563ab9c059b773e5f0d78085df716f2624e547b7259f6789c3ffe0
2024-06-11 17:04:02 +00:00
drh 941602ae4f Change constant expressions to pre-computed constants, because apparently
MSVC on ARM requires that.
[forum:/forumpost/4feb1685cced0a8e|Forum thread 4feb1685cced0a8e].

FossilOrigin-Name: 6c103aee6f146869a3e0c48694592f2e4c6b57ecdb4450f46e762c38b4e686f1
2024-06-11 14:36:56 +00:00
drh 34e4c6f732 Improved header comment on the sqlite3FpDecode() implementation.
For the fpdecode() SQL function (available in debug builds only) limit
the value of the third parameter (mxRound) to be positive.

FossilOrigin-Name: 56af06fa12104a1fe119d7087746011183af053834eac72d0fb69f60d98054c6
2024-06-10 12:43:03 +00:00
drh 2fbb3fb532 Disable the omit-noop-join optimization when there are 64 or more terms in
the ORDER BY clause.

FossilOrigin-Name: 40de3939792e17df25598b3e60d1cebcecde2b00832acd55604f14b21398a9a7
2024-06-09 17:34:03 +00:00
drh df9471b1b5 Allow the query planner access to the argument of LIMIT even if that
argument is a bound parameter.

FossilOrigin-Name: c4a9dda2809c6e0e3d928e11e5553ead82cd9df551bcd35b11a7d869ef80ab8e
2024-06-06 23:56:36 +00:00
drh bd1402487b Omit bogus tests accidentally added by the privious commit.
FossilOrigin-Name: e94dfe9928750dd98145d4d9920b298f7b0868703b487f86e0db77a41d53ccf9
2024-06-06 23:22:32 +00:00
drh 4703b7d092 Better optimize queries that use parameters in the LIMIT clause.
FossilOrigin-Name: e58cb304d1e0ec6e30260a165aaccc2cf096ce6c999efb06683c4ef600ee12ef
2024-06-06 15:03:16 +00:00
stephan 404ddadbd4 Re-enable [dcc2bb2c562e97e090174], as [296eeb26c816bc73] corrects the wasi-sdk build problem the former check-in triggered. Reported in [forum:143e40d7f4e79c66|forum post 143e40d7f4e79c66].
FossilOrigin-Name: f66608bd356efe492d1003663c2e1ccd7cfbf2d40393d256f8720149904ad2d5
2024-06-06 12:22:19 +00:00
drh fd6beda14b Adjust the parser so that the value of TK_ISNOT is similar to the value of
TK_IS.  This helps the compiler generate faster switch() statements on the
Expr.op fields when there are cases for TK_ISNOT and other common operators.

FossilOrigin-Name: 34f05c3d89b2dd15e4b0d1ba292df7de3dfc54b505c0ba145cc3db52cf020845
2024-06-06 01:21:57 +00:00
drh 48f5e52728 Small performance optimization in the operatorMask routine of the WHERE
clause analysis logic.

FossilOrigin-Name: 9d69fc1c87ae673356869ecd89eb19734fd126702c0f9fe595336ecd7be89e08
2024-06-06 01:00:50 +00:00
drh b458cbb601 Small performance optimization in the allowedOp() routine of the WHERE
clause analysis code.

FossilOrigin-Name: 4ba8be544711e07748e8dd3ca6b81f9897906061c0a1a1bb4fb3808dc27f734b
2024-06-06 00:49:36 +00:00
mistachkin 6593b340ff Fix harmless compilation issues seen with MSVC.
FossilOrigin-Name: 816d4749384c7f398912c905a16c83b88f4c55632050b4c6117c61301d1a53e1
2024-06-05 20:50:39 +00:00
drh 0de7da1998 Small performance optimization in the codeEqualityTerm() routine of the
code generator.

FossilOrigin-Name: 8080c6eafd1280ea870a6ab1ba715ac5af67387e69771be6cbd46dda77c3eaa8
2024-06-05 20:41:36 +00:00
dan 6d98c55b56 Fix an fts5 problem with secure-delete mode causing integrity-check to erroneously report a corrupt index.
FossilOrigin-Name: 80bef4d60ba9e3679ea66655ca36fcfaa888775a3d1598d50e9649ad84a95b63
2024-06-05 14:47:54 +00:00
dan e3eefe00dd Fix a very obscure issue where the name resolver could get confused if aliases like "sqlite_schema" or "sqlite_master" were used in a query involving the sqlite_temp_schema table.
FossilOrigin-Name: a096eb7554952f8137c6e9330c328164719fb27e958787fbd503bcd1364e6ae4
2024-06-05 11:36:58 +00:00
mistachkin ecaa021759 Fix harmless compiler warnings seen with MSVC.
FossilOrigin-Name: 93ac8573eead9b785a24715239e71cd3ef730bf16332cf3b5e264a6491374cac
2024-06-04 19:21:16 +00:00
dan 7d24afa0de Better handle WHERE terms that are common to two or more OR branches when planning virtual table queries.
FossilOrigin-Name: 1976c3f7e1fe77cf3367710e8ada230a3672ed374e316425164e42b2622526c7
2024-06-04 17:26:15 +00:00
dan 077f9b14f1 Fix a couple of memory leaks in the shell tool code that could occur when processing errors.
FossilOrigin-Name: e84f09d469ee76a5b5e44baf6a69b90c69d4160fa4c32de04a96f868643acd96
2024-06-04 15:07:38 +00:00
dan c5a0ed7b43 Allow shell command "testctrl pending_byte" to be used in unsafe-testing mode only.
FossilOrigin-Name: afa45c4f5afc9248ca4a1e775404a460280bb9a58a92eae508ae00fb2f675dc6
2024-06-04 14:32:31 +00:00
drh 8741ca0819 Address a corner-case associated with the call-function-once optimization
of check-in [663f5dd32d9db832] that was causing a (harmless) use of an
uninitialized bytecode register.
dbsqlfuzz 8a44f675401a8b1f68a43bf813c4f4f72ad8f0ea

FossilOrigin-Name: b37ac2d020873b78efa164036db4056406a67330679f850775da520cd8027cb9
2024-06-03 18:04:10 +00:00
drh 586b2b2a54 Omit an unused #define.
FossilOrigin-Name: 9c2de28a146e4cdfa4c05bf8dbd4ebb1156c71104001edf68e6dd5db21deb91f
2024-06-03 12:36:43 +00:00
stephan e1b55a5af9 Modify three #if checks in os_unix.c to improve handling of SQLITE_OMIT_WAL on WASI builds. Based on the discussion in/around [forum:57e918431735128a|forum post 57e918431735128a].
FossilOrigin-Name: 296eeb26c816bc734530cf446922f25be970b901c884df1a98083502f0d1e9f5
2024-06-03 07:22:28 +00:00
drh 0507e6e388 Be more aggressive about freeing memory associated with the
sqlite3_index_info.idxStr field.

FossilOrigin-Name: 85dcd0a8479a658203833cfd75f22813faa26d4793ebfbb8843035d683bee105
2024-06-02 10:52:35 +00:00
dan 9c3eef8c0d Have fts5.xBestIndex return SQLITE_CONSTRAINT, instead of a large cost, when no usable plan can be found.
FossilOrigin-Name: 7c470945372dc98610f7c9840ce8cab18c19b655352f0187e4f31040cea77363
2024-06-01 17:56:58 +00:00
dan af35432400 Further tests for OR constraints on virtual tables.
FossilOrigin-Name: ff4a30056f8dbcbd045afdbee485f6671c3580b95187144aa9a77c97dfda2647
2024-05-31 20:32:10 +00:00
dan cadfdd4e09 Better handle WHERE terms that are common to two or more OR branches when planning virtual table queries.
FossilOrigin-Name: 4edd9b29f58621335b8a562280c991c34804bbba090f90c951261d043cff1965
2024-05-31 19:26:22 +00:00
drh 5ef9c89977 Do not even try star-schema detection if the join has fewer than 5 tables,
since 5 is the minimum for a positive detection.

FossilOrigin-Name: a07ec16bbc056fbc23a7bd58e5e32ef691c13e9babeb542918cf9a01cac40c20
2024-05-31 15:39:00 +00:00
drh d72ca8f810 Document the OP_Explain opcode. Add the WhereLoop.rRun value as P3 in
OP_Explain opcodes associated with WhereLoops, for testing purposes.

FossilOrigin-Name: 996c46e61d9a53a54018672dd407b8ba8c480dd6795393428f9d5fcb81b47ab5
2024-05-31 14:39:42 +00:00
stephan 2f755cdc3a Back out [dcc2bb2c562e97e090174], as it breaks the --with-wasi-sdk builds.
FossilOrigin-Name: 62199562c26f77d059aa0bbda42a421a7aece828aa94d14a40c701a6384d1cf6
2024-05-30 21:54:16 +00:00
stephan 4c86b2db0f Remove the unconditional SQLITE_OMIT_WAL when building in wasi-sdk mode, per [forum:80003e91a7a6cb4d|requests in the forum].
FossilOrigin-Name: dcc2bb2c562e97e090174d4d0970bfa1551e5eb4db022e6d232c4dd786818e45
2024-05-30 17:56:26 +00:00
drh 1344b540d9 Ensure that the second pass call to wherePathSolver() always reports a positive
value for the estimated number of output rows.  This fixes a long-standing
(though exceedingly minor) problem that was only revealed by a new assert()
added yesterday.  dbsqlfuzz 545d6debc2da496ab2b915e33ea34d9a35a935df

FossilOrigin-Name: a47c644fef71f3ab3dc584ea917eaab9a8e5b4c9dcb57bdd29747ba32108e85f
2024-05-30 13:27:29 +00:00
stephan 5b7048d5af Update shell.c to use the msteve fork of linenoise, per /chat discussion.
FossilOrigin-Name: ee92fa8366b743266b17c692499087c0d11b9302d096c3dfb4e6356b467e939e
2024-05-30 11:14:16 +00:00
drh 7da33383c7 Remove an unused parameter from fts5ConfigParseSpecial(). Compiler-warning
fix only - no functional changes.

FossilOrigin-Name: c08dd245f7706f2fd2269d700be480477619a722e27e6b439462ae543302c49f
2024-05-29 15:16:17 +00:00
drh c327bd0354 Use a heuristic of artifically lowering the cost of fact tables in a
star-schema query in order to prevent plans where the fact tables are
in outer loops from being trimmed by the path limiter.  This helps to
generate better (faster) query plans in those particular cases.

FossilOrigin-Name: 38db9b5c83dfb3021333365b2f6b11e3b54afd7033e9f4dadf0c0df1d4b082d8
2024-05-29 15:08:20 +00:00
drh 1085eee098 Change an assert() (incorrectly) added by the previous check-in into a
testcase().

FossilOrigin-Name: 5e64b541d175c10a5df67ef3419ecd74fcb51392af5f5458973cea85ee6a3246
2024-05-29 14:32:22 +00:00
drh eccb42eb0e Small performance enhancement in wherePathSolver()
FossilOrigin-Name: bdab69641df4704f7b080cf7841bbf39da9160222f7b664a79ed662b70ca7b0a
2024-05-29 14:04:52 +00:00
drh bb24c75b41 Improvements to comments and debugging output.
FossilOrigin-Name: 85164ee155e42809fe34e6c6b6fbe0a2baa9d616326a811173a0b0c8a885fcce
2024-05-29 13:29:39 +00:00
drh 54bfc9435f Remember the amount of the heuristic cost adjustment associated with
star schemas and compensate when computing whether or not to use Bloom
filters.

FossilOrigin-Name: 21daf2463ef70e6b5dc73ba5bf62b3d2fb504e9189f645ac74b513d3d8b979c2
2024-05-29 10:40:53 +00:00
drh d79ce8a459 Fix typos in the star-schema test cases.
FossilOrigin-Name: 4080937353985eb391f70a3a7fae890823cf01d19c52939e343fb2504f3f8659
2024-05-29 09:53:19 +00:00
larrybr 7c6256234c [forum:/forumpost/c9b8d923d5|Fix sqldiff out-of-bounds char classification error mentioned in the forum.]
FossilOrigin-Name: b31933d87301bd93763f0923e0ce06a183700875e6107c9f185faffbd99797d9
2024-05-29 03:35:40 +00:00
drh c34ff12c59 Use a heuristic in the query planner to help it better cope with large
star schema queries.

FossilOrigin-Name: a98be0f548f277fab8f38a2dec6ddcbe7a7fff27856ba19e76ad8c5641894b7b
2024-05-29 00:38:12 +00:00
drh d60a503b4c Fix a couple of assert() statements so that they cannot fire
when the SQLITE_TESTCTRL_INTERNAL_FUNCTIONS debugging capability
is activated.  dbsqlfuzz f5b347cf167a62fbe08062b2feee65cb9306e363.

FossilOrigin-Name: 273504b74cb9c3cfa2497a1339e706a1f2d2c3ce81b23a16beb47da9292535e0
2024-05-28 19:08:18 +00:00
drh 05665e9de9 Tweaks to the query planner so that it is better able to deal with star
queries with many dimensions and bad indexes.

FossilOrigin-Name: 1e49081001f93b75ee499536f8a61a0f49225a1745117bb9267249c38c4bf803
2024-05-28 18:53:36 +00:00
dan c5f1828df3 Fix another problem with the sqlite3_log() message identifying the table or index that a conflicting page belongs to.
FossilOrigin-Name: 19d5fd8a483d23262c98524386d39bb7f4fb9ed79343c574abe51b57eb8296ec
2024-05-28 18:41:32 +00:00
drh efe474af97 Increase the number of parallel paths in the query solver from 12 to 20.
In the .wheretrace output, sort the parallel paths in order of increasing
cost.

FossilOrigin-Name: 8ba2c2f5cb31f7bcc426bec78457316ef11d0b5debf24e8da8c6fc2f95878b1e
2024-05-28 12:41:11 +00:00
drh 6fa46d0e06 Documentation typo fix. [forum:/forumpost/772d6688f5|Forum post 772d6688f5].
FossilOrigin-Name: 1f2f9c709eaee3c45664afa062f0fb2b912de446581066c87ea144b8ba55b55c
2024-05-28 11:24:18 +00:00
dan 2386943f19 Fix a problem with the error message reported by fts5 in some OOM cases.
FossilOrigin-Name: c7aaa28f786567e29ebee3848283b5c740dd3b8144dc14372e56260ae1b76922
2024-05-28 10:57:24 +00:00
drh 18970dac18 Improve the automatic-index decision such that no attempt is made to create
an automatic index on a column that is known to be non-selective because if
its use in other indexes that have been analyzed.  See
[src:/tktview/8ff324e120|ticket 8ff324e120] and
[forum:/forumpost/b21c2101a559be0a|forum post b21c2101a559be0a].

FossilOrigin-Name: 2a7603c327d11d5e6bc4d40c29151fea544d650f51200ff76efb921adb0c38df
2024-05-28 00:16:52 +00:00
drh 38af125238 Query planner tuning: Increase the maximum number of simultaneous solutions
to track int eh solver from 10 to 12.

FossilOrigin-Name: fe2e1dadbacbe6392ceed44fd287a2ed82189cb8055f35631d37967d9a7a5d1d
2024-05-28 00:01:07 +00:00
drh b23f92b286 Avoid creating an automatic index on a column that is known to be not very
selective.

FossilOrigin-Name: 70409763f70faa2a0f4f072fd318a687b109a0905cc57906ad7f80d2885fe6d9
2024-05-27 21:55:01 +00:00
drh eeb8506c18 The previous check-in was a little two aggressive about skipping the
initialization of entries in the CellArray.ixNx array.  Fixed here.

FossilOrigin-Name: 1b6d1fbcdecf14bc79dc7b5df50e585d2fcc1484a9987b863c190e8f7ba2c39f
2024-05-27 19:52:24 +00:00
drh d081df0a5d Fix typo in comment used to generate parts of the documentation.
[forum:/forumpost/da5a2ca4db|Forum post da5a2ca4db].

FossilOrigin-Name: 428aadf0c71473ccb0681b2b7146281c62949ee99556873e49895d44eec7aee5
2024-05-27 18:15:08 +00:00
drh 09d8e696dc Very small performance increase and size reduction by removing unnecessary
ALWAYS() macros from the btree balance logic and replacing them with
special initialization of CellArray and some assert()s.

FossilOrigin-Name: 32b79041d9b6858e4ffade5841898619c7d25d20c4638bf85447a2ca9cc4a3d3
2024-05-27 15:13:49 +00:00
stephan 20c118faec Doc typo fix. No code changes.
FossilOrigin-Name: eaf07fe69f383938dd394f58cedad3f16f146d9b48e66bc1ba17446cf75ffb3f
2024-05-27 13:41:46 +00:00
drh fefe24ddf8 For compatibility with PostgreSQL, when right-hand side of the ->> operator
is negative, it should index from the right side of the JSON array on the
left-hand side.

FossilOrigin-Name: 82365a45b96536c1146a384e5d3efce80a6ec469a54713c7f40bf15eb834b5fd
2024-05-27 13:24:39 +00:00
dan 09f87094bd Fix a possible buffer overwrite in the ".import" command. [forum:/forumpost/0c447f0548|forum post 0c447f0548].
FossilOrigin-Name: 0fd958fa9b56a8ef254127e29800ca2a267590e86edf739bd339239b25a5da6e
2024-05-27 11:35:05 +00:00
drh ee8f926027 Add new assert() statements to help out a static analyzer. Response to
[forum:/forumpost/17fe8ac32e0de4f5|forum post 17fe8ac32e0de4f5].

FossilOrigin-Name: 857f6d530949221d154b5120ecc2aa906418bec6f69d1c13197a432ba3cad8eb
2024-05-27 11:31:02 +00:00
drh 4ed784d1e8 Ensure that sqlite3ViewGetColumnNames() returns non-zero on any error.
dbsqlfuzz d9de2a508f693e5a228d7a01d5341e3c64326fbb.

FossilOrigin-Name: 57aeb3a287fc190bf8d438a7b03d6715c05fd3fd71559c6a14d7bd910d37b38d
2024-05-25 23:13:15 +00:00
drh a7f3b0f85b Ensure that queries like "SELECT func(a) FROM tab GROUP BY 1" only call the
func() function once per row.  Fix for the issue described in
[forum:/forumpost/bf8f43aa522c2299|forum thread bf8f43aa522c2299].

FossilOrigin-Name: 663f5dd32d9db83249fcf85602a756464ece74e1899dc881ac9519d872638441
2024-05-24 23:44:48 +00:00
dan c598fdd10c Fix another fts5 problem related to unknown tokenizer modules.
FossilOrigin-Name: e78b7783040ef1ad34d4287af8a7a5998d360d50076e0de42a116d9a96fe74b4
2024-05-24 21:38:37 +00:00
dan 0e7a7e57cc Fix a crash in fts5 that could occur within "PRAGMA integrity_check" if the specified tokenizer was unknown.
FossilOrigin-Name: 061d7b69e6a43474cd7123a69f4ee65b5649a0b5e51474343defef5ae5262657
2024-05-24 21:23:36 +00:00
drh 4aaf45e246 When omitting result columns for the vector-IN optimization, make sure that
any result-set column references in both the ORDER BY clause and the
GROUP BY clause are updated appropriately.

FossilOrigin-Name: f890812df0a57fcd201647640d8b9ee26047cd93cd7171bc6fd050f574649345
2024-05-24 20:18:16 +00:00
dan 89b04f3786 Ensure that queries like "SELECT indeterministic(a) FROM tbl GROUP BY 1" invoke the indeterministic function only once for each row of tbl.
FossilOrigin-Name: 4555d66547e28cb110e1012b145bcf3aafb5d4bde05e9d27bcb4ca33837b28f5
2024-05-24 18:31:39 +00:00
drh 31079226f7 Add the stmtrand() extension function for use in testing.
FossilOrigin-Name: 5c97a5b9d163b1c427e002f3734687ca0384bc0da6a90fc4bfd358c654d3a7b3
2024-05-24 14:16:06 +00:00
dan 7c492a250b Fix an "ifcapable" expression in values.test.
FossilOrigin-Name: b8442d2a6012b1f2e15381613267db0982e36bb4c748b15b56e668e0d0a3d0d2
2024-05-24 13:33:39 +00:00
drh e6d3c57204 Fix the window-function group_concat() so that it returns an empty string
if it has one or more empty string inputs.  This fixes
a bug introduced by [c6da39115d3e2b0f] on 2019-03-26 (version 3.28.0) and
reported by [forum/forumpost/bf8f43aa522c2299|forum post bf8f43aa522c2299].

FossilOrigin-Name: cec6bb3fc9932ea78ec8e63d9c2d4e56a4d94b8973b9ea46033cc4baa87c0476
2024-05-23 23:26:04 +00:00
drh 294cd87afc Merge various enhancements that were being held on a branch until after the
3.46.0 release.  (1) Allow arbitrary expressions in the second argument to
the RAISE() function. (2) Performance optimization to the OP_OpenRead and
OP_OpenWrite bytecode operators. (3) Allow FTS5 tables to be dropped even if
the associated tokenizer is not available.  (4) Performance optimizations in
FTS5.  (5) Generate better bytecode to improve performance of ONEROW scans.

FossilOrigin-Name: 1e5385ffc30743412ade1536583504eec36790cac5355042aa1e12d8d2271d1f
2024-05-23 15:05:57 +00:00
drh 18ec0d4a34 Increase the version number to 3.47.0 to begin the next development cycle.
FossilOrigin-Name: 20e228a22e4c5a8d62fae003ac176de62126d8e121871c5ca14a5480c4d4b6a2
2024-05-23 14:58:31 +00:00
drh f5266055a3 Version 3.46.0 for the bedrock branch
FossilOrigin-Name: 483d112ab4690853dd2ab4406a865e9e64a13bafb4d623ea57c04cc3980651fb
2024-05-23 14:09:13 +00:00
drh 0d8cea5109 Version 3.46.0 for the wal2 branch.
FossilOrigin-Name: fdc0e1480a941d494e556331544cb40b2a997e8709e2c7fab1bed572007eb88d
2024-05-23 14:05:34 +00:00
drh bebe2d8be8 Version 3.46.0
FossilOrigin-Name: 96c92aba00c8375bc32fafcdf12429c58bd8aabfcadab6683e35bbb9cdebf19e
2024-05-23 13:25:27 +00:00
dan 3731703dbb Ensure an sqlite_dbdata cursor is properly reset before being used again, even if it has already encountered database corruption.
FossilOrigin-Name: 3210e1ca4d0efedf9710c97abd050ba10d3af98cb1f029c26daa84daf42fbc7e
2024-05-21 17:37:50 +00:00
drh 82e5a2a123 Merge the latest trunk enhancements into the bedrock branch.
FossilOrigin-Name: c65a82f1faa6cdfc710c01869943b12033a43868e8343468e326816e5a2fd5ad
2024-05-21 15:24:33 +00:00
drh 68796a609b Merge the latest trunk enhancements into the wal2 branch.
FossilOrigin-Name: d4217d9f70118a47c0052be940955cf85356b9a08166cef9712668175baea999
2024-05-21 15:20:13 +00:00
dan 2526668556 Do not attempt to run mmapcorrupt.test with builds that do not support mmap().
FossilOrigin-Name: 54a49f1847f95c377f0f611eb10f635b879bba8c2eb25e65e2f66a56f4f9a505
2024-05-21 14:54:46 +00:00
drh 33087dbf30 Fix harmless compiler warnings.
FossilOrigin-Name: aab59f9f7c1c4cfac4d29d7c7a381583295312baf3b9a75e30a617b1541e12e1
2024-05-21 13:34:26 +00:00
drh b8337932b3 Fix the -> and ->> operators so that when the RHS is a string that looks like
a number, it is still treated as a string, because that is what PG does.
[forum:/forumpost/9e52cdfe15c3926e|Forum post 9e52cdfe15c3926e].

FossilOrigin-Name: de8182cf1773ac0d04268d896a613841cf6bf61f9f030342170657d5e06f2acb
2024-05-21 11:11:29 +00:00
drh 00208479fd Omit a capi3 test from 2007 that depends upon undefined behavior (UAF).
FossilOrigin-Name: a60a0b75137c015e97b8dcd0070444fcb608f502ce88b2ee744dd6230dbdc120
2024-05-20 19:59:32 +00:00
dan f765aec406 Do not run the tests in recovercorrupt4.test with the inmemory_journal permutation.
FossilOrigin-Name: c13205d66b8406cccfc937c88ac515fa758fe2ade8b3d30f0c1e79fe974e9600
2024-05-20 19:51:06 +00:00
drh e212493358 Improvements to the documentation of sqlite3_vtab_distinct(). Do not allow
sqlite3_vtab_distinct() to return 2 or 3 if the rowid column of the virtual
table is used, as doing so might lead to an incorrect answer.

FossilOrigin-Name: 08058d66d1bde4fcf8324482ee4c6c030c681383470d5076b6f75b74aac2ae29
2024-05-20 17:52:35 +00:00
drh df903fe056 Improvements to the sqlite3_vtab_distinct() documentation.
FossilOrigin-Name: 6ee041d34f292b94701919f51bbb9e12bcb9e0c4f45e4c0b83f30ff328070637
2024-05-20 16:00:27 +00:00
drh 0462499051 Fix the definition of sqlite3_vtab_distinct() such that return codes 2 and
3 mean that all rows must be distinct over "colUsed" which is a superset of
"aOrderBy".  Also, disallow return codes 2 and 3 if the rowid of the virtual
table is accessed.

FossilOrigin-Name: 922731ce98c0ce7837784ff7966049e59fa73da2aa04abf3506503b6fc4aa048
2024-05-18 20:00:08 +00:00
stephan 79dac84b31 JS error message and doc typos reported in the forum. No code changes.
FossilOrigin-Name: 1a073f9acfb691eebf4a8cc78a72ff47ebbb6aba4acede6755fa3faefae48f2b
2024-05-18 15:21:45 +00:00
drh b834f629cc Improvements to a VDBE comment. No changes to anything that affects queries.
FossilOrigin-Name: c2188da860a1debd7c5d28b223deeef15035c33c2c5aefae9d9b620b6a2b4677
2024-05-17 22:51:03 +00:00
dan c83386cbe5 Merge branches "fix-onerow-opt", "faster-openread", "fts5-delay-tokenizer" and "enhanced-raise", each containing minor enhancements prepared for 3.47, into this branch.
FossilOrigin-Name: 6dc6472175bccbed15ebf6811c209d1a0b5fad60158fb32040210f2cdae916a6
2024-05-17 14:26:32 +00:00
dan 0445cd1020 Fix a problem in where.c preventing SQLite from identifying some ONEROW scans, leading to (slightly) sub-optimal byte-code.
FossilOrigin-Name: b41e71208abcdf988f57c3fd7e6a372bbaf442c32a8e2c051beb8d45c8001e0f
2024-05-17 13:56:38 +00:00
dan fa0763eb34 Add extra tests for the error messages generated by sqlite3_declare_vtab() when passed something other than a well-formed CREATE TABLE statement.
FossilOrigin-Name: 5fa0c2a026f0467af3a76ea80cee8168b7a49553c350f642d1dff02ff9e7aeed
2024-05-16 16:22:51 +00:00
drh c72953a4bc The sqlite3_declare_vtab() interface should return SQLITE_ERROR, not
SQLITE_MISUSE when a nonsense SQL input is provided, for legacy compatibility.
[forum:/forumpost/4b875d7b2a|Forum post 4b875d7b2a].

FossilOrigin-Name: 1ec2961a0341619ae88832942b52606b3246fc0de07ca6da0300e4bfc656ecf0
2024-05-16 15:57:42 +00:00
drh c4439fb134 Fix a case where an error in the ORDER BY clause of an aggregate function that is part of a trigger body might cause an assert() to fail in an ALTER TABLE command. dbsqlfuzz 899d9255555a07f1fe11a0bb51a2fe197b3150cd
FossilOrigin-Name: 953271f29e7bd6ba9b85f73250b1c22b08eef317f9ac3c7c3d92cbaa57d7efd5
2024-05-16 00:20:43 +00:00
drh 61f2ee6ab8 Back out the assert() fixes from [a9443dbfbe25e588] and
[3afaeac56dff58db] (but keep the corresponding test cases) because they
are both made obsolete by the previous fix on this branch.

FossilOrigin-Name: 3120c7c8a3565b8a3ca321129c09db96839c2f9af1777fc7a1668fef81973603
2024-05-15 23:36:24 +00:00
dan 1550771a6a Fix a case where an error in the ORDER BY clause of an aggregate function that is part of a trigger body might cause an assert() to fail in an ALTER TABLE command.
FossilOrigin-Name: b79a8be37b44c1eff7d00f7af7cd9d0e13614ce5961bb88ddb9ec63f6a846c78
2024-05-15 21:38:04 +00:00
dan 9a1a4822e8 If there is no data available for a field in a corrupted record, but the type is available, have the recovery extension substitute 0, 0.0, '' or X'' instead of a NULL value.
FossilOrigin-Name: 8eb326c24981647e2df90e4b9d75db52a552bd997bd821f5898c7ff3fb93a57d
2024-05-15 15:06:27 +00:00
dan a0693f21fc Optimize queries against fts5vocab(row) tables that do use the "cnt" column.
FossilOrigin-Name: 96a591c202cb4218999ece3de063bf466b6fd81bd23383893f3913ecb52dda36
2024-05-14 19:41:19 +00:00
dan 433af7ab93 Avoid calculating the value of the "cnt" column for fts5vocab(row) queries that do not use it.
FossilOrigin-Name: c0c91e306c252ce1b398e92ab958e2a3de0fc43db05cfdffc548380654fc9f13
2024-05-14 19:09:54 +00:00
dan 32ca0dbcdf Have fts5 tables delay initializing the tokenizer until it is first used in all cases where the tokenizer is not "trigram".
FossilOrigin-Name: ca4fdcb8ae95d2a61236b949f852d2bf25ea2dbbff7eedafbd8eb84e8fd96687
2024-05-14 17:16:09 +00:00
dan 12b205c637 Allow existing fts5 tables to be dropped even if the associated tokenizer is not available.
FossilOrigin-Name: 69ef47eeee8b53684c321393be34f03600694fbc86377f8720ff80307846aff6
2024-05-13 20:06:08 +00:00
drh 4f59d09648 Optimization in the implementation of OP_OpenRead and OP_OpenWrite
FossilOrigin-Name: 3a256a1667b15702dc5048fdf2fdba9c1e551106ce8cf391360f156fab35ecc5
2024-05-12 00:26:08 +00:00
dan f722d91fa8 Add tests to bestindexC.test. No changes to code.
FossilOrigin-Name: 58d938c0e03c3c8d8796c537f89e69734ba6263d60ba37e345259cb6fdffbea5
2024-05-11 16:44:56 +00:00
drh 5d783d5c84 Fix aggregate function processing to correctly deal with OOMs inside of
sqlite3ParserAddCleanup(). 
(dbsqlfuzz b2d11ca70e55ee8bde48ae0b53fa3e9355812f95).
Also add improved testing support by causing sqlite3FaultSim(300) to
simulate an OOM inside of sqlite3ParserAddCleanup() and by adding
improved fault-sim support to the CLI.

FossilOrigin-Name: c6fd70b3c23fa00eaac9286d4a67e5c8ac76f926c11c220250c34032647bedc1
2024-05-10 18:24:15 +00:00
dan e09e451a48 Add a test for the fixes on this branch.
FossilOrigin-Name: 716088d673b80446ea5f9cb83f6aad8a9ea2ec769d4894e02ab916930e4cf169
2024-05-10 18:15:03 +00:00
drh 325420616f The sqlite3FaultSim(300) error from [1e8863909ac369e5] must be treated as
an OOM, since it simulates an OOM.  Also fix deferred deletion of Expr objects
from [a53bdd311c4154fd] so that it does not corrupt the AST if an OOM occurs.

FossilOrigin-Name: c623d9a51e0ae6b4a8e1186714d14e7aa3df55246e123323481f07f07749dea5
2024-05-10 18:10:34 +00:00
drh 0e4afddadf Improvements to the ".testctrl fault_install" command in the CLI.
FossilOrigin-Name: 48b34ded82d0cec8a7dd67431f057b320b8c459b1b78c0fbf79bc2b355e85d4e
2024-05-10 17:05:24 +00:00
drh 7e60106d15 Add an sqlite3FaultSim(300) call to the sqlite3ParserAddCleanup() routine
for more precise simulation of OOM errors in that routine.

FossilOrigin-Name: 1e8863909ac369e5e62e82e53b6e5a7cc6794100cef281ede00b0bb9bbc28594
2024-05-10 16:28:35 +00:00
stephan 8292aa7a18 Squelch two OMIT-flag-related warnings reported in [forum:388243d9882067a9|forum post 388243d9882067a9]. No functional changes.
FossilOrigin-Name: 1d6716054d7fc50df237996c3db30e5fb8e32acbf48cb8b9af472360515945c4
2024-05-10 09:26:53 +00:00
stephan a8c43830f2 Add a maintenance-reminder comment explaining [dc58ed65492857].
FossilOrigin-Name: 857f5d7fadce1cfe30b3468826f403f736cfecacb1eb6d3b858b01ea3cfd6ffc
2024-05-09 18:39:54 +00:00
stephan 0e93237b17 Reformulate a for-in loop to a for loop in wasm.xWrap() to work around a downstream framework-level misbehavior, as reported in [forum:b549992634b55104|forum post b549992634b55104].
FossilOrigin-Name: dc58ed654928574ef254af289bd21c654c587aac80aea36ab96772c35d4ca754
2024-05-09 18:00:14 +00:00
stephan f0c9bc596e Rename a shadowed local var, as reported in [forum:2ba65af3148585cf|forum post 2ba65af31]. No functional changes.
FossilOrigin-Name: d030c87c4d410e9ca2b90ec7cb63e752f4490c60e2feac84f233861593142c7d
2024-05-08 17:57:45 +00:00
drh 788ade3487 Allow arbitrary expressions as the second argument to RAISE().
FossilOrigin-Name: 003e1c8c27824cb917b3869bdf9000f32ff0b6887a2aff8516712cfe865cf34d
2024-05-08 17:42:13 +00:00
drh 538ad6ce58 Fix a hyperlink typo in session documentation.
FossilOrigin-Name: 42d67c6fed3a5f21d7b71515aca471ba61d387e620022735a2e7929fa3a237cf
2024-05-08 11:51:56 +00:00
dan d1122a4c0f Fix bug in test script sessionconflict.test.
FossilOrigin-Name: e29decb8b1991bb586f41fd74bbaa4d6c4bd63e22617145828b2500d697d7869
2024-05-07 19:45:05 +00:00
dan a40cae7802 Ensure successive scans of pragma virtual tables return consistent rowids.
FossilOrigin-Name: 6838bf3ac130c5862f5fbc718b3e49ece154fa840b6539af2616cffa31e73808
2024-05-07 17:58:07 +00:00
stephan e871779f7e Fix a compilation error in test/lemon-test01.y, reported in [forum:f0ad095705|forum post f0ad095705].
FossilOrigin-Name: 576b68c8b5132eb8901789b7fafb37e5ad7a00c1c7154090b42ddc5934ac0396
2024-05-07 13:20:24 +00:00
dan 11a088b7eb When extracting links from sqlar archives, clobber any existing file or link, and do not call utimes() to set the timestamp - it looks through the link and operates on the target.
FossilOrigin-Name: 2bf8c3f99ad8b74f707d17272fa12b674bec66082d3e8349ebef3dac42ba0782
2024-05-06 20:21:31 +00:00
drh 07305e3a3a Omit redundant unary + operators from the AST.
FossilOrigin-Name: f81cc149e5e308785502b36e83eaaf3d31c72520793e0596d5496abd70f9cda8
2024-05-06 20:18:30 +00:00
drh ebf9702b34 Avoid unnecessary recursion in sqlite3ExprDeleteNN(). This complicates the
code, but it is needed to prevent nuisance "stack overflow" reports from
OSSFuzz while it is running the latest ASAN.

FossilOrigin-Name: 70abc144ca90a58ea25dc2d90683545246c084d961215c20ec070b0abe640371
2024-05-06 19:04:46 +00:00
dan f4744ff1ff Add new sessions API sqlite3changegroup_add_change().
FossilOrigin-Name: 5eaab43ce48bb8d710f784ecd6aa9a4b3e708b44b0f7e49daf66a3f8bc2b9873
2024-05-06 17:55:19 +00:00
dan cbea8d5c3f Edit comments in sqlite3session.h.
FossilOrigin-Name: 49f84878e21f85d44d8429cdc2908f87383e59737e1af3fdbccf0c276d883567
2024-05-06 17:32:26 +00:00
dan 0f78aafec8 Add tests for sqlite3changegroup_add_change().
FossilOrigin-Name: 9d67bedc20c3a59e5488f0eaea6a6611940a9cb63f7cf5e687b061cb5d510943
2024-05-06 17:21:23 +00:00
dan 1002bcff43 Fix problem in sqlar_uncompress() when being used to extract symbolic links.
FossilOrigin-Name: 4d90c3f179a3d7355b6b0202faf56f0be4be56986920576e528208cb84daa4af
2024-05-06 11:27:47 +00:00
drh 1aa94b5821 Add comment to the test/lemon-test01.y file indicating that it does not work
and is for historical reference only.

FossilOrigin-Name: 9bdf73a8d4a3b4a72f2882252f9ed29a6f1a26d3f50c071f27455bcf10458c4b
2024-05-05 10:00:02 +00:00
dan 5b80dbe6b3 Add new sessions API sqlite3changegroup_add_change().
FossilOrigin-Name: 73647db1ddfdaf40cbf18f1e47c10b4a906489f6d34d7667f0f2ff532f1eb37c
2024-05-04 21:10:24 +00:00
dan c52c00f670 Add test cases to test/in7.test. No code changes.
FossilOrigin-Name: 8c3086f9fe502dfc4a1fa610a23794fb037721df29dc5d2126cfb749a9d44a50
2024-05-04 16:50:47 +00:00
drh e8ba446a9a Assume that an SQL function is able to return a subtype if any of its
arguments are SQL functions that are able to return subtypes.  This closes
a corner-case hole in the patch at [ba789a7804ab96d8].

FossilOrigin-Name: b36d499e4cdb41a5d7e44a1c4347a059d7654f85ade9c5c04d18ac95ddc09fde
2024-05-04 15:10:37 +00:00
drh 2eb9adb564 Assume that a function is able to return a subtype if either (1) it is itself
marked with SQLITE_RESULT_SUBTYPE, or (2) one of its arguments is a function
that is able to return a subtype.  This check-in backs out the code changes
from the previous two on this same branch, but keeps the test cases from
the previous two.

FossilOrigin-Name: f16b200f25a0ec59ad765d254d81c3ffdba21f79e6e82807a7b80d00627952e2
2024-05-04 15:04:45 +00:00
drh 5b9b7cba22 Functions that pass through the sqlite3_value of one of their arguments must
also be marked as SQLITE_RESULT_SUBTYPE, in case one of their arguments has
a subtype.

FossilOrigin-Name: 2f9fba931d9f80b3d5dffb175180098756bccc6a8f665d7aaf8826970ab60d72
2024-05-04 11:31:34 +00:00
drh 0901571f0a The coalesce(), ifnull(), and iif() functions pass through subtype values
from their arguments, and hence need to have the SQLITE_RESULT_SUBTYPE flag
set.  This fixes an corner-case for the patch at [ba789a7804ab96d8].

FossilOrigin-Name: cdd1610c44876623e629bb8e5779ea689e6d23c545552b088eca63ad2d1cf8da
2024-05-03 19:35:43 +00:00
dan 9c0d777a9f Fix a problem with queries that use "IN(...) ORDER BY ... NULLS LAST" or similar introduced by [d7648e21605a0b38].
FossilOrigin-Name: 26b2d9390f1273ee0e3f3c96f6bc5bf0e9c1dcfd0d9ecd993dba487ef5a07f7e
2024-05-02 19:22:23 +00:00
dan 72814933b9 Avoid an assert failure accessible only when internal testing features are enabled. dbsqlfuzz crash-66abacda6bca6bd531e25595e8c8068e2c710326.txt.
FossilOrigin-Name: fbc446daac761dd1b66a85c9b61e1d5af194a8fd665600c43a874da38cc6ec10
2024-05-02 18:16:23 +00:00
dan 01508c814c Fix a case where the recovery extension might write uninitialized data to the recovered database.
FossilOrigin-Name: 8519b4d0393ed17eed06bceffbc891fe8cae0bc1d466a79eba0602ef46b07c56
2024-05-02 17:46:52 +00:00
dan aeb62886c5 Avoid an OP_Next in cases where an IN(...) query against a UNIQUE index may return at most 1 row.
FossilOrigin-Name: d7648e21605a0b381b1935a47e5a77bdfacc4cc96f03d6bb7956f8f4e6a40648
2024-05-02 14:48:58 +00:00
drh d059774365 Fix the CREATE TABLE performance issue described by
[forum:/forumpost/4cf69794d9dfff7c|forum thread 4cf69794d9dfff7c] in two
different ways:  (1) Omit the call to PRAGMA integrity_check('X') that was
being done after CREATE TABLE "X" because the result was being ignored and
the integrity_check was not doing anything other than burning CPU cycles.
(2) Do not interpret the argument to PRAGMA integrity_check as a number if it
is in fact a string that looks like a number.

FossilOrigin-Name: 71f08b912251c8a3ac1bd8e344903336648e4187f7493f8c126e60b3b51b9f09
2024-05-02 12:14:31 +00:00
drh 9b4e7898d7 Add a test case to the fix to PRAGMA integrity_check in the previous check-in.
FossilOrigin-Name: 39a57b596d309d9e2d4c59409fe5d36e0665781ce0307d157ff8b6e883161bbb
2024-05-02 12:00:36 +00:00
drh 78bc1339ac Do not accept a string that looks like a number used as an argument to
PRAGMA integrity_check as a number.  Treat it as a table name that just
happens to look like a number.

FossilOrigin-Name: b04e7a23478f1012e501a810f3e09cca81a66e802f5f72cae80c81120174e2cb
2024-05-02 11:52:31 +00:00
drh 2ae4531c89 Omit the OP_SqlExec to "PRAGMA integrity_check" added by [348fa7aaf7958b3f]
because it is a no-op.  Even if the integrity_check failes, the CREATE TABLE
is stull successful.  The OP_SqlExec just burns CPU cycles for no reason.

FossilOrigin-Name: 532795acd1c800751737fe70148f9ae691e9cf11b836577f8538421d24cab2fe
2024-05-02 11:51:26 +00:00
dan f3ea92cceb Avoid an OP_Next in cases where an IN(...) query against a UNIQUE index may return at most 1 row.
FossilOrigin-Name: 560f64157d2fe40e107582eebb6526185c9c43305e364f4132e182dbec5b210a
2024-05-01 19:48:24 +00:00
dan 2ad1e92858 Fix another problem in the recovery extension where a corrupt sqlite_schema table could lead to excessive memory consumption.
FossilOrigin-Name: 1c7e33a8aa0e6122b5ef606e4a7d95e8ecd9440216d4b099fe8f2a40653422be
2024-05-01 16:25:46 +00:00
dan 19ca99eea4 Fix a problem causing the recovery extension to use excessive memory and CPU time in some cases.
FossilOrigin-Name: 8de85170d53e1d2a2abe14508a222377b9493e25f3174b8f0f773427deb8df26
2024-05-01 14:42:20 +00:00
dan 8d5fe073d2 Avoid 32-bit overflow when calculating ncycle for ".scanstats vm".
FossilOrigin-Name: 2858efa06d4fc7b412b892f35f5e9a766b467b4a5b74d602a030d25443f9efb4
2024-04-30 19:34:15 +00:00
drh b11daa50f9 Fix issues in [/info/1e227ad9f413227f|LIMIT/OFFSET support for virtual tables].
The first problem was reported by
[forum:/forumpost/c243b8f856|forum post c243b8f856].  That report prompted
an enhancement to the generate_series() (also included in this merge) which
in turn identified other similar issues.

FossilOrigin-Name: 5f6c079d847e3664ec5acaf1b3e989efe0d548c211ae4a18936162b36df89065
2024-04-26 19:10:15 +00:00
stephan b95c1d0ac9 Internal JS doc touchups. No functional changes.
FossilOrigin-Name: 626b997b4e727554d7ec8b60fb37b2a94cb861a87325da4ae484bd9aa8961ca5
2024-04-26 18:42:50 +00:00
stephan 6c290ccd49 Additional internal docs in the wasm utilities.
FossilOrigin-Name: 5a2245a9ebae6d23cd343e46b9d730f66ec4d5ffc91b83ed11a0fbd2194ad807
2024-04-26 18:24:23 +00:00
dan 4c3ab545a9 Allow virtual table implementations to handle OFFSET but not LIMIT, but not LIMIT but not OFFSET.
FossilOrigin-Name: 90e5c8226a695e838e8c1703a9b8598e654d216799e8806c4d1a1f20c28c6486
2024-04-26 18:13:11 +00:00
dan 297472a2fc Have where.c ignore any plan from a virtual table that tries to use LIMIT/OFFSET without also using all WHERE constraints.
FossilOrigin-Name: 7d30596496c6a7a37b925f13d8d94d5de224ec31bb86594fa4cc07b10082e776
2024-04-26 17:19:59 +00:00
drh d3a4dbe4b8 Further fixes and improvements to the generate_series() enhancements on
this branch.

FossilOrigin-Name: b7d9bd7ee2f4100608063fdf7648f290351465d393bc876a89704f643358853e
2024-04-26 17:09:33 +00:00
drh 84a231eb61 Use hex flag masks rather than decimals in ext/misc/series.c.
FossilOrigin-Name: a94e2cd02873c283d46bf6c21d0306ad454881d7882bb167d043cc79f79a2396
2024-04-26 14:36:28 +00:00
dan 3afd5b6d65 Fix a problem allowing a LIMIT constraint to be passed to a virtual table in cases where there exist WHERE terms that cannot also be passed.
FossilOrigin-Name: 72c8ed9698dd2aadee7b84fd293e8306233f0fe5b5b5731687482444fdf461c7
2024-04-26 14:32:58 +00:00
drh 99ac2324aa Fix handling of LIMIT and OFFSET in virtual tables that are part of
a compound SELECT.

FossilOrigin-Name: 40421c1c4ed5bb1ed79ad7ee37cb5a4f0b7864c1eb94abd8ee357ab2202cad30
2024-04-26 13:38:43 +00:00
drh 317b7416a6 Enhance the generated_series() table-valued-function to respond to
LIMIT and OFFSET.  Use this to add new test cases for LIMIT and OFFSET
on virtual tables in a compound SELECT.

FossilOrigin-Name: 408d47ecaa3b906d0886f76a22b76339ec5878270ffe8d1838c74de09c29a33e
2024-04-26 13:30:48 +00:00
dan d591a5b93d Add test demonstrating the problem at [forum:/forumpost/c243b8f856|forum post c243b8f856]. No fix yet.
FossilOrigin-Name: 1685495c0a00238c9c92cce01af8108204a2fad22433ed3e7bba3c9da9ee0766
2024-04-26 12:01:17 +00:00
drh fed13d50a9 Fix the TreeView display of a LIMIT clause on a compound query. Debugging
code only - not part of production builds.

FossilOrigin-Name: 38f1e5ce4eedd59fbeb0a0d676d26dfe8a3313189ab9b9e4a5a1ed6721b73a14
2024-04-26 11:32:50 +00:00
drh ec14ef80b0 Improvement to the way that affinity is determined for columns of a
compound subquery.  The affinity is the affinity of the left-most
arm of the compound subquery that has an affinity other than NONE, adjusted
to accommodate the data types coming out of the other arms.

FossilOrigin-Name: e6df846f36209bac3e420dd80ce2bbbd87ab7a20b8063fce05f78a3c7ab6027e
2024-04-25 23:26:11 +00:00
drh ddc6bd9f21 Further improvements to the computation of affinity for compound subqueries:
make sure that the selected affinity is compatible with a literal values in
arms to the left of the arm that is used to determine affinity.

FossilOrigin-Name: bbdf22e3d989f42b963f1f2f219dfeac11db786f17ac27097ab72f72e7638a2a
2024-04-25 17:52:10 +00:00
drh 23afc2c4ec Back out the previous change on this branch. In its place: Determine the
affinity of a subquery by the left-most arm of the subquery that has an
affinity other than NONE.  In other words, scan from left to right looking
for an arm of the compound subquery with an affinity of BLOB, TEXT, INTEGER,
or REAL and pick the first one found.  Or stay with NONE if no arm has a
defined affinity.  Test cases added.

FossilOrigin-Name: b8ec8511b1968bbc1472b3e2e21f0fef1d5becebeb31f9d13ee3ca9e13abb1e5
2024-04-25 16:55:53 +00:00
stephan c859f0267c wasm: add a makefile comment about the static sqlite3.h/c version info injected into the JS files possibly differing from the runtime-emited version info when a user provides their own sqlite3.c.
FossilOrigin-Name: d99a01a0f6e1f70f70c9a0625aeaa8a8015eba352bcfb3978eafca6df10ba5a8
2024-04-25 06:52:19 +00:00
drh f8e195159f An experimental minor tweak to the way affinities are computed for compound
subqueries, when the actual affinity is ambiguous.

FossilOrigin-Name: 779723ad792ca24dd07b7a1425303c76b44bb173e718a33c26c9f7644e9912cb
2024-04-24 19:49:52 +00:00
drh 0868d58e2c If a RETURNING clause contains a subquery that references the table that is
being updated in the statement that includes the RETURNING clause, then mark
the subquery as correlated sot hat it is recomputed for each result and not
just computed once and reused.  See
[forum:/forumpost/2c83569ce8945d39|forum post 2c83569ce8945d39].

FossilOrigin-Name: 9ea6bcc8fdf6aadb756ec5bcaaa7af314167f8973bdd32fd23f83bd964f0c21e
2024-04-24 16:36:37 +00:00
dan dfa6289153 Fix a case where a corrupt stat4 record could go unrecognized due to integer overflow.
FossilOrigin-Name: 240a4a48b27a2b1070bba6d7da76a8df8e3f1808e262045d7ad2cf910df08f3d
2024-04-24 14:18:20 +00:00
drh 35b771e221 Add comments linking the assert() added in [cef4d9e3ba586735] to the places
where the precondition that the assert() tests are actually required.

FossilOrigin-Name: 6f0e7e195275aeb4aefd9da20348af35e3ef7f0a6b2768a34824daeace16eff1
2024-04-24 11:51:51 +00:00
dan 5b27ecafbc Avoid slowdown when exprNodeIsConstant() is called on deeply nested SQL functions.
FossilOrigin-Name: 1c0b0345451853846076467831884f951c92ec79476be40ce7f7b766fd2d8fc1
2024-04-24 11:40:47 +00:00
drh b065eceb56 Ensure that temporary SrcItem objects created by trigger processing have
either SrcItem.zName or SrcItem.pSelect defined.  Every SrcItem should have
one or the other.

FossilOrigin-Name: cef4d9e3ba586735598f03eb5e8f29072c9e6f62b0d34ddd2fb3ed1795f6e21c
2024-04-24 11:21:27 +00:00
stephan 99200b2f07 Remove an undocumented feature from the JS Worker1.open API because it relies on broken behavior which has no VFS-agnostic workaround.
FossilOrigin-Name: 3a640038c70c8511e7051af83aa35d163c4f96f05c5064cffd3e4e2e63cd44a9
2024-04-24 06:43:17 +00:00
drh 14602eafa7 Fix a problem with vector IN operators used with an index where the affinities
and collations for the various vector terms are different.

FossilOrigin-Name: 86e8c782e7494377de3c27b750cd83a7eb2302c1182ee2004d3864db50fca557
2024-04-23 12:02:03 +00:00
dan 0309fb919a Fix an uninitialized variable problem in the new code on this branch.
FossilOrigin-Name: 85625b38718c6e4cf7aa97fa3c52153bd25c810b68f19e7924e1f4b709c6a05d
2024-04-23 11:56:20 +00:00
stephan 3ac612dd7d Remove some dead WASM-side code.
FossilOrigin-Name: 0a07ee27bd6021a6fc1d81133012592351530ffcf6ae09322ea74624cff910df
2024-04-23 06:49:47 +00:00
stephan 642e950e63 Minor touchups in JS docs and exception messages.
FossilOrigin-Name: 9e7fc9370dfca121244f7a2941e8de629b277f1799f8de08a43ff1d86f94b6f5
2024-04-23 06:36:28 +00:00
stephan 6e6dfd6a23 When running the 'dist' target in ext/wasm for an SEE-capable build, ensure that the resulting zip file and directory name include '-see'.
FossilOrigin-Name: 04c552b12e3b77b9dfd83838d35ce19a37ed024a8c18a2000ada10cf3d1eb6ad
2024-04-23 05:38:49 +00:00
dan 46435a04fd Add extra tests for aborting conflicts in the sessions module.
FossilOrigin-Name: b4a6d32662acacb7767cfb9b8e040e6eb1f99322cb7d0cd44e6265e9ac2fb2e8
2024-04-22 20:09:17 +00:00
stephan 5d8daf184d Minor cleanups to [8fbda563d2f5].
FossilOrigin-Name: 5ee2594b657b96aea9e482a175820dcbacfa9298da45b38cf17f39ad076d3f7f
2024-04-22 17:03:52 +00:00
stephan 0a42e9913b Extend the JS/WASM SEE build support by (A) filtering SEE-related bits out of the JS when not building with SEE and (B) accepting an optional key/textkey/hexkey option to the sqlite3.oo1.DB and subclass constructors to create/open SEE-encrypted databases with. Demonstrate SEE in the test app using the kvvfs. This obviates the changes made in [5c505ee8a7].
FossilOrigin-Name: 8fbda563d2f56f8dd3f695a5711e4356de79035f332270db45d4b33ed52fdfd2
2024-04-22 16:46:37 +00:00
drh 4f2f6c74ca Extra robustness in the code that causes cursors to return NULL when they
are participating in an OUTER JOIN.

FossilOrigin-Name: 672c2869ef48e08447d37b0d76a1850cdafbe30ca1906ec98c55e3ab496fd9a6
2024-04-22 13:31:24 +00:00
stephan 2469350ad9 For sqlite3.oo1.DB JavaScript classes, bypass execution of any on-open() SQL in SEE-capable builds because it would necessarily run before the client has an opportunity to provide their decryption key, which would leave the db handle in an unusable state and cause the ctor to throw. This currently affects only the OPFS VFSes. We may want to consider extending the ctor options object to optionally accept an SEE key and apply it when opening the db.
FossilOrigin-Name: 5c505ee8a73f4b4a7053d98a12024d98340676f6ae9982311f9f88a9b46c8ae2
2024-04-22 11:48:03 +00:00
drh 2c26adb873 Continuation of the fix at [8c0f69e0e4ae0a44]: If a viaCoroutine FROM clause
term is participating in a RIGHT or FULL JOIN, we have to create an
always-NULL pseudo-cursor for that term when processing the RIGHT join.
dbsqlfuzz 6fd1ff3a64bef4a6c092e8d757548e95698b0df5.

FossilOrigin-Name: e1040e51ebd04f2a076f477b6f240f849afb10f543ebe518e09d6842cc3cb38e
2024-04-22 00:42:47 +00:00
drh db2a33a3dd Fix an indentation problem in debugging routine sqlite3ShowSrcList(). No
changes to production code.

FossilOrigin-Name: 5560df2726fed215c98d602c5f6ebbf3c5ae23f8ba1074291c509bf446fdfe66
2024-04-21 23:35:58 +00:00
dan 5749e1ee43 Slight performance improvement for the new code on this branch.
FossilOrigin-Name: 500c67f1341fe2a7e7333d525c90df201cc73a683b943ad5c1e41d4a4f639043
2024-04-20 19:19:30 +00:00
dan d737feeacf Fix a problem where an expression like (a, b) IN (SELECT ...) might not use an index on (a, b) if the affinities and collation sequences of "a" and "b" are not identical.
FossilOrigin-Name: 4d870fd8b5450047a7486fc023d1ac9439642e8ed91eadfd5026c4cda7cc9179
2024-04-20 15:14:06 +00:00
drh 0526db32e2 Correct handling of OUTER JOIN when on or the other operand is a subquery
implemented using the VALUES-as-coroutine optimization.
dbsqlfuzz bde3bf80aedf25afa56e2997a0545a314765d3f8.

FossilOrigin-Name: 8c0f69e0e4ae0a446838cc193bfd4395fd251f3c7659b35ac388e5a0a7650a66
2024-04-18 16:11:01 +00:00
drh 06915d0145 The first assert() added in [0ebc65481f4a3e79] is not necessarily true in a
corrupt database file.  So add a term to make it true.

FossilOrigin-Name: 6b21cccdeec92db9f6ce3dd7ea5e61b8b46650cc1e550271aa51bdc619f55b11
2024-04-15 20:43:21 +00:00
drh 557ba97227 Merge all of the latest trunk enhancements into the bedrock branch.
FossilOrigin-Name: 16dac180b5fcc619104c789ed8bfec691fc6b808294aa41d9e0795ad2810a83a
2024-04-15 14:45:06 +00:00
drh ba9c7c10cc Merge the latest trunk enhancements into the wal2 branch.
FossilOrigin-Name: 4a72acbc4b87441dd0cacdcfdeac7cce7b1303d00ab494249d98f4518ae3eacf
2024-04-15 14:39:19 +00:00
drh 0221cacea7 If a build fails in testrunner.tcl, do not attempt to run the jobs that
depend on that build.  Instead, report those jobs as having been skipped.

FossilOrigin-Name: b40580be719a129ecd1aa3c69d1086c967d063920fdd48617c864e73c059abc1
2024-04-12 18:46:34 +00:00
drh 6f4e5ee4c9 Fixes and new tests logic to ensure that the btree overflow page cache is
only used when it is consistent.  This resolves the malfunction observed
in [forum:/forumpost/284955a3cd454a15|forum post 284955a3cd454a15].

FossilOrigin-Name: 5dede50d9e7b6942df9f7b00fbfeaa2103c36c5da01d63d88136fb0ef4b7d26d
2024-04-12 15:02:16 +00:00
drh 1d40237bc8 Add assert() statements to help verify the overflow page cache.
FossilOrigin-Name: 0ebc65481f4a3e7974558adea51c620a025bc0e76c0a139e549b56c5abe0cabb
2024-04-12 13:05:36 +00:00
drh ae7ede4a4f New assert() statements to verify the correctness of the BTCF_AtLast flag
on btree cursors.

FossilOrigin-Name: 4efecd6167de71500c90b63155eba1b8567c90e9d1e282fbea54130f9ee21813
2024-04-12 12:32:09 +00:00
drh 366b419d00 Test case for the fix in the previous check-in.
FossilOrigin-Name: 0cf4d835dae260b01178e94e77be6b8a130f2031e898ef79ceba8df6c2bba58f
2024-04-12 11:28:35 +00:00
drh c494fdb580 Mark the BTree cell overflow cache as invalid whenever the rowid goes invalid.
FossilOrigin-Name: 74c9e19c92c887012aebbe96450f6ed7a60ba22d6e3edbaa39a0f989fb7f2901
2024-04-12 02:57:52 +00:00
stephan b6d108f0b3 Fix handling of the SQLITE_OMIT_WAL check in one place. Resolves [forum:87cc13302de160eb|forum post 87cc13302de160eb].
FossilOrigin-Name: 5dae6e6df4921f42e45c6c8de40853ab63f53a4bd1d9088a8cdac957ce62f196
2024-04-11 19:08:12 +00:00
dan 495ce22d49 Fix a "shift exponent is too large" usan error that could occur in the fts3 snippet() function when a query phrase contains more than 64 tokens.
FossilOrigin-Name: 6d2c7ac4ecfc1ec408cb1820985a70650d7cc5613b4b8fd1829f68ea48fce6a3
2024-04-09 19:23:18 +00:00
dan 73a50fa42c Avoid some performance problems in the recover extension when recovering strategically corrupted databases.
FossilOrigin-Name: e66a834bbab9d71e02b70588ad4c71eae2e13b50e299b0269d488402bc4a7fc5
2024-04-09 15:09:54 +00:00
drh d21d5b21db The read-only CHECK-constraint optimization of [34ddf02d3d21151b] inhibits the
xfer optimization for tables with CHECK constraints.  However, the xfer
optimization is required for correct operation of VACUUM INTO on tables that
contain generated columns. Fix this by ignoring CHECK constraints when
qualifying the xfer optimization while running VACUUM.  Problem reported by
[forum:/forumpost/3ec177d68fe7fa2c|forum post 3ec177d68fe7fa2c].

FossilOrigin-Name: a6e26e778812c8409fca77183e24d3b70189c4d02fce10c7e74cd4ccc8c8ea97
2024-04-09 13:57:27 +00:00
dan 1278ac411c Update tests in returning1.test to account for [c7896e88].
FossilOrigin-Name: c02956cd5cbf87027ba201b5d273e0efd429836ee6cabdc2f6d5b8096c876b7b
2024-04-08 17:55:15 +00:00
drh de4df544c1 Merge the latest trunk enhancements, and especially the enhancement to the
WHERE-clause push-down optimization, into the bedrock branch.

FossilOrigin-Name: 0f93b52c7727034f3dffd90e606f97f54fe8940db79832923a046808bb0a7699
2024-04-08 12:02:55 +00:00
drh 7d13a403c4 Merge the latest trunk enhancements into the wal2 branch.
FossilOrigin-Name: 59ccea7db787a32091466b2b49c7d9f9db4f9d712830d9b3e48aabc9e45f7ace
2024-04-08 11:57:01 +00:00
drh d59afcfbfe Enhance the WHERE-clause push-down optimization so that it is able to
push down WHERE clause terms that contain uncorrelated subqueries.

FossilOrigin-Name: 69ec714b2d698acf9e37635256c01b233ce32f22e8323e226441d5ddd948a940
2024-04-08 11:50:07 +00:00
drh 0389d5f8d0 Fix a harmless compiler warning in the ICU extension due to [acddbc489d5231da].
FossilOrigin-Name: 3d8750e88558f8ea6a1523a5add4ccca3262e31f348b603e56ab193d81908995
2024-04-08 11:45:15 +00:00
drh ac94cf3ece Enhancements to the fuzzer invariant checker to turn optimizations on and
off.

FossilOrigin-Name: 67594481379824823105939fea2ce1fe280667f6db91735ac78b4b6164a78dab
2024-04-08 06:37:19 +00:00
drh 5b613ed41b Merge trunk testing enhancements into the pushdown-subquery branch.
FossilOrigin-Name: 287ff24b26a512ff7648679767e68244f6eef95df6a49c46ed1f2594030ed523
2024-04-07 18:55:57 +00:00
drh fa4c4247a5 Omit all rowid-in-view restrictions from the fuzzinvariant.c test module
as they are no longer necessary, as of the previous check-in.

FossilOrigin-Name: 6431538f0bb3bb8606786f3c3e5c055c4bc387098dd3bdc8a94f6fda61c47f52
2024-04-07 18:36:32 +00:00
drh 1152def76e If SQLITE_ALLOW_ROWID_IN_VIEW is set to 2, then all rowids for views return
a value of NULL.

FossilOrigin-Name: 0a53dde21403aa6de11c5085c16def3f95046c5629daf2675b075e4d6683ef94
2024-04-07 18:23:30 +00:00
drh 660322f96f Merge trunk enhancements into the pushdown-subquery branch.
FossilOrigin-Name: 27865e316f8dfbf4c20290cf1be3024d7518fec46655e34f3fc435e15346c63e
2024-04-07 10:30:29 +00:00
drh 05c6d13027 Add comments to note the name abiguity between the MySQL push-down
optimization and the WHERE-clause push-down optimization.

FossilOrigin-Name: 3d5fb1ec7a0440072d6e3b957903c85d0f32b8b07207a1ef22d1a69cf5e664d1
2024-04-07 10:27:18 +00:00
drh 6951c49779 Do not allow changes to sqlite3ExprIsTableConstant() that support pushdown of
subqueries interfere with the hash-join logic.

FossilOrigin-Name: 8682931f9c6b40e1b09139c10bbe932c38330b5eb0c5c84f2432ad19a6793e37
2024-04-06 18:30:09 +00:00
drh ed27183d83 Merge testing logic fixes on trunk into the pushdown-subquery branch.
FossilOrigin-Name: a4e1f03dcad70e828a4b56e7a332946daf84d6eae078e3ec9a248b8157a53963
2024-04-06 17:42:14 +00:00
drh 8181cc6c10 Improvements to the accurancy of the fuzzinvariants.c testing module when
SQLITE_ALLOW_ROWID_IN_VIEW is defined and the test query involves rowids.

FossilOrigin-Name: c6e873d4db3ef36a0d561e64ead6feada5d1654c0757b4b6e55f671c9db66469
2024-04-06 17:37:30 +00:00
drh de6a4be9b5 Generalize pushdown to allow any uncorrelated subquery to be pushed down.
FossilOrigin-Name: 87c45fb0d5f5ca5d6d1ad27bef83f294231d17d94299e1997364a7975b423e38
2024-04-06 12:19:50 +00:00
drh 743ae4c68b Experimental enhancement in which expressions of the form "expr IN table"
can be pushed down into subexpressions.

FossilOrigin-Name: 2cbd7838fd6ffdf210f34671cd2e3e749a076a3a6f155bbe5f910a67db31c5b1
2024-04-05 20:01:28 +00:00
drh bc4ac5f7c6 Merge the latest trunk enhancements into the bedrock branch.
FossilOrigin-Name: 7a2a64b64655e9fdc551765f9f6b6a4d2898a488230eca240b6ee4df86b5c491
2024-04-05 14:54:26 +00:00
drh b7e87a53c7 Merge the latest trunk enhancements into the wal2 branch.
FossilOrigin-Name: a8a8a2db9b879c543fb3fd25cf66717a3dfdf9857dcd1ab9a98b31fbc85dbfc9
2024-04-05 14:50:12 +00:00
drh b411c4d69e Check-in [a9657c87c53c1922] is wrong: the IndexedExpr.bMaybeNullRow flag is
required for virtual columns if they are part of an outer join.  Add a
test case (derived from dbsqlfuzz b9e65e2f110df998f1306571fae7af6c01e4d92b)
to prove it.

FossilOrigin-Name: 4484ec6d26b31305e31de89bdbae26344d8083a7e7de20861430d31737d9979c
2024-04-05 13:56:05 +00:00
drh 65a1d7a284 When compiling with SQLITE_ALLOW_ROWID_IN_VIEW, if the RETURNING clause of
an UPDATE of a view specifies a rowid, then return NULL for the value of
that rowid.  dbsqlfuzz 7863696e9e5ec10b29bcf5ab2681cd6c82a78a4a.

FossilOrigin-Name: c7896e88850669e18e89d44c4169d4f4a5d4b904bea6ccb2ac64f93b6d348a42
2024-04-05 11:23:51 +00:00
stephan 7fd9b53308 Optimize sqlite3.oo1.DB.exec() for the rowMode='object' case to avoid converting the object property keys (column names) from native code to JS for each row. This speeds up large data sets considerably and addresses the report in [forum:3632183d2470617d|forum post 3632183d2470617d].
FossilOrigin-Name: 8b41ef8690001eb299f5b7182c28f5318333bff5b505e1d59d6e6f4556b1c759
2024-04-04 22:53:09 +00:00
drh f895173c5f Add the "interstage-heuristic" that attempts to avoid wildly inefficient
queries that use table scans instead of index lookups because the output
row estimates are inaccurate.

FossilOrigin-Name: 7bf49e2c54c9f6f336416f01c0e76aaf70f1e2f3fd612232e5a33ae5dabe0900
2024-04-04 14:26:42 +00:00
drh 634f434686 Fix an ASAN problem in part of the test harness. No changes to SQLite itself.
FossilOrigin-Name: 797cda7ddcceb140330d58892c3e73d28df72b638df00fd48f07dfcba7706c5f
2024-04-04 13:36:48 +00:00
drh efe9500af7 Fix test cases due to an error message change.
FossilOrigin-Name: c49ec239eddde6f07f8d528cccd6224c4c80b240dc8f5c974186e3bb08afb83b
2024-04-04 13:07:43 +00:00
stephan 13f5910889 Fix exception propagation when the opfs-sahpool VFS fails to initialize to address report in [forum:be0141c639|forum post be0141c639].
FossilOrigin-Name: cdb3a90055afe371962ec6c0931f1b08797aaeaab0358ef64b1cabda0a855b33
2024-04-04 12:36:33 +00:00
drh 307f290b32 Add an optional 3rd argument to the icu_load_collation() function that
specifies the "strength" of the comparison.

FossilOrigin-Name: acddbc489d5231dacf29890d1aa562da499026690f55c256a07d04a3190faacc
2024-04-02 21:55:45 +00:00
drh 8ce73ce15b Fix typos in comments. Provided ".wheretrace" debugging output for the
interstage heuristic module.  Do omit automatic index loops in the
interstage heuristic.

FossilOrigin-Name: 186dcae19e249db36de15f295999cff25063b54ee3d5d481cd2ba99b6d13148e
2024-04-02 14:12:29 +00:00
drh 7f7d0b19f8 Add a heuristic in between the two solver() passes of the query planner that
tries to prevent a very slow query plan in cases where the output row count
estimate is imprecise.

FossilOrigin-Name: 8018417b0143ea11535f2457bf3e4b3755717c554a17df1076425b4251b5f2c6
2024-04-02 11:44:44 +00:00
drh 9352cfa30b Improved comments in the query planner logic that computes the cost for a
particular step in a query plan.  No code changes.

FossilOrigin-Name: 0b2ac2cdc767db764e3ea8bbc33898cac4e1ec27fe8c9b60ce08a1785f921e6d
2024-04-01 15:38:15 +00:00
stephan 69dbd7a4e7 Make explicit that sqlite3_keyword_name()'s index is 0-based, per forum request. Doc changes only.
FossilOrigin-Name: 090943dc31e7a3af5c11c1c0953cb82ae3ca07ba000189bb85deaecc76921504
2024-03-30 14:11:30 +00:00
dan d12fb81174 Fix handling of an OOM case broken by [11015b4e].
FossilOrigin-Name: 1f3a805d19c03152bffba783f31d7c2043206c43221847005190c93033d97cf0
2024-03-29 18:19:51 +00:00
dan 4b2d8ce622 Fix a problem in SQLITE_DIRECT_OVERFLOW_READ builds that could allow a concurrent transaction to be committed even if it read from an overflow page that was modified concurrently, in cases where the overflow page was written without also writing the b-tree page to which it is linked.
FossilOrigin-Name: 11015b4ed161263ae14bcd0fe6a24074a545bb6d2a0b3f6f1f75ba3eaa91bfc0
2024-03-29 18:07:41 +00:00
dan ed0cb47925 Improve the log message emitted when a BEGIN CONCURRENT transaction cannot be committed due to conflicts so that it identifies the conflicting table in a few more cases.
FossilOrigin-Name: 19ff2d4e1e7c6d29773d76ba752338c1c2e9ab355cdd45aed1d936880fa1c917
2024-03-29 18:04:56 +00:00
stephan cfd4507c83 Document that the order of an update hook call is unspecied vis-a-vis the final result of the operation which triggers that hook. Doc changes only.
FossilOrigin-Name: 3d4b1f0791384d3e531d6757daecf67e5b873954de61f37032474e3ae23cd22b
2024-03-28 10:58:18 +00:00
dan 6921a2d205 Update comments in fts5.h.
FossilOrigin-Name: 862945d5c432c27377e90d93d64c4655eefcc369d086eb51edef925fb3d80b57
2024-03-27 20:34:14 +00:00
dan ed517a7082 Add extra tests to vtabL.test.
FossilOrigin-Name: fe209099f5c348f1280b9b827ffbf6a6742ebdd1a23ef189445147894b1832ba
2024-03-26 18:07:39 +00:00
dan dcc121999c Avoid expanding integer values in columns with real affinity to the full 8-byte representation when editing records as part of a DROP COLUMN command.
FossilOrigin-Name: a49296de0061931badaf3db6b965131a78b1c6c21b1eeb62815ea7adf767d0b3
2024-03-26 11:14:52 +00:00
drh 1d16c3fc2d Merge all recent trunk enhancements into the bedrock branch.
FossilOrigin-Name: 33d3453f467791eecc81dd682890eaadbbb2afc105ef70e22a815f82ead2c350
2024-03-26 10:45:54 +00:00
drh e24103baad Merge all recent trunk enhancements into the wal2 branch.
FossilOrigin-Name: c2139d826355009e6fb30091734c388bdcb8cb0889f7225481c163850a3e71ff
2024-03-26 10:42:52 +00:00
drh a509a90958 The RAISE() operator is not a constant expression and cannot participate in
the VALUE-as-coroutine optimization.
dbsqlfuzz 74cf7c9904360322a6c917e4934b127543d1cd51

FossilOrigin-Name: 6a06dc73847716c88d65651d1bf0e002002303881df1389beac884d0032eae08
2024-03-25 20:35:14 +00:00
dan ca3b9096c4 Add further tests for sqlite3_declare_vtab().
FossilOrigin-Name: f3c4433f6b33087494c8d38ff9dbb008a1bd30b424ca9aaa51c1956d75a86249
2024-03-25 20:00:40 +00:00
drh 1872f8d844 Simplifications to the sqlite3_declare_vtab() implementation. Changes
some conditionals into assert() statements, for coverage.

FossilOrigin-Name: ff7b898a6f9cb9aecb51bd6b63e253b4b7486ac9367f59c3c0491d78cfb39993
2024-03-25 19:31:03 +00:00
dan eb5bd4db91 Fix a theoretical OOB memory access in sqlite3_stmt_scanstatus_v2().
FossilOrigin-Name: 87be9580747b405c2c534beadb0f95cee0d4f34e0245f90e157a6b7ada38e092
2024-03-25 18:30:15 +00:00
drh 791b6f36cc Revert the previous change. Instead, do a pre-check of the CREATE TABLE
statement that is the second argument to sqlite3_declare_vtab() and if
the first two keywords are not "CREATE" and "TABLE", then raise an
SQLITE_MISUSE error.

FossilOrigin-Name: 6a2ff8351244da2336055454dfad2dd40534b7cfb51e840f7f8cf2ddacf8649e
2024-03-25 18:24:28 +00:00
drh f7aab656ff Remove an ALWAYS() from a condition that can be true.
FossilOrigin-Name: 715fcf033a6c0c64fa3076d58be8c39246aebef922c1a44a31831b40e165015e
2024-03-25 17:43:11 +00:00
drh ed6c9aa6ab Fix an inaccuracy in automatic indexes that was exposed by the fix
at [80c4223098c1827f].

FossilOrigin-Name: fdc9406f1c8ba4a7341c1e408f6042ddc788cf65f98e1de2ee101390bfb0abae
2024-03-25 14:44:10 +00:00
drh bcf97e70f5 Fix the pragma4.test script so that it works with SQLITE_OMIT_VIRTUALTABLE.
FossilOrigin-Name: f8fe6fb69c4164d087783a0d6990112afabea78daa4d8985eb6df7df6f45ac27
2024-03-25 13:18:46 +00:00
drh 99b4e6e254 Fix recent test case changes so that they also work on Windows.
FossilOrigin-Name: 38578b5af2e97c420f3106bc486318604aecbaa327365bbc2e9518be84491495
2024-03-25 12:28:06 +00:00
drh 41c9945c74 Use the SQLITE_CONSTRAINT return value from xBestIndex to prohibit bad
query plans in the pragma virtual table.

FossilOrigin-Name: b1259d4448f744861e416f42328c1450854370e5c77102d2a5abe5cf6c7f12bd
2024-03-25 11:34:42 +00:00
drh 4397d28378 Improvements to the vtablog.c extension: Eliminate memory leaks.
More diagnostic output for xBestIndex.

FossilOrigin-Name: 92e9a71bc4daa261d7c9a81fb66f7d7c0f0a74eb9e0c9dec8b4651acc5217bff
2024-03-25 10:55:08 +00:00
stephan 55051d661c Flag sqlite3_trace() and sqlite3_profile() as deprecated so that the doc generator for funclist.html sees them as such, and add 'Deprecated' to their page's title for consistency with other deprecated APIs. Comment changes only. Addresses [forum:0901025836|forum post 0901025836].
FossilOrigin-Name: 87c54f93f5711739741ed0ff3c1a6fe24ffc8a025b43523bf78c1f6be8c1b4cd
2024-03-25 10:28:10 +00:00
drh d2b3b72a2b Improvements to the vtablog.c extension, for better logging of virtual
table interactions.

FossilOrigin-Name: e253bb36a5f4f601c9b08858b55a9ce198239ace8efa8dab7c0ec019028967c1
2024-03-25 00:38:55 +00:00
drh cf84ae0189 Fix the xBestIndex method of the pragma virtual table so that it correctly
gives a higher cost to plans where the schema hidden parameter is
unconstrained.  Fix for the problem reported by
[forum:/forumpost/85b6a8b6705fb77a|forum post 85b6a8b6705fb77a].

FossilOrigin-Name: bc516ff5202ee6e9834266bf755fe26e30ac557dcc7975ca7a06dfe33874fcd2
2024-03-24 21:10:45 +00:00
drh 4a402b80c5 Improved "wheretrace" debugging output for calls to xBestIndex. No changes
to release code.

FossilOrigin-Name: 33966bb06ce9a8e90734de59a8d43c523f60870d20be64789638da4c84a8b81b
2024-03-24 20:26:59 +00:00
drh a631eb378d Do not automatically assume that ROWID is NOT NULL when compiled with
SQLITE_ALLOW_ROWID_IN_VIEW.
dbsqlfuzz 31b38eeb63a4e1562de665078f52b7b47a7543cf

FossilOrigin-Name: 80c4223098c1827ff3a564f1f9a4203164a943e9a83eef99df68378fa3c4764b
2024-03-24 16:33:31 +00:00
drh d9eee787bd Fix an adverse interaction between CREATE TABLE AS and the new
[/info/a120c9235f125e05|VALUES-as-coroutine] optimization.
dbsqlfuzz c2c5e7e08b7e489d270a26d895077a03f678c33b

FossilOrigin-Name: 84b6fdea0bf07c73df0ca8ef110db066164a5f34606e6c069a060476e04ef44e
2024-03-23 15:17:38 +00:00
drh 378bf82e2b Fix incorrect boundary assert()s on the new OP_IfSizeBetween opcode.
FossilOrigin-Name: 8eda4797c573382cbb989a4ab4b1f19d8fd538dbc9818d86a9aa6189cfa90f37
2024-03-22 10:32:14 +00:00
drh 2faaf3830e Test case for the assert() problem fixed by [9ba3cff9d48fe8fc].
FossilOrigin-Name: 865f0de7a17ce6059b8f0708002cff5f13b9dba3ebc41896ee0fdc21ae4f3740
2024-03-21 22:18:24 +00:00
stephan 006a9b51aa Add comments (only, no code) to the JS sqlite3_config() bindings reminding us why SQLITE_CONFIG_ROWID_IN_VIEW is specifically not included. JNI does not need this treatment because it explicitly supports only a very small subset of config options.
FossilOrigin-Name: 4fea396221e84f9532f9327226691621a23bd966a860329fc579141c69cf7f64
2024-03-21 15:52:52 +00:00
drh 529ed1a3a3 Enhancements to testrunner.tcl:
(1) Add the "--config CONFIGS" option, to that only configuration named
on the comma-separated list CONFIGS are run.
(2) Add the "--omit CONFIGS" to omit configuration on the CONFIGS list
(3) Add the Android configuration to "release"

FossilOrigin-Name: 4ccb372967fab779b29be8f01c879a99ef247981466f81f55e18e3961fbd7d5a
2024-03-21 11:46:52 +00:00
dan af64a22762 Have os_unix.c reuse cached file-descriptors in the case when a read-write fd is requested on a read-only file and a read-only fd returned.
FossilOrigin-Name: a678e85402af08c1e387bf30ff2205f84dd7da749755da565d70f831c007a3d9
2024-03-21 11:37:36 +00:00
drh 296ae1e467 Fix all test cases so that they work with SQLITE_ALLOW_ROWID_IN_VIEW.
FossilOrigin-Name: 66c69e2f20f7692e0f34743ae97b09c4d8d11b874cdc5381795f2d1e0410f724
2024-03-21 10:35:33 +00:00
dan 219b4f2061 Change EXPLAIN QUERY PLAN output to say "USE TEMP B-TREE FOR LAST TERM OF ORDER BY", or "LAST N TERMS OF ORDER BY", instead of "RIGHT PART OF ORDER BY".
FossilOrigin-Name: 7e13a2c7583dbcb660adde6b0465da037365971c56726b8f18a541d8803ffeed
2024-03-20 16:56:12 +00:00
dan e555bd4f02 Change EXPLAIN QUERY PLAN output to say "USE TEMP B-TREE FOR LAST TERM OF ORDER BY", or "LAST N TERMS OF ORDER BY", instead of "RIGHT PART OF ORDER BY".
FossilOrigin-Name: cd547c500442f3f58c05d6da8a67c3238560c5204ea62ec14afa844cc8fb94f0
2024-03-20 16:27:08 +00:00
drh daee74c3ac Fix the SQLITE_ALLOW_ROWID_IN_VIEW bug-compatibility option so that it works
again.  Add the SQLITE_CONFIG_ROWID_IN_VIEW sqlite3_config() value
to optionally disable the bug-compatibility mode at start-time.  Enable
testing of SQLITE_ALLOW_ROWID_IN_VIEW.

FossilOrigin-Name: b6802565df0f63286a7ef26c92d3ed817895f244920a24909cd855090fdfa08c
2024-03-20 10:45:13 +00:00
drh 7128c789bd Fix a typo in a comment.
FossilOrigin-Name: 54680d0fbec1f0575ac4be6dca866971795dab2c42b02a492d5cd485f46d153f
2024-03-20 10:40:25 +00:00
drh 697e122a11 Fix typos in the SQLITE_CONFIG_ROWID_IN_VIEW documentation.
FossilOrigin-Name: 1ad1ca498296493c8e89cda911cc153baa51c90ae6c19961841e6b4e37302a23
2024-03-20 09:45:44 +00:00
drh 254729edb7 Change the SQLITE_CONFIG_NO_ROWID_IN_VIEW configuration option to be
just SQLITE_CONFIG_ROWID_IN_VIEW (without the "_NO_" in the middle) and give
it the ability to turn the option on and off.  Otherwise, it is difficult
to test.

FossilOrigin-Name: 5d412edc2e378999ad798d1d7d73c7f7a17ee4e3c751a0dd00b9d5ce32759550
2024-03-19 23:01:56 +00:00
drh 36f01f2ee3 Return the error "ambiguous column name: rowid", instead of "no such column: rowid", when a "rowid" reference is ambiguous.
FossilOrigin-Name: 0615bdae2dfc70c7e0416a28e89ffde31fa44ed4b3dac46e5ef20eed0d89e58c
2024-03-19 19:56:39 +00:00
drh 4aa21492d1 Add the --no-rowid-in-view option to the --help output of the CLI.
FossilOrigin-Name: 134da9c348a5e3b5cef97b79f37d48257afd4fa958a84f6930ac9874284a14cd
2024-03-19 19:48:50 +00:00
drh 27ea83f66e Make sure the new u1.nRow value is copied when making a copy of a SrcItem
object.  This fixes a problem in [ac6f095e13e43d66] from yesterady.

FossilOrigin-Name: 262f8f9d80d37160e4126634b99aa48fce2073e61f6365e8bdbe1cc4fa560a96
2024-03-19 19:42:56 +00:00
dan ffbe492553 Update test scripts to account for [8fcea4cd].
FossilOrigin-Name: 0c543c51a8c5c8916f649a5e7fd3fedc9c05f2df36064b1699ea6f5e08eaa9e0
2024-03-19 16:51:18 +00:00
drh e9feb0acfc In the name resolver when SQLITE_ALLOW_ROWID_IN_INDEX is enabled, if there
are multiple views that might resolve to the "rowid" but only one real table,
then use that one real table and ignore the views.

FossilOrigin-Name: 8fcea4cdfc89dd78eca5e7f62aa31aff0e296f41e79349d3af1cc3a2bc4d77c6
2024-03-19 16:34:32 +00:00
dan f891ef8d61 Update tests to work with SQLITE_ALLOW_ROWID_IN_VIEW.
FossilOrigin-Name: 495b8c7b08b998ddfe2ad055703f058c768e7e94014952e5cd8a81c7c1515cb1
2024-03-19 15:27:15 +00:00
dan a10c2a9055 Return the error "ambiguous column name: rowid", instead of "no such column: rowid", when a "rowid" reference is ambiguous.
FossilOrigin-Name: 021f34fcfed41b607be8169bbda59aef93f130108d944f4741b46e8e345b2bbb
2024-03-19 14:55:49 +00:00
drh 967c303212 Fix the fuzzinvariants.c test module so that it does not generate
invariants that depend on rowid when SQLITE_ALLOW_ROWID_IN_VIEW is used,
since rowid values can be unreliable in that context.

FossilOrigin-Name: 0d711aaaa14472a82f8893e1b07d845073f3d018bd5b8093ec0b00ffbbc8cd5a
2024-03-19 14:38:39 +00:00
drh d75bdf51c7 Improvements to the first check-in on this branch, causing less collateral
damage.

FossilOrigin-Name: 7bc882897b6910f93695ad372445df7791b096412089ccbcd4fde365c62fa074
2024-03-19 14:21:26 +00:00
drh 261c6dfe7b On second thought, change SQLITE_TESTCTRL_ROWID_IN_VIEW into a start-time
option SQLITE_CONFIG_NO_ROWID_IN_VIEW.

FossilOrigin-Name: b8e045c9e1d098d116f8745704b10ed76569d4b063c0b81cce16bc136930755d
2024-03-19 13:55:10 +00:00
drh 4b42b5259f When compiled with SQLITE_ALLOW_ROWID_IN_VIEW, rowid-in-view is on by default
but can now be turned off using SQLITE_TESTCTRL_ROWID_IN_VIEW.  Without the
compile-time option, rowid-in-view is always off.

FossilOrigin-Name: 8a6196ab29052071be753c5c77ac945c2d62ecc8019c6160f954eafe34ab05a8
2024-03-19 13:31:54 +00:00
drh 1fe31dcfab Adjustments to the test cases for the previous check-in to make them
cross-platform Linux/Mac/Windows.  Test chance only.  No change to core code.

FossilOrigin-Name: 0dddadbd5b217a48bb5c1b0568a03fbafb89dba98c577727bf4f79ff7e0ca849
2024-03-19 10:41:18 +00:00
drh 5ed044e893 Fix another problem from [c63e26e705f5e967] involving infinities and the
Kahan-Babushka-Neumaier summation algorithm.  Problem reported by
[forum:/forumpost/23b8688ef4|forum post 23b8688ef4].

FossilOrigin-Name: 45d272ef38d5ee6189f81369aefb8e3ac35868a67687012d93c84f2bbc42520a
2024-03-19 10:16:17 +00:00
drh 108dd6a52d First steps toward getting -DSQLITE_ALLOW_ROWID_IN_VIEW to work again. That
compile-time option is untested, undocumented, and unsupported.  But it was
mentioned in the release notes for version 3.36.0, so I think that means we
need to support it forever.

FossilOrigin-Name: 7c46ff640247584f1ea260c062de29a0e49f649d1894a526574c1a479006a1fc
2024-03-19 02:30:24 +00:00
dan 28cd72ac52 Fix asan problems in sqlite3rbu.c. Enable RBU for sanitizer release tests.
FossilOrigin-Name: ad2ae7717e7a10cf42aeabd3c544e7884fa082295c2babf0fdb19a684d2aba1e
2024-03-18 21:08:33 +00:00
dan 365b6e9769 Avoid spurious SQLITE_IOERR_DATA errors when reading partial pages from a cksumvfs database using a SQLITE_DIRECT_OVERFLOW_READ build.
FossilOrigin-Name: 0f485dbece9146eabe5a5c0998a9e35c685a170c3fe6d4509e92aa8dae18a2c0
2024-03-18 20:52:45 +00:00
drh 8c6bc81290 Merge the latest trunk enhancements into the bedrock branch.
FossilOrigin-Name: 10ab5e85700945d3e87dab15e39189537ae87d138ece0045050b8b7d02662ef1
2024-03-18 18:55:47 +00:00
drh 4891bf00db Merge the latest trunk enhancements into the wal2 branch.
FossilOrigin-Name: c0af1242d86a85cf32d9fbaa614a097e77142e7ba89c3e3c1be74c0e9768126b
2024-03-18 18:51:49 +00:00
drh fc2d862318 Remove unnecessary blank lines from build commands in the default Makefile.in.
FossilOrigin-Name: 76fb3a908f45230b956cb659c754e47603e80aa72a2aad50d479437a9c013f61
2024-03-18 18:03:17 +00:00
drh 43a511927d Add the "VALUES-as-coroutine" optimization. Large VALUES clauses on an
INSERT, for example, prepare and run in about half the time and with half
the memory.  This check-in also includes enhancements to the internal
sqlite3ExprIsConstant() routine to recognize pure SQL functions as constant
if they have constant arguments.

FossilOrigin-Name: a120c9235f125e05be494038c16a9dd326fd79837698bef17e7879cd0cd75831
2024-03-18 18:00:17 +00:00
drh 2ad2584581 Fix harmless compiler (scan-build) warnings.
FossilOrigin-Name: c86f9f2a15ffc726b7f0d9bba5a8c4dfdaeea6a297e0b591c554fff3d1fe6e1c
2024-03-18 17:13:52 +00:00
drh c195e2374b Allow the VALUES-as-coroutine optimization to be applied to later rows of
a VALUES clause even if earlier rows do not qualify.

FossilOrigin-Name: 9a47ea7f0f675f7bf4710901487ce34c7689e618cd1d8b9f94f0ff7ebc3f2841
2024-03-18 16:30:00 +00:00
drh b03805b70b Change a test case to use a window function rather than random() to disable
the VALUES clause optimization.

FossilOrigin-Name: c589149a138fa0a6be3153986fc0f315e052e13ebc0e7006b6529ca02bae413b
2024-03-18 15:10:00 +00:00
drh ac7c6f5854 Add assert() statements to validate access to the SrcItem.u1.nRow union member.
FossilOrigin-Name: 21f616d9b948efca441f8d45d0a95f4c052ce8b6daec7fa582ad9a00b82ca570
2024-03-18 13:31:24 +00:00
drh 871786b792 Fix the patch at [10ee6fcba08ab128] so that it works on reduced-size Expr
nodes.

FossilOrigin-Name: 260bd764c3f2d6d067adb9cd0045b7c24d5e00b02ab2735b1cba455a6143ff4c
2024-03-18 13:10:54 +00:00
drh 27a5ee855d Improvements to EXPLAIN QUERY PLAN output for multi-row VALUES claues.
FossilOrigin-Name: ac6f095e13e43d66c06552c8b01f6bec3407c9d41a34c4cdb0be57b0b828ad0d
2024-03-18 12:49:30 +00:00
dan 75924d3ae2 Ensure the database schema has been loaded and the database encoding gleaned before beginning to code a multi-row VALUES clause.
FossilOrigin-Name: 2ff476eb3d1f4a2146f4a48b57895a00a3ff5beb29afa679ae53ea58cac07c76
2024-03-18 11:12:22 +00:00
dan 2f4a8cc2d8 Fix a problem caused by a non-aggregate function with an OVER clause in a multi-row VALUES clause.
FossilOrigin-Name: 10ee6fcba08ab1281235197602148fe062560e1d5034a477b8e7b574dd3e2907
2024-03-18 10:54:48 +00:00
drh 54bf109e80 Omit the "noop(X)" test SQL function. Accomplish the same thing using
the idiom:  "coalesce(X,random())".

FossilOrigin-Name: 0eb2fbb8910e7a372c50db3ae44238d7b161f0e45858b74061b5228aec5fcc7e
2024-03-17 16:01:23 +00:00
drh e4a1b4da4f Add an ALWAYS on an unreachable branch.
FossilOrigin-Name: 0dce6211690210e802545cbca9b3375e07075e117fad3f8c921879da7671d981
2024-03-17 00:13:12 +00:00
drh 1c5257dafc Recognize when an SQL function is constant and can occur inside an
optimized VALUES clause.

FossilOrigin-Name: c0e5ceb87d3cf28f22d381f863b8f91043804fd7901f23f3afb02fe698e6a12a
2024-03-16 21:12:51 +00:00
drh ec00a4d111 A formerly impossible case has now become possible because of the new
ExprIsConstant enhancements.  So deal with that.

FossilOrigin-Name: 9ba3cff9d48fe8fc8a0a51291169599209d464465e44bdfd60058fd1e314a2b1
2024-03-16 21:04:32 +00:00
drh 7dc3e923c4 Further expand the scope of usage for the enhacement to expr-is-constant.
FossilOrigin-Name: d7eadcf7dd089f0f4aa963c58f0df32edd951368c9906375fd62b0a61cd393b8
2024-03-16 20:15:34 +00:00
drh 4713b2d73c Assert that the eCode is always non-zero upon entry into exprNodeIsConstant().
FossilOrigin-Name: f71b28f6713126745bca53e098dfe3444d4235d6a2b87ced5b333a7f0ee79be8
2024-03-16 19:48:51 +00:00
drh b918a0f821 Activate the enhanced constant-expression for the
[constant propagation optimization](/info/f4229707ac08d66c).

FossilOrigin-Name: 92afea38881a1a4f266c79ddc6e1027cf17c288dcc8f7e7d50466899ac0054c4
2024-03-16 19:36:18 +00:00
drh bf4105d768 Extend the scope expr-is-constant for function to cover the IN-to-EQ
optimization.

FossilOrigin-Name: dae669245c86021bcd28716aff6e316257cc8075d02081745d4b7de17f8ad553
2024-03-16 17:04:08 +00:00
dan 40514dd1f6 Fix exprNodeIsConstantFunction() so that it returns WRC_Continue, not WRC_Abort, if the function really is constant.
FossilOrigin-Name: d85dd4de2d0989127e0ae6a2eec9d83a577777f8d5ba40700084a0b498016634
2024-03-16 14:27:10 +00:00
drh f696591dd4 Enhance the sqlite3ExprIsConstant() function so that it recognizes
constant functions.  So far the enhancement only applies to the multi-row
VALUES clause, but it could possibly be applied in many other places.

FossilOrigin-Name: c9e0488c6c0135932b6e76b0f3f3acd69ef65327e0a54daa59777f35da1aef26
2024-03-16 13:18:48 +00:00
drh d31fc6e9cf New options for testrunner.tcl: --stop-on-error and --stop-on-coredump.
FossilOrigin-Name: 82035b9cfd28ef6b0ecc6f469f03d2b001189aa4925147cdb784b6b1964eb3b2
2024-03-15 17:57:58 +00:00
drh 825596481f Fix (totally harmless) memory leaks in Lemon to avoid warnings during ASAN
builds.

FossilOrigin-Name: ce009205a8edc02b7d45ac01bd0e692c3d2c3ffeadb68e4f1bad20c39075e692
2024-03-15 17:04:55 +00:00
drh 0ebd1c382c Tweaks to the func4.test test module (which tests the rarely used totype.c
extension) so that it works on i586 compiled using gcc-13 with -O0.  Details
at [forum:/forumpost/0a7553b0f734c033|forum post 0a7553b0f734c033].

FossilOrigin-Name: d0fbe779bc2460e120da4f39063a9f4121c700f0b86d85f4311715b1366c3515
2024-03-15 13:16:13 +00:00
drh 932b5f4575 Fix testcase to account for new behaviors with this branch.
FossilOrigin-Name: 823e579362c05bb8accf6c3b158c5162a16eb23cf81d6021c9e3246e32583d1c
2024-03-14 20:39:03 +00:00
dan 26a3ef7557 Fix a problem with ALTER TABLE and correlated multi-row VALUES clauses.
FossilOrigin-Name: d543c829ef74dbd64105bd757ca660e4f02e9ce562be4f1688a701fa535351c4
2024-03-14 19:31:06 +00:00
dan 400992b1c4 Fix a problem handling "INSERT INTO ... SELECT ... UNION VALUES(...), (...)" and similar statements.
FossilOrigin-Name: e8a2a8198a97046ff376bc5d38e4bc0a24fcac79f5a0dadb9d29d953a862a012
2024-03-14 19:01:17 +00:00
dan f0f4323824 Avoid some OP_SCopy instructions in "INSERT INTO .. VALUES" statements that insert more than one row in cases where the VALUES clause contains a value for all columns of the table.
FossilOrigin-Name: 988f0ea70cd21b3194011e0cd1ddd4990a07669e0fd9c37afa48f69698ab7212
2024-03-14 17:04:18 +00:00
stephan 5a0fd81fce JS: remove an unused function argument and some debug-related comments. No functional changes.
FossilOrigin-Name: f15709430d4ba2fcf5729bc30a45dacb77102927b82cda12de0b67935b526cc0
2024-03-14 13:18:19 +00:00
dan a06c046e29 Add further tests for VALUES clauses in various contexts.
FossilOrigin-Name: 973d197d9676c066a95cea77a27e36d2d7f5778283919d8cac4424d2ec952f47
2024-03-14 11:45:39 +00:00
dan 56be6f6c08 Change the way parse.y handles multi-row VALUES clauses to save a few cycles.
FossilOrigin-Name: 88d5bc91a52675d2a83748b79ad50de0e9732b2afd23107627bdc604f744d275
2024-03-13 20:04:11 +00:00
dan aa2e244e14 Remove unreachable code from this branch.
FossilOrigin-Name: 657c7b4f9df53e041a33cfad00ee2f507ac09c97a2ae61164bd189d660d167da
2024-03-13 18:41:05 +00:00
drh 1d4bf383d7 Merge the latest trunk enhancements into the bedrock branch.
FossilOrigin-Name: 5c8e80f435dc24d5af9f217f9034215bf0b39fa5888eda9f4416da832da2735f
2024-03-13 18:00:23 +00:00
drh 467d804e1a Merge the latest trunk enhancements into the wal2 branch.
FossilOrigin-Name: 2b12da723400a9191c21108e7eaa76e61f373267380c46813031b66b61446067
2024-03-13 17:55:43 +00:00
dan 35057fb4bc Fix some OOM handling problems on this branch.
FossilOrigin-Name: 2085c7f12a3916ec883c31795e29f2e2b6641c30ecf748cce9bff7b13b061d1f
2024-03-13 17:33:45 +00:00
drh 81a244c514 Minor change to the "mdevtest" and "sdevtest" in testrunner.tcl, to make it
easier to add configurations in branches.

FossilOrigin-Name: 9349d94e05dcf266b02afcd89f1e433cdc45f23e8536b2f3f9aa242d5d89c307
2024-03-13 16:32:05 +00:00
dan 693d3867ea Merge trunk into this branch.
FossilOrigin-Name: 38635651f08d42110c01d6b24f6e362e340511846294f8696af4afc795dae80d
2024-03-13 15:47:52 +00:00
dan e116fa153f Ensure that if a multi-row VALUES cannot be coded while being parsed, the SF_MultiValue flag is correctly set.
FossilOrigin-Name: 94791824a6f04782aac6bc16c182685e2d6ebf5c688dc9f8d59e479c6fd5a40e
2024-03-13 15:44:31 +00:00
dan 609aba00d9 Avoid creating a co-routine incrementally for a VALUES clause that has affinities other than NONE.
FossilOrigin-Name: 4229b12b327b05561dcf49b8585a66467d17d4e998b14eff65eb886f2434b53c
2024-03-13 15:34:44 +00:00
drh 6bb6a9c941 If the NJOB environment variable is a positive integer, it causes
testrunner.tcl to run that many jobs.

FossilOrigin-Name: 924281b94d8e6ba674d6fe2f7f01da890351355a854e9e6fe623fad1180f7392
2024-03-13 13:43:21 +00:00
drh 4565c8417b Document the testrunner.tcl --explain option in doc/testrunner.md.
FossilOrigin-Name: 5a7b5fc99a9d5c7693a5707c2050a8c59a8f97732e4763a56982579f8d5959ec
2024-03-13 13:27:16 +00:00
drh 4659ec34c1 Add the --explain option to testrunner.tcl.
FossilOrigin-Name: f0c9eb0326f189fe5ebbee83532ac2c413e91c6a8e9d5b93aa6170cba66fdd22
2024-03-13 13:09:58 +00:00
drh 9b6e535609 Improve one of the error messages from testrunner.tcl.
FossilOrigin-Name: 233155c174575fa99b34030c1936bc2b04489417f4d5223f18cdb3c8fc274a56
2024-03-13 11:46:23 +00:00
drh c2764dbb33 Minor tweaks to the testrunner documentation.
FossilOrigin-Name: 3128b98d2656c7f23f6071c173f96329c7bdc49cd641050fbb7b105b810ba7c7
2024-03-13 11:29:04 +00:00
drh 484193fe7f Improvements to the "help" message generated by testrunner.tcl. Add the
special "list" permutation that does nothing but list all allowed
values for the PERMUTATION argument to testrunner.tcl.

FossilOrigin-Name: 1f5bfc6a9030f9905d258c5b665b9ff27efc7da71556c912ab3dce0f8f5aa8da
2024-03-13 10:55:47 +00:00
dan 1769640c8b Add pointer to documentation to testrunner.tcl help message.
FossilOrigin-Name: eff855102f9c0b4e9d876d1cff9d06c07de427b4880c2fa326d4d6e3e1dd13c8
2024-03-13 10:54:04 +00:00
dan 3de08a6ee0 Have testrunner.tcl set SQLITE_TMPDIR on windows as well.
FossilOrigin-Name: 64f4dad1f928716b8688c650d043f18afcc72ec78d990145c580e2b98905b919
2024-03-13 10:47:54 +00:00
drh eea6bdce84 On testrunner.tcl: (1) Add the "help" command. (2) Add the "script" command
to the help message.  (3) Improve the error message generated by "script" when
an incorrect CONFIG option is provided.

FossilOrigin-Name: 1b7f0be44036fb90d763eabae84b95734e766f8010b39122f8787189308a7fc1
2024-03-13 00:37:37 +00:00
dan 5badd048d2 Correctly initialize the SrcItem.iCursor field to -1 when creating a co-routine from a multi-values VALUES clause.
FossilOrigin-Name: c32953ff6d72910815eaff29ab1b790c412e1a741d02c03c5911dda0c08e5130
2024-03-12 20:10:46 +00:00
dan f99ab38bdb Have testrunner.tcl have each test store its temp files in its working directory. To avoid unlikely, but possible, collisions.
FossilOrigin-Name: 18842d0d72048561312c2c83035c01253150e1fe1ff37160f266d24d41a01df3
2024-03-12 18:33:57 +00:00
dan 3ac5c08180 Have testrunner.tcl have each test store its temp files in its working directory. To avoid unlikely, but possible, collisions.
FossilOrigin-Name: 775a10d10b9bf2227071e778718ba4054d4b2b4238751eb147338d687b4e0d58
2024-03-12 18:04:40 +00:00
dan 38152d9d04 Fix another testrunner.tcl problem on windows.
FossilOrigin-Name: 7805844bc4433ae088dae6d54eb972e082c5a3df06e844cfc8b69e7d97aeb432
2024-03-11 20:34:02 +00:00
dan f4783d4d7f Fix a missing brace problem in testrunner.tcl introduced by the previous commit.
FossilOrigin-Name: 1f5e334d98d98cd90a2fb53502015a883d96fed05c04c1c3e1e78c302bede50f
2024-03-11 19:43:43 +00:00
dan 2dbf276e93 Update mdevtest and sdevtest to do shell builds in parallel. And only if one or more of the tcl scripts run requires it.
FossilOrigin-Name: 6eae4547d4d50c798d1f05eaa1da02f4682d261dbd64e94748179923839024e6
2024-03-11 19:33:55 +00:00
drh c6481e0289 Run shell*.test modules with "make mdevtest".
FossilOrigin-Name: 76c471061809307c1922a9460f1ea37291ff5f4dc11a71af594351e875fdc8e3
2024-03-11 18:21:02 +00:00
dan 815e055bff Attempt to reduce the memory used by VALUES clauses in as many statements as possible, not just INSERT. This branch still has problems.
FossilOrigin-Name: 17d1f7cfabc7593d0725051b0c7c9619a23a482265f30f15ab9493fef5caeeb0
2024-03-11 17:27:19 +00:00
drh 5dfff386e5 Fix the ".import" command in the CLI so that it works correctly with tables
that contain computed columns.
[forum:/forumpost/ca014d7358|forum post ca014d7358].

FossilOrigin-Name: 95a9c88b258f18ac671a4c712603931167cc8bd0b86e41481b200c08043338b5
2024-03-11 11:24:59 +00:00
stephan 289507d78b Add more docs for the OPFS delete-before-open feature.
FossilOrigin-Name: cb8d9c269c01f78f09069a2361e15087d9aeac63e9f1fd63fbd96b951384e9aa
2024-03-11 09:39:52 +00:00
stephan 88abf9ad44 Add delete-before-open=1 URI flag to the 'opfs' VFS to tell it to xDelete the db file before opening it, primarily to enable users to work around a corrupt db without having to reach into OPFS-specific APIs to remove the db file.
FossilOrigin-Name: e87cf0d7700d33a439c91725460fbfe3a1765b48f71b4d73c38cadf6c166e0bd
2024-03-11 09:34:38 +00:00
stephan bf030d53b1 Add delete-before-open=1 URI flag to the 'opfs' VFS to tell it to xDelete the db file before opening it, primarily to enable users to work around a corrupt db without having to reach into OPFS-specific APIs to remove the db file.
FossilOrigin-Name: e83f9788636f7f9bcca7d2a09620c13ab4eb83436d5b2946a827e48addf0267d
2024-03-11 09:32:38 +00:00
drh 5007833f5f Fix the .expert command in the shell so that it does not leak memory if not
followed by SQL and so that it works with reverse_unordered_selects.

FossilOrigin-Name: 7ead022edaf7a0cd6a8976a1261246084975c9a5be5c893f6c751bb5f963ac0f
2024-03-09 18:41:40 +00:00
drh aa13f4c132 Fuzzcheck is run automatically by testrunner.tcl, so there is no need to
include fuzzcheck as a separate dependency for the releasetest target in
the MSVC makefile.

FossilOrigin-Name: 76629b2bff01df3d42eef2e93d626c291a2f129bd923498941465e5bca74e9f1
2024-03-09 13:31:10 +00:00
drh 61b77a6fe1 The NOT NULL strength reduction optimization from [de9c86c9e4cdb34f] should
be applied to the WHERE clause only.  Otherwise, the operand of the IS NULL
or IS NOT NULL operator might be a reference to a bare column of an
aggregate table, and we can't tell if it is NULL or not based only on its
NOT NULL attribute.  [forum:/forumpost/440f2a2f17|Forum post 440f2a2f17].

FossilOrigin-Name: 51704feae224eff601db5607f8651da11b3c2ed8a58ffe5b6ee8260cab50695b
2024-03-08 21:37:18 +00:00
drh 96f5ae6bd7 One of the assert()s added by the previous check-in was not quite correct.
This commit fixes it.

FossilOrigin-Name: d401358329f5a70f9a0b9b033609a4db2af89b83c6b40242be0c76f3d6474def
2024-03-08 19:03:30 +00:00
drh 744e5672bf Add new assert() statements, which if they had existed three years ago,
would have detected the code generator problem fixed by the previous check-in.

FossilOrigin-Name: 166d1e5d26ef88e995f44182144891f60bd51c1aa585b4a148f01a920b2a8eea
2024-03-08 15:39:59 +00:00
drh 926fb60b05 Silently ignore redundant ON CONFLICT clauses in an UPSERT. Only the first
ON CONFLICT for each index is active.  Do not issue an error, since that might
break legacy queries.  But ignore the redundant ON CONFLICT clauses to prevent
problems such as described in [forum:/forumpost/919c6579c8|forum post 919c6579c8].

FossilOrigin-Name: d0ea6b6ba64dba9d68c2b391ccf1171ea96fcdd7409dafdb2b697accb00246b8
2024-03-08 14:01:48 +00:00
dan b89aa10cd0 Omit some extra tests for the intck extension in OMIT_VIRTUAL_TABLE or OMIT_PRAGMA builds.
FossilOrigin-Name: 29d9eb7d55755604781e507f6ca36c50d62fa8d8589ab932d7cefca94ba24f3e
2024-03-08 13:49:43 +00:00
dan 3041642b1a Fix pragma6.test so that it works with ENABLE_OVERSIZE_CELL_CHECK builds.
FossilOrigin-Name: a02551de4c2d7085569ee76f88bfef5da429e40137cc1d1349b4637f88259ad1
2024-03-08 11:13:20 +00:00
dan a219fb5bb9 Do not run tests for the intck extension in OMIT_VIRTUAL_TABLE or OMIT_PRAGMA builds.
FossilOrigin-Name: de8130c020821a1615cceeb730423a2d6e01578caaaddeb11950363c4f8ea412
2024-03-08 11:08:53 +00:00
drh f10de5360a Must use sqlite3IntFloatCompare() for accurate comparisons between very large
integer and floating point values in RTREE.  Otherwise the comparison does
not work on all platforms.  Further fix to [027e5336acc26f57].

FossilOrigin-Name: 820f106acff5f2cd01da0e95a0e6f2bcc087705bf8c08b730b1fdb08db5679c8
2024-03-08 03:24:09 +00:00
stephan 67fc3e0ac2 Introducing JS worker1 promiser v2, which initializes via Promise (instead of a callback function) and can be loaded as an ESM module.
FossilOrigin-Name: 2fbaf2f51d37f70ee26d45f0c62f32c15a9e03f68b6d2e2892115e7dc028b929
2024-03-07 19:29:53 +00:00
stephan 84016759dd Resolve duplicate loading of promiser v2 demo code (a side effect of having done precisely what it was told to do). Consolidate demo-worker1-promiser(-esm).html/(m)js variants into central copies processed with c-pp.
FossilOrigin-Name: 9347d9b9a69277e40ea2f3ec6e1ff37ea19d24b4af80c6230b10624173f2f17c
2024-03-07 18:53:27 +00:00
stephan ffa0fa69b6 wasm promiser.v2 is essentially working but the demo code is double-loading the module for as-yet-undetermined reasons.
FossilOrigin-Name: aa877ce0c3b3aa1accd6e5fcd722d1bfaa79bea28c04c6badd8a547cea4bbc63
2024-03-07 17:56:08 +00:00
stephan 61405c463d Initial work at getting sqlite3Worker1Promiser.v2() to return a Promise instead of using an onready() callback, and also creating an ESM build for promiser1 per user request. It seems to work but requires more testing.
FossilOrigin-Name: 0e272123ace55ed63fe86632671cca48e8965a28fc3625324984028729fc203f
2024-03-07 16:04:43 +00:00
drh c395f08099 Fix harmless compiler warnings in test code for the intck extension.
FossilOrigin-Name: 7fbdc1a849af3440579459bbb8797ebc7f9cce7b34d95675b8baa82db194ea9c
2024-03-07 15:58:06 +00:00
drh 5fb718aaab Do not allow the query planner to be tricked into thinking that an index on a
constant expression might be useful for something.  Problem reported on
[forum:/forumpost/ecdfc02339|forum post ecdfc02339].  This is a follow-up
to the fixes at [44200596aa943963] and [2d2b91cc0f6fed8c].

FossilOrigin-Name: 720ce06d93a9e4cc25c34c873c82165d8801f208c22701e51538f3210de84f65
2024-03-07 12:34:26 +00:00
drh 2b6cce16ff Add the json_pretty() SQL function.
FossilOrigin-Name: ceb51c1cc36dfb78db8f2be042a0d74ab9a877d14a53dadf876d09e1a0616d6a
2024-03-06 20:49:05 +00:00
drh 1b977e3ae8 Add test cases for json_pretty().
FossilOrigin-Name: 6448b90708eeedef03e82dcb10d2879e1bc859d422b450c5fc403ffbe0343bed
2024-03-06 20:38:52 +00:00
mistachkin 1bd583e201 Work around obscure floating point issue seen with older versions of MSVC.
FossilOrigin-Name: 42d39f9140cc315df04b048f3811311fa50b51e1831d6cdf982fbce029778e43
2024-03-06 20:24:02 +00:00
stephan 31c3ac9049 Add a couple of json_pretty() examples to /fiddle.
FossilOrigin-Name: d5e1687b1d49cec5daa8793ea138fdf3bd29436c1e67071707ec0594bd1c66c6
2024-03-06 14:42:06 +00:00
drh b4e7d59f4f Add the json_pretty(J) function for pretty-printing of JSON. An optional
2nd argument is text used for indentation, with a default value being four
spaces.

FossilOrigin-Name: 39552bd36c06fe9ee346cb71e0659baceccde031b67c0974f2dd14eb11ebc055
2024-03-06 14:30:42 +00:00
drh a64342ee9c Correction to the previous check-in.
FossilOrigin-Name: 483fa2969e1e10cd8e8d2f9e3027871c65b1360b6c23897efe3ce63a3a55ae13
2024-03-06 12:28:55 +00:00
dan 53ca213e73 Fix handling of "id=?" corner cases in rtree when the value on the RHS is a real value.
FossilOrigin-Name: 027e5336acc26f57f21df4980928731026c30cf88688fa0b66f13ffa0b5da3a0
2024-03-06 11:35:36 +00:00
dan 7ab40cda7b Remove code that added a P4 parameter to the OP_Variable opcode. This is no longer required.
FossilOrigin-Name: dd5977c9a8a418be3fbd646d74933996381099a9263a02eb4a990f0084463dc1
2024-03-05 18:41:03 +00:00
stephan 75aed69158 Reformulate [34439fe3aeea7cbb] slightly to resolve a false-positive OOM reported in [forum:2eadfe94e3|forum post 2eadfe94e3].
FossilOrigin-Name: 8290646792bc5411112b0c01dc5ac0837743056a7679725ee2edefef2e10d146
2024-03-05 17:33:04 +00:00
drh 28001204f4 The value returned by the json_each.json field needs to survive longer than
the json_each virtual table itself, in the event that the value is used in
an aggregate expression.  dbsqlfuzz 5120bd0b3bb0b73eebbcb79ac37c6b0663fccad6

FossilOrigin-Name: 952ed71b9b61e79fe807ac3134bdfa7c94a415e02f13f94440a34d6d548d171e
2024-03-05 16:47:48 +00:00
stephan 891aa05450 /fiddle: reorder/replace example entries to make them more coherent when tried in order and clear the example selection list when the input area is cleared, both based on forum feedback.
FossilOrigin-Name: b4ce270b3f19499385d0923f0f28f702c8f664476def3994fdba5d418e3c5edb
2024-03-05 12:48:26 +00:00
stephan 7eb0ddf68b /fiddle 'Mandlebrot' typo fix reported in the forum.
FossilOrigin-Name: ceefe6ba380c18279c2ae88443649e00e5e8110b529d447131ea9726de61f5d0
2024-03-05 08:48:40 +00:00
stephan d5045a9246 Update fiddle.make to account for Makefile changes in [178b7d46f9].
FossilOrigin-Name: 7a5d810549f2686ae923fd0ca03ad8f536c9080b6879128ee8f00e653bb7f3f1
2024-03-05 07:55:20 +00:00
stephan 2c3973fdff /fiddle: before resetting a db, roll back any transactions (resolves problem reported in [forum:0b41a25d65|forum post 0b41a25d65]) and remove an obsolete/broken reference to a long-gone API which could cause initialization to fail prematurely.
FossilOrigin-Name: ee164ca73cf4151b1a1bf351729afa9b0ec95bd5004a5d5bfce3ed46268bfbf3
2024-03-05 06:31:37 +00:00
drh 18281494a2 Fix a compiler warning in date.c. Update makefiles to include all necessary
dependencies for building shell.c.

FossilOrigin-Name: 178b7d46f9c8e1694ed6f506d2785a9f5c49e2b428e04c8096d557cc1b5c9680
2024-03-04 18:22:18 +00:00
drh 4cf953731f Change test results in corruptC.test to account for the added detail in the
output of "PRAGMA integrity_check" from [b736519d3d2e93c7].

FossilOrigin-Name: 0d19eeaafb296aad212ee4f55a0f6df8c7c5185b14d7f7f49c020fe4cbe370c1
2024-03-04 16:44:37 +00:00
drh 90170503b4 Fix harmless compiler warnings in the increment integrity_check extension.
FossilOrigin-Name: 596668e6625208ff5d2ffbb4e0530530ebb3a400efa0df54bad1d91953e32694
2024-03-04 16:18:40 +00:00
drh 3f273db39b Adjust date/time functions so that they do a better job of keeping track
of whether the current time is UTC or localtime, and no-op the 'utc' and
'localtime' modifiers accordingly. See
[forum:/info/e7a939e074|forum post e7a939e074].
Also add the datedebug() function, available
only under -DSQLITE_DEBUG, for improved visibility of the DateTime object
during debugging and testing.

FossilOrigin-Name: dc569683748354a6db83438904422e802d3ea780775c48da85b474fff03ca8a1
2024-03-04 13:58:09 +00:00
drh 8d25d07d50 Fix assert() statements in date/time computations: The month and day numbers
can be zero if an error has been seen.

FossilOrigin-Name: fc773f6c76ec114be8b6e25b13885acb5adcc9d052dca0d8d0ff94e2a0743d64
2024-03-04 11:12:15 +00:00
stephan 2775f10ebc Document why the wasm build does not use Emscripten's -sSTRICT=1 flag and account for a behaviour change in emsdk 3.1.55 regarding the HEAPxyz symbols.
FossilOrigin-Name: f6e887203365b30cea6e9c145366345e48256a347388577edf2bada65e0655b7
2024-03-04 09:56:47 +00:00
stephan bae55a7685 When warning about multiple invocations of JS's sqlite3ApiBootstrap(), use the warning logger installed by the first invocation, rather than console.warn.
FossilOrigin-Name: d162af2d9941cd8c53e3928efb42aaaf4e59454f7d39aebf980a30b1427ea75b
2024-03-04 09:23:52 +00:00
stephan e6af60c14f After wasm bootstrapping has read the optional global-scope config objects, delete those objects because any further changes would not have any useful effect. Expand the docs regarding the sqlite3ApiBootstrap() config object and explain the necessary evil of a separate global-scope object for that configuration.
FossilOrigin-Name: e6b14f73136aae4c1a92d6d6ab8c5d1510db5244d0b0f168eed5b4b257c99064
2024-03-04 06:54:41 +00:00
drh aefa7afddb Back out the previous change. Replace it with new date modifiers "ceiling"
and "floor".

FossilOrigin-Name: f0831cced2c919e409214d936c81473ae321a98c5bd78b5b729c1269bf71bc45
2024-03-03 20:15:36 +00:00
drh ebc14e409f New date/time modifiers "mnth" and "yr" work like "month" and "year" but
resolve day-of-month overflow by truncating rather than rolling over into
the next month. [forum:/forumpost/232d1abb5d|Forum thread 232d1abb5d]

FossilOrigin-Name: 5d392c16bb5658bc92f60d250ab82c45cc109e32d58a073498c9a487b8625e96
2024-03-02 21:02:41 +00:00
drh f52afaf738 Change the magic time-interval names that do truncate-to-same-month to be
"mnth" and "yr" - "month" and "year" without the vowels.

FossilOrigin-Name: 296b46c529d79385d7f4db2c789254a992c01ff66fe8ba6d3a3ce4aaac0203fb
2024-03-02 20:50:56 +00:00
drh 6964005847 Proof-of-concept for new time-interval operator "pg-month" and "pg-year" that
use the truncate-to-month algorithm for month overflow instead of the
wrap-to-next-month algorithm that is used by SQLite by default.

FossilOrigin-Name: b606c0965788f56547435c9318b60e32990877e14a9640d887f33f39d1bdc7c3
2024-03-02 13:38:46 +00:00
stephan 91d7bd8ec7 Fix two -Werror=lto-type-mismatch warnings reported in [forum:ef62b57bd5|forum post ef62b57bd5].
FossilOrigin-Name: 803481f25020f3c25941f1e7d1a8071937820dea951e8798198b0b0fa3fb48ce
2024-02-29 10:55:02 +00:00
stephan 087e6c9ada Fix two -Werror=lto-type-mismatch warnings reported in [forum:ef62b57bd5|forum post ef62b57bd5].
FossilOrigin-Name: 29f94610dc8319709b8d3726a60e2a7a33ffbf4998c681f772da26c9701cc0b0
2024-02-29 03:45:18 +00:00
drh 09a71d8d44 Add in a VdbeCoverage() macro needed by STAT4 that should have been part of
check-in [63ef234e88857a65].

FossilOrigin-Name: d51c699ae413215d534f954c129691ef5d95d540dc5d4304cbba576c976d931c
2024-02-28 15:32:32 +00:00
drh 8597eee119 Always convert 32-bit integer literals into EP_IntValue notation, even if
they contain "_" separators.

FossilOrigin-Name: 2dfc427f676255cbe189a26bfec2405d41d31ccc4512c55b31e6e633261d7a23
2024-02-28 01:12:21 +00:00
drh b542933cba Fix an eputf() macro without its argument in the CLI.
FossilOrigin-Name: c2f5e257d2cfb91341f5261ed3924dc4ec2a6f854f3692fb416043f5c654a963
2024-02-27 20:19:32 +00:00
drh 6324cdfdcd Add an assert() to help scan-build recognize that a variable is initialized.
FossilOrigin-Name: 10d26f1ee8e7624cbb957332b4397d2f1fcdaf916acec93077f2ddd33e01f1d8
2024-02-27 19:57:31 +00:00
drh 2aea6083bd Adjust a VdbeCoverage() macro in PRAGMA integrity_check to account for the
fact that arguments to the OP_Eq opcode used are never NULL.

FossilOrigin-Name: 759ccea3f6e21493c0153f884b601988efd59e409b113b28f8cf23f59a74e418
2024-02-27 16:36:40 +00:00
drh 39670a5345 Remove unreachable branches from the PRAGMA integrity_check enhancement of
[b736519d3d2e93c7].

FossilOrigin-Name: 6d02fb233fb86f9907cc374d06a7ed71a638091d6e75abc199001c42eab51e5c
2024-02-27 15:33:54 +00:00
dan 8732fce985 Ensure memory cells are properly freed before they are overwritten by OP_IntegrityCk.
FossilOrigin-Name: e50acaf934f44f8833bc588e0ae1abd59df907b9a42d4cafd5518427a8c7b627
2024-02-27 14:31:31 +00:00
dan f8dc7d20f7 Have "PRAGMA quick_check" compare the number of entries in tables and indexes.
FossilOrigin-Name: b736519d3d2e93c76fa36253143f2664b9dd12e26acba555f0bf9c4d44705499
2024-02-27 11:03:10 +00:00
stephan 5b985fee64 Extend sqlite3.c makefile rule to support EXTRA_SRC=list-of-c-files to append to the generated sqlite3.c, as discussed in/around [forum:ccda88cf6f1754c5|forum post ccda88cf6f1754c5].
FossilOrigin-Name: 61676f1e18a405e4f3b6007488f2e68f3f55b312c107359ae5e52929f0c104c8
2024-02-27 11:02:31 +00:00
dan 1ffaa03adc Allow "_" characters to appear between any two digits in an integer, real or hexadecimal SQL literal.
FossilOrigin-Name: 0e6700f43f133510c8049b2c5a2610cb3be29da7ed4d39b1fa65dc22c4cf529b
2024-02-27 10:52:41 +00:00
stephan 32888c2b30 Bring the extra-src branch up to date with the trunk.
FossilOrigin-Name: 12ff5c5c4162951a29b638a5bc6cffa50e057c5a5e8f5e9c627af5f4ab1e4cdb
2024-02-27 00:58:51 +00:00
drh f79b0bdcbf The quote() SQL function should convert +Inf into 9.0e+999 and -Inf into
-9.0e+999.  See [forum:/forumpost/6675b25108|forum post 6675b25108].

FossilOrigin-Name: 85dd79a6edecfc8c6307c6d215998f76dab086aa14528ddc64eb9955501becfd
2024-02-26 22:28:21 +00:00
drh 7c6433cfff When inserting a 3-byte cell into a btree, ensure that the extra padding byte
is 0x00.  This is not necessary for security, as far as I can tell, but it
seems like a reasonable precaution.

FossilOrigin-Name: 5766f1279dab91e030d4dcf5133659e5cedf914a1628ccf00d67d8e50a9957fd
2024-02-26 15:27:33 +00:00
dan 09e1900cdc Add the intck extension. For performing incremental integrity-check on a database.
FossilOrigin-Name: 141d8bb059f8987d05d18327b97c4d36e98258f657b41a3d4d8877fe8b4e72e3
2024-02-26 12:16:02 +00:00
drh 1103eb423e Fix header comments on test scripts.
FossilOrigin-Name: 9fe9670c977ce7ad2c19657783a63998769dbb7741a587889c72ecc04d895793
2024-02-26 11:51:21 +00:00
drh f286ad9211 Remove a local variable from sqlite3IntFloatCompare() that was being optimized
out anyhow, in order to get back to 100% MC/DC.

FossilOrigin-Name: 52b13d6acbb738b9281f7dd44edd6c3c9585d22d98b1951b7059534fbd16fac0
2024-02-26 11:43:44 +00:00
dan afeec79847 Merge latest trunk changes into this branch.
FossilOrigin-Name: b6371ff9f5c3d4e87a5b1127a82970202f74c790b828dda20df88dd727bcc9b4
2024-02-26 10:56:54 +00:00
drh dfd991c3b9 Make sure key comprisons are done correctly if the index key contains NaN
values that have not been shifted into NULLs.  That can only happen due to
database corruption, but we need to deal with it nevertheless.

FossilOrigin-Name: 7e4c743f9e6ef33500795543e6db9a77c533025bf00c2ee97abd433a3871b5a1
2024-02-25 21:30:33 +00:00
drh 7598e7d70e Add two GIFs that contain the feather logo.
FossilOrigin-Name: 5f21e6a8ee709d8c5ec12c45e8dc702a7d0bb041f31b5715e28cd904dfd566b0
2024-02-24 19:58:15 +00:00
dan 9c59c87448 Add further tests for the intck module.
FossilOrigin-Name: c253e276b29de28a18270d01b60d95157ce3fc4b37e246d991f9119d26e718d7
2024-02-24 16:26:15 +00:00
dan ee299cc7d4 Add tests for the new code on this branch.
FossilOrigin-Name: 351d46b2373f08bc8033d0902d9f67cd6c8bcc16c0d9f787e4fb279c0a76da87
2024-02-23 20:51:06 +00:00
dan 645f21f15b Add the ".intck ?STEPS_PER_UNLOCK?" command to the shell tool.
FossilOrigin-Name: cfd051836b72f7d4e38cc9614f6ae5c003de4ce377359fd391adf06fe1ddf6b9
2024-02-23 18:21:51 +00:00
stephan b088e892a0 Typo fixes [forum:c19bbf245d|reported in the forum]. No code changes.
FossilOrigin-Name: 67d8bae0f695c0839e724c1aadea0a22f736fdd9664d40a29b4d82c91fb89ab3
2024-02-23 18:02:31 +00:00
dan 5956d1d942 Fix detection of surplus index entries when all indexed fields are NULL.
FossilOrigin-Name: 5f310fb7be450c96f01c21e6fd1377d4a274784377d0bd811922ab63e612bd11
2024-02-23 17:10:39 +00:00
dan 1545243368 Fix problems with resuming integrity-check operations on indexes with mixed ASC and DESC columns, and on indexes that contain NULL values.
FossilOrigin-Name: 0f68b35a000ef9f4691c59797c66ed6c3435fc5c503e9d24f891afec6aceeada
2024-02-23 15:13:53 +00:00
mistachkin f7ad684c5a Fix harmless compiler warning seen with MSVC.
FossilOrigin-Name: e5db08993127dfc18cc2b621a13cbfef52974f0f504f4d8208093bfad1ffdcc5
2024-02-22 18:15:08 +00:00
drh 0b8c3126f4 Fix debugging code so that it compiles on MSVC.
FossilOrigin-Name: ce5df19dc4aff3fde03ef62261a5e095a16a8f4e46e2924becea4fed56ce49e3
2024-02-22 00:50:54 +00:00
dan 639db50a02 Fix various issues in sqlite3intck.c.
FossilOrigin-Name: 8a7bfa74525a495f45b1ea212b1718633b637295090d514dd777f9263477d514
2024-02-21 20:58:48 +00:00
drh 9591a9f069 Fix typo in comment. [https://fossil-scm.org/forum/forumpost/cc20266bf2|Fossil forum post cc20266bf2].
FossilOrigin-Name: 7e3c46eed1290964a3e25662686c6ba1149bf8687d05719b684772e65d660c09
2024-02-21 20:21:46 +00:00
dan 6a00f707e8 Simplify the way the restart key is saved internally by the intck extension.
FossilOrigin-Name: 0e39962baae8a82a3021077676b792ac30c79426bcd8c075b5e92bee55e8c3a5
2024-02-21 19:31:00 +00:00
dan b783f56c58 Add documentation to ext/intck/sqlite3intck.h.
FossilOrigin-Name: 4cc19bd74f05fe92658cc392a1d1afa173d93181a77303af6bc5684436ae832e
2024-02-21 19:17:45 +00:00
dan b2897c221d Ensure intck tests are run by testrunner.tcl.
FossilOrigin-Name: 11d6816c060b6edb9cd61f29297ab95e75e2b46f29c0a796820d94fc13586f6d
2024-02-21 16:15:50 +00:00
dan ebd99d5708 Merge trunk changes into this branch.
FossilOrigin-Name: 63e8846ac1dc1cf1f7071c4634ccbfec3c13560db6afec376cd91515b62430d3
2024-02-21 16:12:23 +00:00
dan eb715f022f Have the intck extension better handle corruption at the b-tree layer.
FossilOrigin-Name: ecd775d108f77d39a1303316c1e0f0b0ae3ffc5218222e1ebfe2ef6783829b85
2024-02-20 20:18:02 +00:00
dan cfcb3b9208 Use fewer cycles to generate the "next key" value used by sqlite3_intck_suspend() function in the intck extension.
FossilOrigin-Name: 95f01426f948cf435d0b400dc7ae06fa699eee32cff498fe77e74a1257a4e09b
2024-02-20 18:17:06 +00:00
dan 626d619218 Consider using "=" and IS operators with even low-quality indexes in cases where they are selected explicitly using an INDEXED BY clause.
FossilOrigin-Name: 43cbbea82132db2d0ddb4f34cc2b6910b3a1243ae6d4e837b1b27bfe91b84834
2024-02-20 16:04:27 +00:00
drh 21eda699d7 Fixes to comments associated with the recent PRAGMA optimize enhancements.
No changes to code.

FossilOrigin-Name: 27a2113d78b35e324e9aedda7403c96c56ad0bed8c6b139fc5a179e8800b9109
2024-02-20 15:38:36 +00:00
drh b468e68322 Remove an unreachable branch from printf(). Change it into an assert().
FossilOrigin-Name: 4c9886b692706723f4b68f58dfef07ab19b192fd19b953917f23b55655af3e02
2024-02-20 13:11:27 +00:00
drh 4b2eeb289b Remove an unused variable from the merge in the previous check-in.
FossilOrigin-Name: 5c26bf379e18e28d800525bd03e2d5597e2f37674979302f351b99abfc064749
2024-02-20 13:10:46 +00:00
drh 68d92c4ad3 Enhancements to PRAGMA optimize and ANALYZE. Add the 0x10000 flag to
PRAGMA optimize.  ANALYZE now records zero-size partial indexes in the
sqlite_stat1 table.  PRAGMA optimize looks for both growth and shrinkage
in table sizes, and uses tighter bounds (10x rather than 25x) to trigger
a re-analyze.  PRAGMA optimize automatically uses are reasonable
analysis_limit to prevent excessive runtimes.

FossilOrigin-Name: 63ef234e88857a653fa3541e80d59802ceccb806ac8296e8bae79a385b7086f7
2024-02-20 12:48:00 +00:00
drh 4189c44cff Another simplification of the PRAGMA optimize logic for improved coverage.
FossilOrigin-Name: 6c5a0c85454e3c658e51fab611c169c034447174022eebc52fd8619b528a4765
2024-02-20 12:14:07 +00:00
drh ae71fa5d59 Minor fixes.
FossilOrigin-Name: ed5afebc410e3d430e2428d35f027bffb218e5f11036a42f97b080153e89016e
2024-02-19 23:58:26 +00:00
drh a1507a3fa0 Fix a minor hiccup in the computation of the number of btrees to be scanned.
FossilOrigin-Name: 91302d9b2720d6b5184e1c24db55d351544adfde4b3d2ddcdc1561f79f11985a
2024-02-19 20:47:59 +00:00
dan ebea8458e2 Use more efficient SQL to verify that indexes contain entries that match their tables.
FossilOrigin-Name: c01e008c28895e50b14531b2a1f3f1110aab3b54df41ebdbd416fbac7b1bba94
2024-02-19 20:15:44 +00:00
drh 9a28311b6a If there is no mention of a table in sqlite_stat1, use OP_Rewind to see if
the table is empty prior to invoking ANALYZE.

FossilOrigin-Name: 0cc93b19de597866292e0696f89fbd0e22a6cbb374678f5cc096fb889210fe3c
2024-02-19 20:12:30 +00:00
drh 74b0aad09f In PRAGMA optimize, scale back the analysis_limit when many different tables
and indexes must be scanned.

FossilOrigin-Name: 636615358aebfef80a657d09f15f608df9e64882b2aaa5574ce31a4f1c71fd70
2024-02-19 19:56:40 +00:00
drh f40da9226d Merge trunk fixes into the better-optimize branch.
FossilOrigin-Name: 2cf78a5b5fce7f2d49ce185d27f50722c91a1d6c91bbcecebd4bb5244a76769f
2024-02-19 18:55:32 +00:00
dan cfc62ceefd Add implementation of sqlite3_intck_suspend().
FossilOrigin-Name: c36ada868da74e030ff5002de1f3b31b639b0c43714b91c2e5ca0eda16bb6bc2
2024-02-19 18:03:53 +00:00
drh e7bdb2172c If a table has one or more rows and it has a partial index has zero rows,
still make an entry in the sqlite_stat1 table for the partial index, so that
we know that "PRAGMA optimize" does not need to redo the whole table.

FossilOrigin-Name: e147b18991dd462fff367442acb0504fdf193a31843ed34ec8c1ced30747bf8a
2024-02-19 16:22:58 +00:00
drh 6c6356f7f2 Simplifications to PRAGMA optimize to make it easier to use. It always
tries to ANALYZE unanalyzed indexes.  The 0x10000 flag just makes it check
for size changes in all tables.

FossilOrigin-Name: 44ed7f4cd07a88a2fdd303a2c78e6babe01d7344b399bd2b80ed68d75a77aaa2
2024-02-19 13:50:09 +00:00
drh 9f34a05e98 Change the 0x20000 bit (use analysis limit) to 0x10, meaning that this feature
is on by default.  The default analysis limit is changed to 2000 which is
almost always sufficient for accurate analysis results.

FossilOrigin-Name: 4abd47b5917099a2f74e53e12c987da0722304a5e9a93b6d43015c1f45c48444
2024-02-19 13:06:27 +00:00
stephan e0a9935be1 Extend the strerror_r() result type discrepancy check in os_unix.c to include the Android NDK, as reported in [forum post 3f13857fa4062301|forum:3f13857fa4062301]. The NDK's strerror_r() signature is version-dependent, whereas this change assumes int return (POSIX semantic) across all versions.
FossilOrigin-Name: f18b2524da6bbbcf372b292df52fbe6efa49fd6e1f1f13ef2447279d559f7a08
2024-02-19 12:20:46 +00:00
drh d43e7adca7 Hold a transaction during PRAGMA optimize, for performance.
FossilOrigin-Name: d13b79eae6df7f9d1f3b8062ddc75a12ff038196b3d752d2672a9925fa45ca56
2024-02-18 01:12:22 +00:00
dan 99a94a124c Add start of extension for incremental integrity-checks to ext/intck/.
FossilOrigin-Name: 444e3c9210026da7eae1ed98850722e002433aa2cc77dbc6b6f80327a6b7a390
2024-02-17 20:55:01 +00:00
drh 42eb6a91cd Add new MASK bits to PRAGMA optimize: 0x70000.
FossilOrigin-Name: dd4497062569eec9ddfdaa7d6394c83ff40a7a59c6697a161bc4ff6d8af3bb29
2024-02-17 16:39:52 +00:00
drh 6161cdd446 Fix rounding in zero-precision %f and %g printf conversions.
[forum:/info/393708f4a8|Forum post 393708f4a8].  This bug was
introduced by check-in [32befb224b254639] and first appeared in version 3.43.0.

FossilOrigin-Name: 7fca1bc482fc2456d75392eb42f768fda72631c9070de46b8123b1126e78306f
2024-02-17 03:32:31 +00:00
drh 837efb4cbe The PRAGMA optimize command invokes ANALYZE if a table shrinks by 25 times
in addition to if it grows by 25 times.

FossilOrigin-Name: 40532ffba91cf332c1ea4add80184031a9d6e10514d2d9b9a6cfd613091b81f7
2024-02-17 01:12:58 +00:00
drh 4dd59fd997 Fix the subtype on the value column from json_each/json_tree for cases
when the value is an array or object.  Fix for the bug reported by
[forum:/forumpost/ecb94cd210|forum post ecb94cd210].

FossilOrigin-Name: 1c33c5db2e05019d1a375109f79ad8588a3c17f81e4f4b8d66c880c3c860e87e
2024-02-16 21:30:08 +00:00
larrybr fd2500f98b Retweak last check-in to retain compile-time checking of xprintf() arguments.
FossilOrigin-Name: 670174916c660b24ba70e96a42984eb65ee52da50e9828bdeca4c9ff4bf92e20
2024-02-16 18:34:21 +00:00
stephan 816f677c1c Work around a __VA_ARGS__ complaint in fiddle builds.
FossilOrigin-Name: 7d750248c58ccbf87036ea7db053def1c83c7a8ed428a5c47895756302b36200
2024-02-16 16:04:31 +00:00
drh 9d00aba895 Rename the TF_StatsUsed flag to TF_MaybeReanalyze, to more accurately
reflect its usage.  No logic changes.

FossilOrigin-Name: b4790da5e0d6f0f617e6c3a17fc2606842e41cab14339392abd335410fec0aa3
2024-02-16 12:57:04 +00:00
dan 5e5831a760 Handle an OOM case in fts5 code to query tokendata=1 tables.
FossilOrigin-Name: 01222d96b9ab24c7013a2587544304ba32c1f039b08c06a04d8d58ec9c790db0
2024-02-16 11:41:47 +00:00
dan 5f11f50340 Fix a problem with the sqlite3_log() message identifying the table or index that a conflicting page belongs to.
FossilOrigin-Name: 11a4178f0e6cd760303a5874b5a3529ccd78a2398b59700e878c7e6b1251de91
2024-02-16 11:05:35 +00:00
drh 95b5295f52 Fix a bug in PRAGMA integrity_check introduced by enhancement (8a) in
[https://sqlite.org/releaselog/3_42_0.html|release 3.42.0] and first reported by
[forum:/forumpost/ee4f6fa5ab|forum post ee4f6fa5ab].

FossilOrigin-Name: 460353dfff8f2fb03f9c8666d0c367ec7be4cfa96dfcb0cea10f144d043223bb
2024-02-13 18:41:46 +00:00
drh 54eb54c7de Changes to vtabCallConstructor() to avoid a harmless static analyzer warning
and to help prove that the UAF reported by
[forum:/forumpost/cafbe582e8|forum post cafbe582e8] is a false-positive.

FossilOrigin-Name: 4892440b93306e5a245f18c0d3d295d851e6712260e420016c0d70e12abf8901
2024-02-13 17:11:19 +00:00
drh 8e821c251d Replace the assert() that was removed by [f023cb541b5dd72c] because
[1cd5d4623f44af25] made it true again.

FossilOrigin-Name: 189a49f165dfc348a8730ad2b2288b9b2b2089b1de99fa2d515f2a90c9e7bd4e
2024-02-12 19:12:30 +00:00
dan 98a8ffb3b3 Have rtree avoid keeping a blob handle open following an error.
FossilOrigin-Name: 1cd5d4623f44af25ab4f5af580354b3ae22cf5de4109e008ca41055125bb708d
2024-02-12 16:12:25 +00:00
drh 38e0492b91 Add a new sqlite3FaultSim() to btree for better fault analysis.
FossilOrigin-Name: 2c675bd30568d3c530300beb3fcfaa17c9f063cdcb9a39ac68b9902100f641a6
2024-02-12 14:44:35 +00:00
drh 84dea8264b Add a new sqlite3FaultSim() call in the btree logic, for testing.
FossilOrigin-Name: e0558f79b8184ea6741fb20b710dab854f5bdb71aa47d5a6701eeb7389540c8c
2024-02-12 13:51:08 +00:00
drh b29047838d Add support for the ".testctrl fault_install" dot-command in the CLI when
launched with the --unsafe-testing option.

FossilOrigin-Name: 5ba7fee1a9b49df82387e355d2ab7de58663d3fa824e50717608b27e0f7ac874
2024-02-12 13:28:10 +00:00
drh 18b6019cdd In the RTREE extension, allow the xBeginTransaction() entry point to be
invoked multiple times without intervening calls to xEndTransaction().

FossilOrigin-Name: f023cb541b5dd72c996f0574210344179217666a2229bc8d3fe057fdbc5c2245
2024-02-11 22:56:12 +00:00
drh 3c5a810646 If a term of an ORDER BY or GROUP BY contains an aggregate function or column,
then it is not an alias that needs to be resolved, so don't try to.  This fixes
a harmless assertion found by dbsqlfuzz.  This yet another problem that
orginated at check-in [6e6b3729e0549de0].

FossilOrigin-Name: d4ec2a5d2297cd9ead0a8768dcf003ea76c74d8d68d88c40f62363f484a4a4d3
2024-02-11 20:53:14 +00:00
drh ece3567aec Improved AggInfo tracing in debug builds. No changes to deliverable code.
FossilOrigin-Name: bb31f9c44794e02a46ed3edbcc96eb48c5870d3f82cb404586ded9a3efa7a859
2024-02-11 18:53:48 +00:00
larrybr 589c407f12 Further clarify sqlite3_exec() callback parameter lifetime.
FossilOrigin-Name: 993a1a843177b19a9cb5598f57d1e05c01e1695f164faaa9ae22810ad4533eba
2024-02-10 03:11:42 +00:00
larrybr 25e172d429 Mention limited lifetime of string arrays passed to sqlite3_exec() callback.
FossilOrigin-Name: 717f67f3189a59842f5c17a25e8db0e24ec4405886fc22ab6eeb3954fba5a964
2024-02-08 01:19:46 +00:00
drh 0f777cd622 Fix an #endif (added in the wrong place by [8f4b1ceafe4a271b]) so that
the build works with SQLITE_OMIT_VIRTUAL_TABLE.

FossilOrigin-Name: 7070924eebce1d089fde911dc0427ab8f3764d57ace24caff2b79954b42d1b51
2024-02-07 20:45:38 +00:00
drh 722bd1fe89 Bring test cases into alignment with the latest enhancements.
FossilOrigin-Name: cd017c28d516399e25b5ee9e8fcd7390ca2be9128c7c41f1007f37bc01a35717
2024-02-07 19:52:03 +00:00
drh 204b41955e Turns out the branch is reachable, so back out the NEVER().
FossilOrigin-Name: 0af36a3223d2dabda887830390a603ab4781baca7e2f1698ec6c5f2cf03faf04
2024-02-07 19:17:44 +00:00
drh 71cdea8f5f Detect and respond to an OOM on the jsonStringTerminate() call of
jsonReturnStringAsBlob() routine.

FossilOrigin-Name: 881f814c09b6896a448a2360b0b43dc99592ea3a1d72ac359770cbb0a2c7d38a
2024-02-07 14:05:38 +00:00
drh 8babc01892 Slight change to OOM handling in JSON to catch a corner case found by
dbsqlfuzz.

FossilOrigin-Name: 6a2b3266c586d12b51e11fe63cda5881be9373207aa16671ab77f547830eb790
2024-02-06 18:33:01 +00:00
stephan 72c7bb3e1a Correct docs about the first OFPS VFS's importDb() method being synchronous (it's not, whereas the second VFS's importDb() is).
FossilOrigin-Name: ae7505fa5e112f39a329befb35eff6ddf46a53f5deb610906550bf67f56fc157
2024-02-06 08:59:59 +00:00
stephan db9cc74c9c Make explicit that using the importDb() methods of the OPFS VFSes has undefined results if the being-imported db is currently open. Doc changes only.
FossilOrigin-Name: f28b9924b401b359e59a73918a34b29cde719164a6b4ac5a9d23da6294b701bd
2024-02-06 08:25:23 +00:00
dan 5e922865aa Return SQLITE_ABORT if the underlying shadow tables change in the middle of an rtree query in such a way as to invalidate an rtree internal priority queue entry. This replaces the SQLITE_ABORT_ROLLBACK mechanism added in [af5c4251].
FossilOrigin-Name: 32f85a5ce8e32506ad0cf309c86589958f38d924b4b3de532bec8bdb8e385fad
2024-02-05 17:54:03 +00:00
dan 8e4d004ba1 Return SQLITE_ABORT if the underlying shadow tables change in the middle of an rtree query in such a way as to invalidate an rtree internal priority queue entry.
FossilOrigin-Name: 478280ef67efed854988ab4f740a38ae1937204c0434ad8da11f1869a12a6d06
2024-02-05 17:35:36 +00:00
stephan 21ded98df5 Extend [d294a23ed6d] to apply to all wasm speedtest1 builds.
FossilOrigin-Name: 26f848e5e0ac34e545d2f27cf33abc46eac13e04ed9cd71084b0f7d47136ff97
2024-02-05 03:56:02 +00:00
stephan 7d3cd38034 wasm: pass the promiser function to the promiser onready() callback to simplify access to it in certain usage patterns.
FossilOrigin-Name: f8a8b9ee2eddf5f875c7c4399e750ccf1941f767560ebc2c88c083560f5aaae0
2024-02-05 03:37:54 +00:00
stephan 0009d40325 wasm: squelch a new (and, in our case, invalid) warning from emcc 3.1.52.
FossilOrigin-Name: d294a23ed6d1d2b567a02552b8a2e4921cd21891ee86e63f8f9945569e3262d3
2024-02-05 02:36:10 +00:00
mistachkin 7718c866e1 Fix a minor test file locking issue on Windows.
FossilOrigin-Name: 6cd70b71df5c0d45412081590316bced302ec4403d565250b56e611882204821
2024-02-04 04:18:29 +00:00
mistachkin 3ab9c021ff Fix harmless compiler warnings seen with MSVC.
FossilOrigin-Name: e52c87420b072fa68d921eda66069542d50accbfaf1110ac4cc1543a4162200d
2024-02-04 04:01:11 +00:00
drh 7d6f33046f Following a ROLLBACK that reverts changes to an RTREE, any pending queries
against that same RTREE abort with code SQLITE_ABORT_ROLLBACK.
dbsqlfuzz de7d17b72d0e842352c998dd86a47b7d0f707be9.

FossilOrigin-Name: af5c425114f32c2f84aea20edd4fa46eb1bfdeb3747fce357540e15978a070c8
2024-02-03 19:19:18 +00:00
drh 50f9caf270 More extensive use of SQLITE_CORRUPT_PGNO.
FossilOrigin-Name: 3838332cffb87f77a7c6b357066a59b9dc4f65ef4b3496767ad7ad7c14a77be2
2024-02-02 18:42:09 +00:00
dan d90ecb5d6e Have "PRAGMA quick_check" compare the number of entries in tables and indexes.
FossilOrigin-Name: cc294c041b4c7a044ff344989f872415ced5263a0b654112371b2da7c852a688
2024-02-02 16:51:24 +00:00
dan 3a32690a55 Fix typo in vdbe.c comment. No changes to code.
FossilOrigin-Name: 62010ba488f65aec3c6bd17b05f64c5b3e885dee4221e6016c4eb1053e19284d
2024-02-02 11:37:03 +00:00
dan 09768559b6 Ensure the fts5 xIntegrity method correctly returns error codes unrelated to corruption or missing SQL elements.
FossilOrigin-Name: d18d9a05c2d6f290d7b16eafced0cde4eb8dd3f869778a49103812d35a0cee65
2024-02-01 15:42:22 +00:00
dan adea051707 Ensure the fts3 xIntegrity method correctly returns error codes unrelated to corruption or missing SQL elements.
FossilOrigin-Name: 1bdb8cbaf7e733e0731de2ecd7a05ddf44db5d8595dcc7aaf0927ed0aa5b33d3
2024-02-01 15:21:55 +00:00
drh 64b76c0ed4 In PRAGMA integrity_check, defer running xIntegrity on virtual tables until
after all ordinary tables have been checked.

FossilOrigin-Name: 8f4b1ceafe4a271b23e17493a244a34c1732a3d35c5533c37394b9f3dc158435
2024-02-01 14:57:24 +00:00
drh 1d09f4d018 Add tracing logic to the shared-cache locks in btree.c. The tracing is
off by default.  Enable by changing a single "#if 0" into "#if 1" and
recompiling.  Debugging code only - no changes to release builds.

FossilOrigin-Name: f2b943f97ad7e47848ac6df3a3a1eba134b9e63c4a631f8eaf8bda77cc02ba7b
2024-02-01 14:17:01 +00:00
drh d87299cece Add the test_oom_breakpoint() routine on debug builds, to serve as a
convenient breakpoint to intercept OOM conditions.

FossilOrigin-Name: e45df7dcd6b5766d7593ee87e59dd422a217cce0a1a8d369c03144bb21859428
2024-02-01 11:38:58 +00:00
drh ed0a614c21 Replace an conditional assignment that was made obsolete by [d4c193f0b49f4950]
with an assert().  The conditional was added by [d6fd512f50513ab7] as
a fix for tickets [c36cdb4afd504dc1], [4051a7f931d9ba24], and
[d6fd512f50513ab7] which means now [d4c193f0b49f4950] is the correct fix
for those tickets.
that check-in

FossilOrigin-Name: 44b5524d522e749ad6bf76c94d754ff16c309c32439ec46802924663f64e8b09
2024-01-31 20:11:54 +00:00
drh d844e0a1c4 Allow control characters in JSON5 string literals.
[forum:/forumpost/05182119f69c3a92|Forum thread 05182119f69c3a92].

FossilOrigin-Name: 380f09c194caff557640692d2f255f8cdc1dcfed5976711686466692f4d7a60d
2024-01-31 15:29:29 +00:00
drh 744581d3d6 Add NEVER() and ALWAYS() macros for the JSON5-control-character change.
Also fix an incorrect comparison used to determine if a buffer needed to
be resized.

FossilOrigin-Name: e3c0c0e686f3b7710f79587cf465b5aac52d8f2f2986a3de885a656f652cbfd3
2024-01-31 15:20:13 +00:00
drh 9486178f39 Fix failure to detect unterminated string literals in json_valid() in the
previous check-in.

FossilOrigin-Name: 5d725644621cf640fb743d59ccf89dac777540410e67501eb00b83638c8c857f
2024-01-31 14:44:59 +00:00
drh c24f53635c Allow control characters in JSON5 string literals.
[forum:/forumpost/05182119f69c3a92|Forum thread 05182119f69c3a92].

FossilOrigin-Name: 34709c7cc910539e23a830ad8b589a97a88be25e924a59670c1017fb51447dad
2024-01-31 13:46:44 +00:00
drh b202a452ad Fix a memory allocation bug in the (debug-use-only) json_parse() SQL function.
FossilOrigin-Name: 32ce7dacf58bbf35cf70da8b03fa25f97fcea970edbc1f8aa12739ac4c8608fb
2024-01-31 12:15:57 +00:00
drh 91981fe74e Detect malformed nested JSONB earlier and stop rendering to avoid long
delays.

FossilOrigin-Name: 72f3e1d0eca2b9ebd7c4b4cdccc6d7a0173b2fdb204cb1aa207917aedab6098a
2024-01-29 21:09:56 +00:00
drh af3824d5fa When generated text JSON from JSONB, do not continue descending into
nested structures after an error is seen.  This avoids long loops and wait
times.

FossilOrigin-Name: 97666ec052ebaceab002874d7ca5c5e6883c3d04fb7d3992235a8c4c8d08407a
2024-01-29 20:36:17 +00:00
dan 581bf002ce Consider an index for queries like "SELECT count(DISTINCT col) FROM ...", even if the index records are not smaller than the table records.
FossilOrigin-Name: ad06868807a27f0c96fa5649ebd981e07909eb2c5e03781377c876fdec49abdc
2024-01-29 19:30:16 +00:00
dan 7f9a1ff3f2 Avoid a potential buffer overread when handling corrupt json blobs.
FossilOrigin-Name: 738473dc0ac353731080d0785cc7dc9810b564906c176916bd91d6cfbb1a7b18
2024-01-29 15:30:35 +00:00
drh 3fc7a34efc When rendering JSONB back into text JSON, report an error if a zero-length
integer or floating-point node is encountered.  Otherwise, if the node occurs
at the very end of the JSONB, the rendering logic might read one byte past
the end of the initialized part of the BLOB byte array.  OSSFuzz 66284.

FossilOrigin-Name: b0eb279ea83c1c788c39fb90e178ec99fa4c782195c376a420c661fedf4545a7
2024-01-29 12:50:32 +00:00
drh 4c11a5251a Use __ppc__ instead of __POWERPC__ to identify 32-bit PowerPC processors.
[forum:/forumpost/34794846ce|Forum post 34794846ce].

FossilOrigin-Name: c974d9313b60591bcd554c3ec652a8040d382930e30778e6be8a875145b0b3da
2024-01-28 20:42:12 +00:00
drh 5b480ead99 Automatically disable the DISTINCT optimization during query planning if the
ORDER BY clause exceeds 63 terms.
dbsqlfuzz afd1d61fc27d14938a0d78a50970fa1e9fbfee58.

FossilOrigin-Name: d4c193f0b49f4950b20c2f0e6aa037d2ed7d8c0b4687c14923b3a0d0d4a1b3fd
2024-01-28 17:39:31 +00:00
drh d921533954 Apply the same fix found in [99057383acc8f920] to descending scans.
dbsqlfuzz afd1d61fc27d14938a0d78a50970fa1e9fbfee5.

FossilOrigin-Name: e1d463c2d6e93e0ae0a60a05a79cd346bd07142de2fe631b370a9b946763b5d6
2024-01-28 00:31:47 +00:00
drh 052f0a95d0 Improvements to parser stack overflow testing.
[forum:/forumpost/99e181b5bf|Forum post 99e181b5bf].

FossilOrigin-Name: 556c43a937f1d1c19117da24e6d892852241189f4e47e1c86aa1e8c36461c5de
2024-01-27 20:17:05 +00:00
drh 21bdfe5884 Performance enhancements to the parser template.
FossilOrigin-Name: 2db8b30acdeaeaf7ec92dc0382a25f96bca4561fb68a72713ff963e27f39c63b
2024-01-27 12:47:51 +00:00
drh 48e899a86e Use an alternative memory allocator for parser stack space that includes
a call to sqlite3FaultSim() to facilitate testing.

FossilOrigin-Name: 7c36d560ff4e8e1b3f8fad972ec7f07837e4fa4e6861fafde970ffccfda5a2f1
2024-01-27 12:25:07 +00:00
drh ad2689fe4e Allocate new parser stack space from the heap if needed, eliminating the
possibility of a "parser stack overflow" error as long as heap memory is
available.

FossilOrigin-Name: d87a2054774aa6ce54d9ccd78899b638f1eaf4f9a1d847bf22500018049c9f8d
2024-01-27 11:35:35 +00:00
drh 7659ce22c5 Optimizations to ParseFinalize() to make up for the extra cleanup associated
with the allocated parser stack.  This branch now runs faster than trunk
and is less than 300 bytes larger.

FossilOrigin-Name: f7290db63cc2568090c14dffc4ea4eadfacb5b94b50a1852ef6eefd9e2e32533
2024-01-27 02:21:25 +00:00
drh 3467698000 Omit the "parser stack overflow" error message, since it no longer occurs.
Replace it with an ordinary "out of memory".

FossilOrigin-Name: 33f17ec24880937892d47a5a321c43785ca18e03dde28a9a0d50ae86edb29679
2024-01-27 01:35:07 +00:00
drh 51f652de10 Bug fixes in the function that expands the parser stack.
FossilOrigin-Name: e91179fe849760771c3508b1e7d75325183e5c3b029752d0a97dbdbd57188b97
2024-01-26 20:57:19 +00:00
drh 82bf13796a Experimental changes that prevent parser stack overflows by growing the
parser stack with heap memory when it reaches its limit.

FossilOrigin-Name: 3fd062905fc20507b7cfc97fa976ac5b57c5b68926bf9136bd5ea4265d2d6528
2024-01-26 20:34:48 +00:00
drh 539085ddf5 Add NEVER() to a branch that is no longer reachable.
FossilOrigin-Name: 9411337a7b3237366768fc708396da53d67a7a17b6cdc5c6f8932c5ab32217a9
2024-01-24 21:08:57 +00:00
stephan 4821b15dca Update ext/wasm/SQLTester for recent internal-API reorgs.
FossilOrigin-Name: 996cfdf9b5f70408faeaa68ba2ea9494e419be8f2c59d89ab702419056e3569c
2024-01-24 19:54:49 +00:00
drh fa079680f1 Slight adjustment to test results for Windows in mmap1.test due to
the previous check-in.

FossilOrigin-Name: 5cf9ea9b1fff4625a50a2343d0fd3812da6612d6b24c07e10edb6c1ff03ca2df
2024-01-23 21:10:01 +00:00
dan bcf3df0192 In os_unix.c and os_win.c, do not allow xFetch() to return a pointer to a page buffer that is right at the end of the mapped region - if the database is corrupted in a specific way such a page buffer might be overread by several bytes.
FossilOrigin-Name: 2684feac3bc9c5463604900d72710be861527614f4957224c74a16a3b3c702f5
2024-01-23 16:09:22 +00:00
drh 1ff9a7bc8d Improved error message when a double-quoted identifier name cannot be
resolved - ask the user if they intended to use a string literal.

FossilOrigin-Name: efc5c3c5e6ae1f9f0be61ea4ce7b3e34f3dc318791c78e0f4bebebeaccf6951e
2024-01-23 14:45:22 +00:00
drh 5ff513c16e When a JSON input is a blob, but it looks like valid JSON when cast to text,
then accept it as valid JSON.  This replicates a long-standing bug in the
behavior of JSON routines, and thus avoids breaking legacy apps.
[forum:/forumpost/012136abd5292b8d|Forum thread 012136abd5292b8d].

FossilOrigin-Name: e5dc81d5c7ee97866feb688dfa9b6fc225dabff2b020b9b96b49a8fea5640aec
2024-01-23 13:51:55 +00:00
drh 6bb8ce6375 Fix typo in the tag-20240123-a in a comment.
FossilOrigin-Name: 1f09541eb4203ccce476f6040ff134353850f22f07fb84f96a1d2c27b304a91b
2024-01-23 13:28:21 +00:00
drh e318f10ce2 If a BLOB looks like JSON when cast to text, then treat it as if it really
were JSON.  This replicates a long-standing bug in the JSON processing
routines, and thereby avoids breaking legacy.

FossilOrigin-Name: d79a37690ce7ebb91df203170d73511da44546328043c2b3fe1786b2f0087093
2024-01-23 13:21:40 +00:00
stephan 91cee17d83 Change the shell's --help flag to exit with code 0 instead of 1, per /forum and /chat discussions.
FossilOrigin-Name: df563a18578c5cf1aaecf5818c93967aaa70475598d3c099779622ac8911f926
2024-01-23 12:51:17 +00:00
dan 406eb5a146 Add extra checks for the validity of a numeric literal to sqlite3DequoteNumber().
FossilOrigin-Name: d57407ef59baf699b72c8c4859abfaa7977dd41f6f16eb8fe1d53a68806eb966
2024-01-23 11:20:58 +00:00
drh 727b35cbfc Simplification of the error reporting logic.
FossilOrigin-Name: 59eb9d29e796886db17a578b64e75fd2797aebf969cbeeda70ecf2fda378c98f
2024-01-22 20:49:47 +00:00
dan a545de6827 Fix a problem in the previous commit with hex literals that start with "0X" instead of "0x".
FossilOrigin-Name: c063c89b11487e6e712b97de604db316fa97bcf91ed810bb2dcbbcb54c68dbf4
2024-01-22 19:42:56 +00:00
dan 8374f7dfaf Allow underscores to occur in hex literals.
FossilOrigin-Name: 81a56229460cc5b6acfd3c3729fcf89ea3cccb546ca2b4f4035b140c60911e18
2024-01-22 19:38:55 +00:00
dan fbb72fae8f Add extra tests for the code on this branch.
FossilOrigin-Name: b15bbb201a7727806b9fd6c02f2579b01e0e38155a8f9e24c5d6f487fe3e8b6c
2024-01-22 19:00:50 +00:00
drh 87ad06ed99 Do not allow digit separators that are not surrounded on both sides by digits.
FossilOrigin-Name: 5bbc378a3022e2013048c8e296d95de21d15a8d7c1b62e8ffb64782f952cfe12
2024-01-22 17:18:41 +00:00
drh a67d63b481 Improved error message when a double-quoted string is used and it seems likely
that the user wanted a single-quoted string literal.

FossilOrigin-Name: 0a834bd81122b6e9c34c8253f9e45d53dab3274be28df60968366fa42237617b
2024-01-22 15:26:33 +00:00
drh 41fb2eed07 The -DSQLITE_JSON_BLOB_INPUT_BUG_COMPATIBLE compile-time option causes blob
inputs to JSON functions that are not JSONB to be processed as if they
where text, immulating historical bugging behavior which some applications
have come to rely upon.  See
[forum:/forumpost/012136abd5292b8d|forum thread 012136abd5292b8d] for
discussion.

FossilOrigin-Name: 65572223583d43e1d61ec029641f9d3ff340e68ecfba8342c8d1b0a91a680f2a
2024-01-22 14:16:10 +00:00
dan b4b7088d5f Fix test script literal.test so that it works with SQLITE_OMIT_ALTER_TABLE builds.
FossilOrigin-Name: 4dc00f577632c7b13135505007804d42a12f81a9ebd58d4c380727256b054b7e
2024-01-22 14:01:07 +00:00
drh 776fe42c70 Fix userauth so that it works together with SQLITE_OMIT_SHARED_CACHE.
([forum:/forumpost/0bfc5888a384d430|Forum post 0bfc5888a384d430]).
However, also change to code to issue a deprecation warning whenever
SQLITE_USER_AUTHENTICATION is used.

FossilOrigin-Name: 249048b0cbc37058c229a785182e07b46029584aff9c385269b0b93503153476
2024-01-22 12:56:58 +00:00
drh 3b799b0fca Add a notice to the user-authentication documentation to say that the
extension is deprecated and may disappear in the future.

FossilOrigin-Name: fe6fc7b91ad1771a2d2d4e3d9d60831b3056d528a700932bf11a336c7e3414a2
2024-01-22 12:30:17 +00:00
dan 17589f7811 Fix test script literal.test so that it works with SQLITE_OMIT_ALTER_TABLE builds.
FossilOrigin-Name: b57327be4be9638f5ee834a6c9e06135aaef9e499516f8b53f927ffe81033f61
2024-01-22 11:12:38 +00:00
larrybr 05ab0f72c3 Make CLI .dump immune to pragma reverse_unordered_selects, as reported in [forum:/forumpost/c42be01880e961eb|Forum thread "Missing ORDER BY in .dump?"]
FossilOrigin-Name: 488caed24453fcef6fddcf40353e3b44cf37fb9d21e0ed743d39d60cbff020a7
2024-01-22 01:11:24 +00:00
drh 199b0b2d6e Fix a harmless typo in a comment.
FossilOrigin-Name: 382a8f94bffef3e5f715116e363a30e21e2c53cd42d4959dfbe90c4624143e6d
2024-01-21 21:20:33 +00:00
dan b75dc61a53 Merge trunk changes into this branch.
FossilOrigin-Name: 03ade4a810516ff84bc5c1a716b0e454cc8a89374100415ff21a82449112811b
2024-01-20 18:45:30 +00:00
dan f28bff745c Ensure that values generated by DEFAULT clauses that specify real numbers that can be expressed as 64-bit integers (e.g. -1234.0) are not silently converted to integers.
FossilOrigin-Name: 298d6977285c71be917896bc875a8a26d985dcf0a74069b7c4d290e8ff0ac618
2024-01-20 18:41:13 +00:00
drh 514bf99b6a Simplifications to the strftime() logic.
FossilOrigin-Name: aaa5a044d8a4942278eb4269fa8c7252f9f1dc161408e618e7c97c55a12283ff
2024-01-20 18:26:28 +00:00
drh 6c64e24f9d Fix a memory leak in new memdb1.test test cases that were added by
[e638d5e408ea2e18].  No changes to SQLite itself.

FossilOrigin-Name: bb2b7a658e0186d8b09199170db17744536602d8282b83c71ed969494d2ca465
2024-01-20 18:21:50 +00:00
dan 95295a7e9b Add test cases for the new code on this branch.
FossilOrigin-Name: 49f29a7b4f44f691ecf7a57b26477971e6ffaf2a0b1690b4926ffbb43fa3929b
2024-01-20 16:46:25 +00:00
drh 2e12890732 Make sure that %V and %G are testing for every since day in between 1970-01-01
and 2023-01-19.

FossilOrigin-Name: 39c475f5fb86526622a715860385f0a3a4b2c18d8ef209779c423527e0639385
2024-01-20 16:38:00 +00:00
drh b23f61b73f Rig sqlite3_serialize() so that it will initialize a previously uninitialized
database prior to serializing it, so that it does not have a zero-byte size
and does not return NULL (except for OOM).
[forum:/forumpost/498777780e16880a|Forum thread 498777780e16880a].

FossilOrigin-Name: e638d5e408ea2e189b6771d16bbc2e42c606e88e05fbea78079b6e39e41f344c
2024-01-20 16:29:19 +00:00
dan 3eae6664a0 Allow "_" characters to appear following any digit in an integer or real SQL literal.
FossilOrigin-Name: 401650aaccbc99246bd4e1ff37a28b78f528178aee2f294d87b9f7fecd7432bb
2024-01-20 16:18:04 +00:00
drh 4c43f1881e When doing a text-affinity comparison between two values where one or both
have both a text and a numeric type, make sure the numeric type does not
confuse the answer.  This is a deeper fix to the problem observed by
[forum:/forumpost/3776b48e71|forum pose 3776b48e71].  The problem bisects
to [25f2246be404f38b] on 2014-08-24, prior to version 3.8.7.

FossilOrigin-Name: 709841f88c77276f09701bf38e25503c64b3a0afbe2fbf878136db12f31cbe21
2024-01-20 15:13:13 +00:00
drh 8dca1905ed Ensure that the replace() SQL function always returns a TEXT value even
when its first argument is numeric and its second argument is an empty string.
Fix for the issue reported by
[forum:/forumpost/3776b48e71|forum post 3776b48e71].

FossilOrigin-Name: 01868ebcd25fadb2034da234c0636e82d07c5abc902ef66493cadfc988e74d7b
2024-01-20 13:18:22 +00:00
drh 777a08893b When backing out a character in a constructed string in JSON, first make sure
the string has not been reset by on OOM.
dbsqlfuzz 2fffbea91a5376526ea118d4fe4188c8dd35e317.

FossilOrigin-Name: 666690eb433fe38fa527ccbbb8e2c00041a33939da4f6b8bfb737d664f28f0d8
2024-01-20 12:13:00 +00:00
drh a74a7f5b9e Fix the version number in the TEA configure script. This should have
been changed by [1481baf3d55effcc] three check-ins ago.
It is unclear how testing missed this.

FossilOrigin-Name: eaf2cf73fca3355edd05fef5b0d0262a6555633908794068805d12b1948beedc
2024-01-20 12:00:20 +00:00
drh f08287cc12 Implement a new algorithm for computing ISO week values in strftime() based
on the idea (from [forum/forumpost/3681cb1bcd|Nuno Cruces]) of shifting the
date being tested to the Thursday of the same week.

FossilOrigin-Name: b06ab46a9ee98719159ed3e05cdfbf26281353d781206f56ed7cb12859210fed
2024-01-20 00:31:44 +00:00
dan a4802720a1 Allow large hexadecimal literals to be used as DEFAULT values.
FossilOrigin-Name: 8cccc1f27d7470d3cdd3c9c6d74f6a5ac49ec6eaa7002bcf96f4842fb8c79e1a
2024-01-19 16:51:34 +00:00
drh 0a626b2b52 Increase the version number to 3.46.0 to begin the next development cycle.
FossilOrigin-Name: 1481baf3d55effcc117f2097e2d49a6f60f5f74b21190c3be943fd785b8a4c5b
2024-01-18 16:52:11 +00:00
drh d0603eda6c Add support in the strftime() SQL function for conversion letters %G, %g,
%U, and %V.

FossilOrigin-Name: e1155d6aa4b960ecfd14fa3467f28672af3327699c547f5b9e75da3ac1348ff7
2024-01-18 16:50:25 +00:00
drh 31ebe65b5e Fix harmless "unused parameter" compiler warning in the new fts3IntegrityMethod
implementation.

FossilOrigin-Name: bb1fe53a97672fc868e3f3897162ea0d714dc1da95187f9c7e35255049f2b0e7
2024-01-16 16:14:52 +00:00
dan 75f9ba32bd Ensure that the xIntegrity methods of fts3 and fts5 work on read-only databases.
FossilOrigin-Name: b855886c4ccce0745af6957943e77be18949722f09821688725d546d3d79b4fb
2024-01-16 16:05:21 +00:00
drh 6ad928d91d Clutter the code with "fall-through" comments in order to suppress nuisance
compiler warnings.  No logic changes.

FossilOrigin-Name: 05d2cf5e90d2a6b8e7fbcdb9e12e7c1281db7cfbe212997bd63c8aa66797edda
2024-01-16 15:04:19 +00:00
drh 9239fb59ad Remove the LLONG_MAX preprocessor macro from the series.c extension as it
is apparently only C99 and later.
[forum:/forumpost/4af649419b|Forum post 4af649419b].

FossilOrigin-Name: f106bc0d21b7a815f0d23a97b6fd63c54d3f5353e965dfa550fa715e698ec6e3
2024-01-16 14:54:54 +00:00
stephan 6f30cac209 Various build- and code-reorg cleanups for ext/wasm. No functional changes.
FossilOrigin-Name: 835bd4a11e56ede378235379604ac01350ee92433f952a23c761f7be492fd475
2024-01-16 14:24:35 +00:00
drh f352e0a9cf Version 3.45.0
FossilOrigin-Name: 1066602b2b1976fe58b5150777cced894af17c803e068f5918390d6915b46e1d
2024-01-15 17:01:13 +00:00
dan 83b448f75a Merge latest wal2 changes into this branch.
FossilOrigin-Name: c39f6585226dfc460372d55189b37fba07468126d3961bff625cea8a95956783
2024-01-13 20:36:34 +00:00
dan ff46ff5c73 Merge latest trunk changes into this branch.
FossilOrigin-Name: 68c2b8d0ebd1a180c4828cb4996c6c5fd67386c3b6304edf587c255f8d067c24
2024-01-13 20:21:36 +00:00
dan 4c6554223e Have the shell tool automatically enable SQLITE_CONFIG_DQS_DDL when executing a ".dump" script against an empty db.
FossilOrigin-Name: f47a5f4e0ce078e6cc1183e6cbb3c4013af379b496efae94863a42e5c39928ed
2024-01-12 11:44:49 +00:00
stephan 2fe917cfa5 wasm build: reformulate an awk invocation to account for awks which do not support the -e flag. Problem reported on the forum via a docker-hosted build.
FossilOrigin-Name: 5836657929d7d6d3f0d5865c9f6de4e4c00edd53d15a246802b375fa15d0f255
2024-01-11 16:10:22 +00:00
stephan 7a543f6fbf Internal JS doc cleanups.
FossilOrigin-Name: c8284170f66374fc528e9d6e5a03ee7321f27e80acf4360701a8b176497b7130
2024-01-11 14:21:15 +00:00
drh e5b2132df6 Fix a comment in sessions. No functional changes.
[forum:/forumpost/8c20dc935b|Forum post 8c20dc935b].

FossilOrigin-Name: b0eb6d3628c1f70399a22d9fd3b79a796bc343adfeba50515440db609565961a
2024-01-11 14:13:17 +00:00
dan c08a8a33d3 In fts5, flush the contents of the in-memory hash table whenever the secure-delete option is toggled. This prevents spurious corruption reports under some circumstances.
FossilOrigin-Name: ccf552319a62bfb329820a3bc1f490bacbaa6e90694a257fc65a568a605542c3
2024-01-11 14:03:08 +00:00
stephan bb9ab35ab1 Remove some dead JS code. Minor doc cleanups.
FossilOrigin-Name: bf23cf204976516651b1c4c39ced21cd858dea4ba88052d96fc4f5f11525f170
2024-01-11 12:56:03 +00:00
stephan 598328209f Split the JS vfs/vtab helper code into discreet units as a step towards a build which optionally elides those pieces. This is an internal restructuring change and does not affect the API.
FossilOrigin-Name: ede945fd2360097d9961b8a4b8fb48fea57399cb9163534ed1c3c6b86588b0a5
2024-01-11 12:31:58 +00:00
larrybr 3a39c44c21 Fix a few compiler gripes.
FossilOrigin-Name: 2e74a938f771a889dd8c49648b5a125ff748835ba1925388cc6b379a0abde9ac
2024-01-10 20:13:38 +00:00
larrybr 20728a599e For CLI shell and other utilities, optionally avoid C runtime file I/O on Windows in favor of WIN32 calls, while nominally preserving FILE* API interfaces. (a WIP, awaiting build and testing)
FossilOrigin-Name: 6be68be17927e72b1f1c4b93e781b7e9235876b3ad9512fe168e5b9d0b7b763a
2024-01-10 05:52:02 +00:00
drh 5fdabcf9ff Put an SQLITE_ENABLE_SETLK_TIMEOUT branch inside the appropriate ifdef with
an assert on the else since the condition is always false if SETLK_TIMEOUT
is not available.

FossilOrigin-Name: d81e7a036ac5d70b6a6ee6ab7d81e041c1f5fc04b70bcee47e203d521caf7e93
2024-01-09 23:15:46 +00:00
drh ff6905a1a5 Improved resolution of unqualified names in the REINDEX command.
[forum:/info/74cd0ceabd|Forum thread 74cd0ceabd].

FossilOrigin-Name: 97709ce2a1f5ae05495e412ca27108048e5b8a63a1e3bca4be13933f7527da7b
2024-01-09 12:28:51 +00:00
dan e452bf40a1 Automatically turn off DEFENSIVE mode in the shell tool when executing scripts generated by the ".dump" command against an empty database. Add a warning to the top of generated ".dump" scripts that populate virtual tables.
FossilOrigin-Name: cd016f26bb61549a304f2148035e050f76a8f4a35cdb7131bba2f5fc5d09f49e
2024-01-08 19:55:40 +00:00
dan 62f0c4d139 Fix date on new file shell9.test.
FossilOrigin-Name: c82da712113d5dcd63b764dbc68842026989627abc840acb4a33f3a4972b832a
2024-01-08 18:53:27 +00:00
dan 7cda91c33e Automatically turn off DEFENSIVE mode in the shell tool when executing scripts generated by the ".dump" command against an empty database. Add a warning to the top of generated ".dump" scripts that populate virtual tables.
FossilOrigin-Name: 6e9e96b7e7afb9420110f4b93d10b945c9eadfde5e9c81e59ae9ee8167e75707
2024-01-08 18:46:34 +00:00
drh 3e2ffbd476 Minor change to os_unix.c to facilitate 100% MC/DC testing.
FossilOrigin-Name: 0dfa7b4da134db281c3c4eddb4569c53a450f955f0af2f410e13db801aff4ea2
2024-01-08 15:23:45 +00:00
dan f640f79d6f Ensure that SQLITE_PROTOCOL is not returned too early when a SQLITE_ENABLE_SETLK_TIMEOUT build fails to open a transaction on a wal mode database in cases where blocking locks are not being used.
FossilOrigin-Name: a8e9af1356f5fb2ec460f932dfbe89283bb4e3cf9fa677d1acdbe77ffa11dd04
2024-01-08 13:38:15 +00:00
stephan 6b36d0b461 Make explicit which JS APIs are for internal use only by moving the JS-bound internal-use-only functions out of client-visible reach and renaming the WASM-exported ones from sqlite3_wasm... to sqlite3__wasm... (with two underscores). These have always been documented as internal-use-only, so this is not a breaking change except for clients which have ignored the docs.
FossilOrigin-Name: 0eddc20f37988df6bce5f407b69e4a315e5cca4af104586e6fe942f0d656cccd
2024-01-08 07:52:47 +00:00
drh bf64cbd463 Remove an ALWAYS() from RTREE. Dbsqlfuzz found a way to make it false.
FossilOrigin-Name: 40f0a29e6dd90fcb969d7c0e49728ba0ee8f31d9e8f502b9a21469620a8ad283
2024-01-07 20:27:54 +00:00
drh cd6ea30b82 Updates to RTREE to facility testing.
FossilOrigin-Name: 7a5b42ff74882c58493dc8b710fde73d4ff251f5d42271d84be73ceaabc01698
2024-01-07 00:45:53 +00:00
dan 4e50f7763b Ensure that SQLITE_PROTOCOL is not returned too early when a SQLITE_ENABLE_SETLK_TIMEOUT build fails to open a transaction on a wal mode database in cases where blocking locks are not being used.
FossilOrigin-Name: b934a33671d8a0190082ad7e5e68c78fe0c558d102404eafc1de26e4e7d65b92
2024-01-06 19:16:54 +00:00
drh 1ba138401e Update JSON performance testing procedures for clarity and to describe how to
do performance testing of JSONB.

FossilOrigin-Name: b115b4f75bc7c4e6d9bab5edf13297f27a36f30083c80d2c502b01208da5dfc0
2024-01-06 15:22:16 +00:00
dan f537bba1a1 Update extension ext/misc/totext.c to avoid both ubsan warnings and dubious real->integer conversions.
FossilOrigin-Name: 541436004df6458dc2f38bdfa421099f78cab397f5f4795ca5f227531aaa6a3e
2024-01-06 13:58:54 +00:00
dan 1cfc040bba Update extension ext/misc/totext.c to avoid both ubsan warnings and dubious real->integer conversions.
FossilOrigin-Name: c626aa108a7a30cef54af8d93ac9e45749568ed38e4e06623a6bad6b4bf6e8ec
2024-01-05 15:53:58 +00:00
drh 76da0dcbc4 Avoid errors with SQLITE_OMIT_VIRTUALTABLE builds in json106.test and unionall.test.
FossilOrigin-Name: 8940e2a1054fbc19fae3f76e743d744840c3a5aad001be8d3d56ca134226c34b
2024-01-04 17:13:39 +00:00
drh 7936fef022 Merge the latest trunk enhancements into the bedrock branch.
FossilOrigin-Name: 707f79c70c8b0fd889aede8806148457c691a008d0ce030084ba79f1e66ec53d
2024-01-04 16:28:49 +00:00
drh b60eb8bf6e Merge the latest trunk enhancements into the wal2 branch.
FossilOrigin-Name: 8fb42df89a47b716c824de8742b7e3bda1a5c0f9a85ce3f328d7aa94ab735497
2024-01-04 16:16:41 +00:00
dan de70e8b9c4 Avoid errors with SQLITE_OMIT_VIRTUALTABLE builds in json106.test and unionall.test.
FossilOrigin-Name: 90e8a233549a2d31e6959ce3fec927693b772ab3c0abce65e81d7350d2ca5cc6
2024-01-04 16:15:21 +00:00
drh 267721e72a Restructure some code to fix what appears to be a false-positive UBSAN warning.
FossilOrigin-Name: fe952c12903ea2150880c8bb57cda2efc00ce9fa801568a68c619e0745f30567
2024-01-04 13:01:02 +00:00
drh aa08f885ba Fix a #ifdef in sqlite3_test_control() that was preventing builds with
SQLITE_OMIT_WSD.

FossilOrigin-Name: d546a9c94caf7408cc6e4530ec190d3a13fae09dc15b71b03d6369e02ee62abd
2024-01-03 20:40:17 +00:00
drh 4093b29610 Convert the JSON functions to use lookaside memory allocation whenever
feasible, to avoid hitting the global memory allocator mutex.

FossilOrigin-Name: a79a244954f728596da3c0e28fa3b887258d1bd831f53881970f418f3fba84c7
2024-01-03 16:41:50 +00:00
drh 433e904d99 Change a constant from decimal to hex to avoid a compiler warning on Mac.
FossilOrigin-Name: e3acb8a43ad544fd5b5341058276bd3b61b6bdb6b719790476a90e0de4320f90
2024-01-03 15:49:04 +00:00
drh 6db9206c18 Enhance sqlite3_analyzer.exe so that it uses the ext/consio extension.
FossilOrigin-Name: 769de0b98e136e4a0945b80216d0c9583c1ccd9de69cb0494875c2300e172646
2024-01-03 14:13:27 +00:00
drh 4167033eaf Update the sqldiff.exe utility program so that it uses the sqlite3_str
string interface, and so that it does console output using the
ext/consio extension.

FossilOrigin-Name: 4443b7e592da97d1cb1b3b79ed0559452d8057a33aba4d184c2fffbf200e05f5
2024-01-03 12:26:02 +00:00
drh d059153deb Back out [b517a52fa36df0a0] which is no longer reachable due to early
error detection enhancements in [166e82dd20efbfd3].

FossilOrigin-Name: 704943e96f2620b99260667ac9922c2f72bc3e92e2dfe1d9c2a91c7b704564d9
2024-01-02 22:49:14 +00:00
drh f5740f72ec Adjust the sqlite3PagerDirectReadOk() routine (part of the
SQLITE_DIRECT_OVERFLOW_READ optimization) to use less code and to be
more easily testable.

FossilOrigin-Name: eed670ea2a9424f7df4eeb01c152fc38f7190a5e39aa891651b28dc91fcdc019
2024-01-02 21:37:51 +00:00
dan 4eb626350a Fix a problem in fts5 caused by a COMMIT involving fts5 data that immediately follows a ROLLBACK TO that does not.
FossilOrigin-Name: 55c61f6a8d6a1bc79497b05669beac5c5397b06382bf24b6bec54845962d219b
2024-01-02 21:08:25 +00:00
drh 533622fb8b Increase the default "max_page_count" to its theoretical maximum of
4294967294.

FossilOrigin-Name: ffb35f1784a4305b979a850485f57f56938104a3a03f4a7aececde92864c4879
2024-01-02 20:34:27 +00:00
stephan aff7664939 Elaborate on the various build flavors used by ext/wasm/. Doc changes only.
FossilOrigin-Name: d489232aa492618d4c8e5817addb2323d0ca067742d7140216914239a66fb221
2024-01-02 09:20:53 +00:00
stephan 3d4d3fb59d Update and clean up the in-makefile docs for ext/wasm.
FossilOrigin-Name: 7a7b295e6d7e95ee4a46cc42761895d11700ab295870c5a4380072bb4a5b7099
2024-01-02 09:03:42 +00:00
stephan fe1d8210c7 Back out [99d11e6d0ae6] (enabling of STAT4 in WASM/JNI), per /chat discussion.
FossilOrigin-Name: cd7929ee2e2c305475fa5a4dff2edaccf90067126ef04a1c2714cf464925453f
2024-01-01 23:28:02 +00:00
drh 92e970ae98 New logic to avoid using indexes that ANALYZE has identified as of little
practical use.  Also a performance optimization in ANALYZE.

FossilOrigin-Name: bcac937526d9a6ef914a74b4d6757fa91cd74edab871bcd934fde4a2f9b6debd
2024-01-01 19:20:00 +00:00
drh 4cfc19f34c Remove some unnecessary computations from ANALYZE so that ANALYZE runs with
fewer CPU cycles.  These changes were spotted while working on the nearby
enhanced-stat1 branch.  So even if enhanced-stat1 is abandoned, that effort
put into it will not have been in vain.

FossilOrigin-Name: 5527e8c4abb904b1a438ec1c353d4a960bf82faaf3a2c742af1df7c613850441
2024-01-01 17:58:57 +00:00
drh 4ccf707c6e Extra steps taken to avoid using low-quality indexes in a query plan.
This branch accomplishes the same end as the nearby enhanced-stat1 branch,
but with much less change and hence less risk.

FossilOrigin-Name: c030e646262fee43a59b45fdc1630d972f8bf88ac3c142b6bdaf4cbb36695a4f
2024-01-01 17:48:02 +00:00
stephan 307f326ba3 JNI: move the ByteBuffer-using APIs from public to package visibility for the time being because they have UB-inducing possibilities which need to be worked out. Update test code to account for a change in custom FTS5 columntext() impls.
FossilOrigin-Name: dc501275fcfab3ad9b6ebbadf7588b225a9dd07a0abac5be83d96f15bfba99e9
2024-01-01 06:58:32 +00:00
stephan 805cd395a9 WASM: various build cleanups and add initial infrastructure for a build which elides the oo1 API and its dependents (worker1 and promiser). Sidebar: an attempt was made to move generation of the build rules to an external script, but the mixed-mode make/script was even less legible than the $(eval) indirection going on in the makefile.
FossilOrigin-Name: 563d313163c02b398ae85b7c2ed231019a14e006726f09a7c1f294a58bf4363f
2024-01-01 05:58:47 +00:00
stephan 7a482b3bcd Use SQLITE_ENABLE_STAT4 in both the WASM and JNI builds.
FossilOrigin-Name: 99d11e6d0ae687ff6bac5119027f7b04d5e7185214e79cf8c56289cfa809b0f9
2023-12-31 04:01:36 +00:00
stephan e24cd27c39 Minor doc touchup in the JS bits.
FossilOrigin-Name: 8d2120c35425081e2158d6a8a6b083c4adf8d694046b2d98f5fd235520920432
2023-12-29 04:29:07 +00:00
drh 2aae3a95a9 Enable SQLITE_DIRECT_OVERFLOW_READ unless it is specifically disabled using
the -DSQLITE_DIRECT_OVERFLOW_READ=0 compile-time option.

FossilOrigin-Name: 630604a4e604bfb36c31602917bfa8d42c10c82966d0819932bf8f827b9158b8
2023-12-28 21:02:08 +00:00
dan 751c4e2152 Update fts5origintext4.test to work with SQLITE_DIRECT_OVERFLOW_READ.
FossilOrigin-Name: 15ed002aed12556aeb9bbe537c4ba839f0c95bac65a69d03401b37cc3fd11b92
2023-12-28 20:54:12 +00:00
drh 4c13d3ca92 Performance improvement by unwinding a loop in jsonAppendString().
FossilOrigin-Name: 190ab3c08431a0ba24d76392eab251f5c1792add05e4ec780998b299208eca95
2023-12-28 19:18:08 +00:00
drh 42203e1b75 Fix harmless compiler warnings in FTS5.
FossilOrigin-Name: 3cd5ef44e40570c357f913a9483fa1cd72e7f2827a5ed5826bff99febae213b1
2023-12-28 16:25:48 +00:00
drh 0546a284fa Enhance the (undocumented, debug-only) json_parse() SQL function so that it
returns the text rendering of the JSONB parse of the input, rather than printing
the rendering on stdout.

FossilOrigin-Name: 056de8d551dcbdf1d162e2db15ed418fa9c786f900cd3972ef8a1dea3f4f3aa1
2023-12-28 16:21:22 +00:00
drh 5f1abc5402 Merge the latest trunk changes into the bedrock branch.
FossilOrigin-Name: 5a17b972ed455aac751453cd8eaa8f059db1ac5c7f27965dce4956563e7911ea
2023-12-28 14:01:09 +00:00
drh 5d30acad2f Merge the latest trunk enhancements into the wal2 branch.
FossilOrigin-Name: 1f592dd32d165456d40a90a2757225e05cdb810518beee87f0700863dc73d2d0
2023-12-28 13:44:43 +00:00
dan d9ac37fc44 Fix a problem in the shell tool (not library) causing an out-of-bounds write if an ".open" command failed, then the user pressed ctrl-c to interrupt a query running on the substitute in-memory database.
FossilOrigin-Name: 026618b9e321576f616a32e41329066ba629814170c6cfeef35430343f5003f3
2023-12-27 16:24:53 +00:00
dan d82320ac9a Ensure that the xColumnText(), xQueryPhrase() and xPhraseFirstColumn() APIs all return SQLITE_RANGE if they are passed a bad column or phrase number.
FossilOrigin-Name: 1a8a9b1c89519d265869251e8b6d3c5db733f0d3a7dea6c7962811a8f1157dff
2023-12-26 15:52:40 +00:00
drh b2b7490862 Improved handling of malformed unicode within JSON strings.
FossilOrigin-Name: e252bdf5f5de26ba8e2bcc6b0ad94121ed6fc4d86c02fe4a2a058ada93747beb
2023-12-26 13:20:57 +00:00
drh 71a32aede3 Remove an ALWAYS() added in [c50e6c2ace49d092] because it is sometimes false.
dbsqlfuzz c393a4f783d42efd9552772110aff7e5d937f15e.

FossilOrigin-Name: b9daf37e57cde12c4de271a2b1995e8e91b6411f8c2e8882e536241929609b3a
2023-12-24 12:02:36 +00:00
drh 5a85cf5879 Fix harmless compiler warnings associated with [5db30bcc338aac1c]
FossilOrigin-Name: e55d1c2333f35fc20615aa83a7843d08cae7945710a2156d44eee0cc37d90ade
2023-12-24 11:43:49 +00:00
drh c1805ab222 Avoid signed integer overflow during integrity_check of FTS5.
FossilOrigin-Name: 5937df3b25799eceaadfb04d7226c9995d44c8d8edb5ac3ad02af9d7e3570726
2023-12-24 11:31:20 +00:00
drh 57c98747cb Improvements to the query planner to address the inefficiency described
by [forum/forumpost/2568d1f6e6|forum post 2568d1f6e6].

FossilOrigin-Name: 72fcc12cda910a0e3f7875eb3d117b2a5608705c97703985427a02960f1ab5c5
2023-12-23 19:03:50 +00:00
drh 4ed0b117f3 Add debugging output routines sqlite3ShowWhereLoop(X) and
sqlite3ShowWhereLoopList(X) that can be invoked from a debugger to show
a summary of the content of a single WhereLoop object or a list of WhereLoop
objects.  No change in release builds.

FossilOrigin-Name: 5db30bcc338aac1cf081de2deec7e60749ae012e2b6f95ccf745623adb4a31dc
2023-12-23 11:31:47 +00:00
drh 8a630c2aa3 Change parameters on a debugging function to include "const".
FossilOrigin-Name: 94c3e1110c6590261bd30ba317fba4dd94023d69b81a94f4b216cce748fe7489
2023-12-22 21:22:55 +00:00
dan ba5043f818 Add the -fno-sanitize-recover=undefined to the sanitizer builds used for sdevtest and release testing. To ensure that any test that provokes undefined behaviour fails.
FossilOrigin-Name: 89563311adb0ab7c7a3eadb11c2e27fbca50c56fce8ca616628facbc00d72b88
2023-12-22 16:03:45 +00:00
stephan 09e6c82d56 Update #ifdef checks in pager.c and util.c to account for [0462a2612d1fc1d0] to resolve the build problem reported in [forum:9819032aac|forum post 9819032aac].
FossilOrigin-Name: 0f22d809a1c6c80e381f6bcd931fe4ec36dca0e28d07ab4f4f7f83c813424f60
2023-12-22 15:41:13 +00:00
dan 0810150532 Fix a usan complaint about signed integer overflow.
FossilOrigin-Name: e65907e0279f4814ec957f0790777d8b94a86926cd27c52442b311b27efc0185
2023-12-22 14:47:30 +00:00
drh 15bff25111 Add a new comment to debugging output routine sqlite3WhereLoopPrint() to
remind us of what the various fields of the debug output mean.  No changes
to code.

FossilOrigin-Name: da5f34fd4052432b1ae27bb12e56b358cdc5c1282653d60ed0f0fe62f727e4ee
2023-12-22 12:57:49 +00:00
drh c2eff91bd3 Add internal core-developer-only documentation of the JSONB format.
FossilOrigin-Name: 4d30478863b2a60512010de9ec6e3099bfaf75d4afee20acec536713fe94334d
2023-12-21 18:08:05 +00:00
dan 0d7f0e49a4 Fix SQLITE_ENABLE_SETLK_TIMEOUT assert() statements in os_unix.c to avoid reading past the end of the unixShmNode.aMutex[] array.
FossilOrigin-Name: 029a05cd2928d43d81e4549cce5388c432e2c9e75e3fa0b2fe6e91021b2fb9ac
2023-12-20 19:33:41 +00:00
drh 95cf95841c Avoid harmless integer overflow in pager status statistics gathering.
Response to [forum:/forumpost/7f4cdf23f9|forum post 7f4cdf23f9].

FossilOrigin-Name: 206d8c650d937bc700946c40a82a62ea6bc4a80e5f3fb42d0ae2968de25f0644
2023-12-20 11:34:17 +00:00
drh 3262ca83a7 In JSON - minor code cleanup and refactoring with a small size reduction
and performance increase.

FossilOrigin-Name: 215fabda38daecdbd38b1eca5a6aafbc61b6a36a8303f1d7164d5a1138e63134
2023-12-19 21:39:58 +00:00
drh c52aebb443 Remove redundant conditional from sqlite3ExprCanBeNull().
FossilOrigin-Name: 257f96a2d22c605885fa66220c28cf7dc5941c330bccee3f132b9e7b70d89d30
2023-12-19 15:51:14 +00:00
drh 7c1033b010 On second thought, we don't really need sqlite_dbdata accessible to the CLI.
FossilOrigin-Name: 36fe6a61ef8fb393281a5e15119d716521219c7b971fbfd63bdea07d27a78ac9
2023-12-19 15:10:25 +00:00
drh 4449a1b66d Fix harmless compiler warning in the randomjson.c extension.
FossilOrigin-Name: debe7060b16669ada7304ffb9bf7616c8fa30bd286d8be871ed17fd6d64a3d4c
2023-12-19 15:06:40 +00:00
drh 611b9d3efd More precise computation of the size of data structures in the query planner.
Response to [forum:/forumpost/7d8685d49d|Forum post 7d8685d49d].

FossilOrigin-Name: 0c8d88e41167ea92341dd1129be01b596a73f46bdcd5b0dd931441a979c013d0
2023-12-19 14:54:52 +00:00
drh 09259aff6c Add ALWAYS() and NEVER() on branches made unreachable by recent changes.
FossilOrigin-Name: c50e6c2ace49d0928b05cbfd877c621e9a0f77dc4e056ccb1dbe5cf118a00d00
2023-12-19 14:53:39 +00:00
drh 3e2ec83e0b Ignore COLLATE operators when determining whether the result of a subexpression
should be shallow-copied or deep-copied.

FossilOrigin-Name: 34ae36a45e814bed7c8340412c7ef3fc849b82357656d0eb5f0f805e59d846d0
2023-12-19 13:45:05 +00:00
drh 9c36095367 Test case for the previous check-in.
FossilOrigin-Name: df5a07e1a5122e08c2fa6076ac08adb2820f997ee11dd88b84863666899dfb57
2023-12-19 13:00:35 +00:00
drh 60c435dc4b When unable to resolve an identifier, change the Expr node into TK_NULL
rather than TK_COLUMN, to prevent any downstream misuse of the non-existent
column.  dbsqlfuzz 71869261db80a95e4733afa10ff5724bf3c78592.

FossilOrigin-Name: d2e6117e4f97ab98b01deb5fcad5520f8181d00bed8d904d34963c01d73df857
2023-12-19 12:49:35 +00:00
drh 3a0e82611a Always make the sqlite_dbdata virtual table available in the CLI.
FossilOrigin-Name: e5fd3b32ad87586a7413570e568c9c1859a37a4f836cca074126471b125fb682
2023-12-19 12:23:11 +00:00
drh a3569fb2ca Extra ALWAYS() macros to verify state in the sqlite3ExprCanBeNull() routine.
FossilOrigin-Name: be19b84c9f3fe127165809908add148dbe9a827a55608b0490de7e69b7f7f191
2023-12-19 11:57:29 +00:00
drh 459f7b1090 Remove a stray comment in the JSON code.
FossilOrigin-Name: 6618bdf0679405b43911ea8cd94050b12a5dc469f3dfe4759ee3ff850a55229e
2023-12-19 00:07:38 +00:00
drh d9252d09c2 New JSON invariant test cases.
FossilOrigin-Name: a6a1367b0bf364b1a2e20e153c5f4a578624b8846f9ec0b7c9c3cba0ea2ec346
2023-12-18 19:18:46 +00:00
drh c4dd6b42ec Add NEVER() to an unfalsifiable branch.
FossilOrigin-Name: 9a0c67db366d38a0b0741f6a1ae333cf27cfe6f6b7c6eed94bdec9686f9f9f8a
2023-12-18 18:50:47 +00:00
drh fc76750f61 Fix JSON to JSONB translation so that it deals correctly with Infinity
and NaN.

FossilOrigin-Name: 178cb84f36bdb45ba17511900d6d8ea8dfa14912fc5bf7094a20348174a36c95
2023-12-18 18:31:27 +00:00
drh 095f2c5082 Ensure that the insert/delete size delta on JSONB objects in the JSON cache
are always set to zero.

FossilOrigin-Name: 4b4581668a908473dbf1322a3e98bc7cca122998c44518ea183af7f0d1ba9f95
2023-12-18 15:53:48 +00:00
drh cf6fe5abe3 Add randomjson.c to testfixture. Use it for a new set of invariant tests
against JSON functions.

FossilOrigin-Name: f1c040606bfe784804134d8f3ca130908fad5212b47e3c32792baab977470943
2023-12-18 14:24:13 +00:00
drh 0a46ff995b Ensure that all object labels for individual objects generated by
randomjson.c are unique.

FossilOrigin-Name: 29c46aca231b3f1e997ef306a5a651408185bf3ad09ab9fc1fe21ed18caa4d02
2023-12-18 14:16:58 +00:00
drh 52a4691fff Bug fix in the randomjson.c extension.
FossilOrigin-Name: 1f3a33df530dbe330ea8b14a69369b807b413b25a167d1a3938f8f0faf97cc91
2023-12-18 13:51:54 +00:00
drh 2ae95649de Enhancements to ext/misc/randomjson.c.
FossilOrigin-Name: a4e6d1f86f3a502e4170f5a90031e269e48363e95114a66b84d373e3ce0b2704
2023-12-18 12:18:47 +00:00
drh 0d201598a5 Enhancements to the "randomjson.c" extension. Automatically load that extension
into fuzzcheck.

FossilOrigin-Name: 70620405ab01d6a5d38bafa9ae175fd6e4eabaf2efb7854734278dafd7b05c99
2023-12-17 20:41:48 +00:00
drh 13264418ba Enable SQLITE_STRICT_SUBTYPE for default builds of the shell, fuzzcheck,
and testfixture.

FossilOrigin-Name: 5a0c517ed7e46c0f8a3db752cf5b9f8010c60f35084606abe9e7c1c4f993b4a7
2023-12-16 15:48:42 +00:00
drh 5f3dd8668c New test case based on Chromium bug report 1511689.
FossilOrigin-Name: 2c7ef4b4d215f99f8d6787adb64e2037ae96e5dd6cb49c8b81634249f5e1b328
2023-12-16 10:50:06 +00:00
drh 40e614e393 In the count-of-view optimization, deferring freeing obsolete parts of the
parse tree, on the off-chance that some other part of the code might be
holding a pointer to those parts.

FossilOrigin-Name: da442578856c87137eb1677d9b13b7c1cf15828cc41d4756572b278060f69bae
2023-12-15 20:13:09 +00:00
dan 97f7ead3f3 Avoid expiring prepared statements in the middle of an integrity-check.
FossilOrigin-Name: 88beb48472da4667c0727c8ebabe046ea526450ff837fe789d041ed3f1ff105e
2023-12-15 19:26:16 +00:00
drh b7fad0dca3 Add mention of --buildonly and --dryrun to the testrunner.tcl usage screen.
FossilOrigin-Name: 23b92d915c12ee768857e2c3c961832f390cad9b53b8bcfc2b97664baab25bb7
2023-12-15 16:28:02 +00:00
drh baa1aba8d7 Fix a new JSON test case so that it works even if SQLITE_OMIT_VIRTUALTABLE
is defined.

FossilOrigin-Name: b995aae510888a9746b46545d176a0885d4738e1f1bc0b7ad7937ed023efd7d6
2023-12-15 15:22:03 +00:00
dan f2194605f8 Do not run test script fts5origintest4.test with either "memsubsys1" or "mmap" permutations.
FossilOrigin-Name: 05a63d9603ef42cbee6dadff72d97583a9c78e549f70e9a808534d5c1ae7c28a
2023-12-15 15:17:39 +00:00
dan 76dbb58700 Avoid running the "no_mutex_try" tests with SQLITE_ENABLE_SETLK_TIMEOUT builds as part of the release test.
FossilOrigin-Name: 6b4e1344a28c213cbe8fb97f7f3f6688de93fb73ed96bf460ff74c959da1a712
2023-12-15 14:33:50 +00:00
stephan f062506002 Correct --enable-sab flag in ext/wasm/GNUmakefile to fix a silent alhttpd args-parsing error.
FossilOrigin-Name: 7b9b757d872a31395b0f6454e2309a6a4664b8bdd8749f6a15371cbe72c05b60
2023-12-15 13:38:47 +00:00
stephan e982385619 Use SQLITE_STRICT_SUBTYPE=1 by default for the JNI and WASM builds unless they're explicitly built with SQLITE_STRICT_SUBTYPE=0.
FossilOrigin-Name: 990211357badf0ab08bd34cf6d25b58849d0fd8503e289c1839fc837a74e1909
2023-12-14 22:01:55 +00:00
drh eb78d56ec5 Add assert()s to FTS5 to fix static analyzer warnings.
FossilOrigin-Name: 27d4a89a5ff96b7b7fc5dc9650e1269f7c7edf91de9b9aafce40be9ecc8b95e9
2023-12-14 16:34:47 +00:00
drh 4a5c96ace3 Fix harmless compiler warnings in JSON and FTS5.
FossilOrigin-Name: 90135efccfeb1046f002bfcbd8dfec9a1a3b40cbe1b5c714ae065b06368e354f
2023-12-14 15:38:57 +00:00
dan e6289d6dd0 Improve the error message returned by an fts5 'rebuild' command on an external content table if there is a problem with the content table or view.
FossilOrigin-Name: 0fbf4b8a58fde1c187908934da6f59999b146f32e07ac255cc531c5c4d7007fd
2023-12-14 15:31:35 +00:00
drh 8a861a81e8 Enhance aggregate order-by so that it transmits subtype information through the
sorter.  Fix for the deficiency reported by
[forum:/forumpost/87347ad2fb5a8f76|forum post 87347ad2fb5a8f76].

FossilOrigin-Name: d302a389460d0c15775a8b5f5afbac2c1d8a91bc282bc9b04c583ca04a8c09c6
2023-12-14 15:11:39 +00:00
drh 07117f8118 Pass subtype information through the aggregate ORDER BY sorter for
aggregate functions that use subtype information.

FossilOrigin-Name: 3536f4030eab6d650b7ed729d2f71eb6cc3b5fbe16b4e96b99008d66522aaccb
2023-12-14 13:58:50 +00:00
larrybr 9e488a5a40 In CLI, fix .read inability to open 2GB+ files on WIN32.
FossilOrigin-Name: 56c80a62d2e033d64ba5d545ae9cbe3ed7c9d046c0a3fafb6cfa2f0b562d1ef0
2023-12-13 20:37:20 +00:00
drh 3e4195c60d Avoid invoking sqlite3ExprColUsage() on an unresolve column reference.
dbsqlfuzz fc34aa62df4de103705d11b807074687ffafbda5.

FossilOrigin-Name: ac9314c0e335694b48c613145f5397247bb88c51806cd0dc3ed4ec306db4bbad
2023-12-13 16:45:18 +00:00
drh f7c78a2408 Bug fix in the JSONB validator.
dbsqlfuzz ac6fa521a08609a642198e7decf64180e750b3c4

FossilOrigin-Name: 3e940a6a08b0a0434650cd3d8dd286e09ad8ab805b0a4d515e57bba5d3608577
2023-12-13 15:27:03 +00:00
drh 001d1e795c Improvements to UTF8 handling, and especially the handling of invalid UTF8,
in the JSON routines.

FossilOrigin-Name: 1b229c1101d6c384a30f343c5e47b471ab084b2d8e81170eb8f642afc1c67e3b
2023-12-13 14:31:15 +00:00
drh 891f1dc054 Fix the JSON object label comparison object so that it works correctly even
if the label ends with escaped whitespace.

FossilOrigin-Name: 4d5353cadd7b7c5f105bc197f3ec739e2d041472d6b3e939654c9f9cfc2749ae
2023-12-12 18:38:53 +00:00
stephan 9710fa119e Worker1 Promiser API: when multiple db connections are active then use the requested connection instead of always the first-opened connection. Bug reported in [forum:894c330e7f23b177|forum post 894c330e7f23b177].
FossilOrigin-Name: 194276e18e0268829061c09317e7f9f527a703eb45f1755ff1dd30bd99dc1b68
2023-12-12 17:52:27 +00:00
drh df25a8fd4b The json_valid(*,8) function does a much better check of the validity of
the JSONB input.  The json_error_position() function returns an approximate
byte offset to the point of the first detected error in the JSONB.

FossilOrigin-Name: 840efb33a3934a9bdccad65ea15c70d5c335a517502c9844aaad8405d16d1604
2023-12-12 17:31:17 +00:00
drh 78e636bba6 Add NEVER to two unreachable branches in JSON.
FossilOrigin-Name: c96ebb086feb89341565cc52b970ae7799ce1327fe1ad4fc790f1b0dcaa6e229
2023-12-12 17:13:10 +00:00
drh 87399a56db Improvements to JSONB validation - catch more cases where the input does
not conform to spec.

FossilOrigin-Name: be1864eac4eb75cc30bf98f73092c8608467f4bd956240df6a0cbea9f1e09e85
2023-12-12 14:33:52 +00:00
drh cc1a39fd24 Validity checking of text nodes in JSONB.
FossilOrigin-Name: fa5160687c2f970d407e8af73c246f7cd806bb4ce35f29a79ac534a8646a6c8e
2023-12-12 02:31:12 +00:00
drh ad6bc61804 The json_error_position() function now reports an approximate byte offset
to the problem in a JSONB if there is a problem.

FossilOrigin-Name: 80d5d94dff6a2d2557039be3d7d47c1a6003c4b98defe0bd411acfeb963ad5dd
2023-12-11 21:00:55 +00:00
drh 5a890b4ed2 json_error_position() now uses jsonValidityCheck() to find the approximate
position of an error in a JSONB blob.

FossilOrigin-Name: c3d60cf7028a333b825d5b89516945a73e0c158ac81d8bcc117d21bfd98602c8
2023-12-11 20:44:21 +00:00
drh 001caa714f json_valid(*,8) allows minus-signs on hexadecimal literals.
FossilOrigin-Name: c0d7f4520d839a268b3fd2474d0897a9832aa608bd6238b3e287fabecf07a350
2023-12-11 20:19:10 +00:00
drh 7445659559 Activate JSON_SELFCHECK within fuzzcheck.
FossilOrigin-Name: 4d14e733bb521aed65e98533969d2303738232ae87dab70fdf7962e6513195f5
2023-12-11 19:21:06 +00:00
drh ba5505657f Rename the new test-control to SQLITE_TESTCTRL_JSON_SELFCHECK. Make it so
that the current value of the setting can be interrogated.

FossilOrigin-Name: 7aff1d9a4cb17ecd5abab21ab032f35a78741dd641ddd8cbcc85fc4a81a0707d
2023-12-11 19:00:44 +00:00
drh 7d2eaae83e Add SQLITE_TESTCTRL_VALIDATE_JSONB, which if enabled under SQLITE_DEBUG causes
cross-checking of generate JSONB.

FossilOrigin-Name: b410a4db74a650003539ffaaea18519d5159b504daac47db6a4874b730f40ac8
2023-12-11 17:03:12 +00:00
drh ce46e0eb11 Work toward enhanced functionality for json_valid() with deep checking
of the JSONB (second argument has bit 0x08).

FossilOrigin-Name: c370d573198b151767f04e91bf8baa4ae0076751ae468c5709742a0b0ed16770
2023-12-11 14:01:38 +00:00
drh b89e64d822 Fix a potential use of uninitialized value in json_valid() with 2nd
argument of 8.

FossilOrigin-Name: fa102036fe46eeb71b7df3e265be1935ae5c78e0b939b08841bcfb8abadbc77a
2023-12-11 02:39:11 +00:00
drh bfa0de86e6 Fix a harmless UBSAN warning.
FossilOrigin-Name: 1503cba6d17e9bade7a5c103ddd23241ff4741f9a2e3032ffe2987af243dae65
2023-12-08 16:56:50 +00:00
drh 2a27be2107 Fix a potential problem RCStr access on a JsonString object that is not
really and RCStr.  Fuzzer/UBSAN find.

FossilOrigin-Name: d2f2174ce2cc89606034e158149a2d05fc3627ec4d5cdb772add7a2250f29d78
2023-12-08 14:54:22 +00:00
drh 9d2446dc13 Fix a harmless compiler warning about "confusing indentation".
FossilOrigin-Name: 34f9e9a8c4bea13f60f43062e25cd7d9422f2e7f5b371ed0ddadc9abeb3ca256
2023-12-08 12:58:41 +00:00
dan 05980f5931 Avoid dropping an error code in new fts5 tokendata=1 code.
FossilOrigin-Name: a66596e33dc9aa4bab2ec3ff45546e1321d0a11bdc764f8381b315292ca92423
2023-12-08 12:04:32 +00:00
dan 0b427eb796 Ensure an fts5vocab table never uses a special tokendata=1 merge cursor.
FossilOrigin-Name: 1e26510e83b40c9bd2e8bfa2a0e81f2cb915e78fed773204ef537683e48b61dc
2023-12-07 21:09:33 +00:00
dan 7c12666fe9 Avoid an assert() failure when querying an fts5vocab table that accesses a tokendata=1 fts5 table with corrupt %_data records.
FossilOrigin-Name: 386ba9e20423fb2f623d6adc9d3c310fb1b135f54a1dad15ef3b593d97886926
2023-12-07 20:46:40 +00:00
dan 1ddcf7dd95 Fix a null-pointer dereference in fts5 tokendata=1 code.
FossilOrigin-Name: d69fa8f0504887f968d9a190ecb889ddb40bb1b56d0d4479f9819c106aec719b
2023-12-07 19:08:25 +00:00
dan 5f2f8d0028 Fix a problem with handling OOM and other errors in fts5 when querying tokendata=1 tables.
FossilOrigin-Name: bc911ab5953532956510c199be72b1d3c556f2d0ddbd7fc0ae6f5f917b337b48
2023-12-07 18:41:49 +00:00
dan 2f3ccb8886 Different fix for the fts5 COMMIT-following-OOM problem first fixed by [fba3129d]. This one does not cause problems if an fts5 table is renamed and then dropped within the same transaction.
FossilOrigin-Name: d8c6b246944934a7a6e027b3f5b986fd64a19dd5c5c5175f4ea8586da59a6764
2023-12-07 14:41:58 +00:00
drh 5a238ffcae Rework the jsonEachPathLength() routine in json_tree() so that it is
less susceptible to problems due to goofy object labels.

FossilOrigin-Name: 858b76a00e8ff55215f7a2e6a4cd77fc4d4f98dea7224cd90488744f5ce246a4
2023-12-07 14:09:25 +00:00
drh f86a07b0c4 Add ALWAYS() on branches added in [ec0ae4030968c782] that are always true.
FossilOrigin-Name: 451cef8609e96dd9244818adc5c6f240544694bcb4ae620e88f90e403e59d70f
2023-12-07 13:14:34 +00:00
drh 5b6b703461 Improved detection of corrupt JSONB in the jsonReturnFromBlob() function.
FossilOrigin-Name: b014736c1f80ccc46fb4b24ac04310a6ce5cb5b6653665efff366cb3bc742257
2023-12-07 12:55:39 +00:00
drh ca86e10cdd Merge the latest trunk changes into the bedrock branch.
FossilOrigin-Name: c2e53000f4740be9f0492fda1857c5f2f5539fe17bdaafcafdf8d14c8f3218ce
2023-12-06 21:18:03 +00:00
drh d352e7d6ac Merge the latest trunk enhancements into the wal2 branch.
FossilOrigin-Name: 457724e7380a8e768d945afb37fb634a6ec83ddebbc4ad144229c0dd498a5b2e
2023-12-06 21:11:11 +00:00
dan 00f3ac5544 Fix compiler warning about shadowed variable in fts5_index.c.
FossilOrigin-Name: ee70e4c1c9c41617850228e48d8df44f105cf2fbbe789340ceca6f27ad6ce5eb
2023-12-06 18:34:59 +00:00
drh 82fc1b63f6 Work around LLVM's newfound hatred of function pointer casts.
[forum:/forumpost/1a7d257346636292|Forum post 1a7d257346636292].

FossilOrigin-Name: ec0ae4030968c782af48d1c776351c14b2ada21d40aeb97915f33df30706e18f
2023-12-06 18:25:41 +00:00
dan 83ac79282a Update documentation comments in fts5.h.
FossilOrigin-Name: 38c50e22c98607e6c1fd78d7615cda534773b6d4fd85c712b54749fcd7af0c83
2023-12-06 18:10:50 +00:00
drh a9c8469d03 Do correct comparisons between object labels in JSON even when the two labels
contain different JSON escapes.

FossilOrigin-Name: bda2e30cc22e180b19a7a05824dd345880eb402ae5450b2d2dd954946c3ae135
2023-12-06 17:50:16 +00:00
drh b597fea894 Correctly handle 8-byte sizes in the JSONB format.
[forum:/forumpost/283daf08e91183fc|Forum post 283daf08e91183fc].

FossilOrigin-Name: 73d390f39c0bbbc017e01544e4d43c76761f2599bd57f900131c706270dfd202
2023-12-06 17:39:31 +00:00
drh 9df01b5ccf Fix the routine that determines the json_tree.path value for the first row
so that it correctly takes into account escape sequences in the path
argument.

FossilOrigin-Name: b9243ee8a37c62eb8848e765bd4af83bc1b3d3eb24fb4268a1357ad1f8b2e1fb
2023-12-06 16:57:18 +00:00
larrybr 3207199c98 In CLI, move -interactive flag handling back to arg-loop pass 2.
FossilOrigin-Name: 63cb05a862532d2d56e9e81fe32ced09bf58f03146587a118f11c2a84e195e69
2023-12-06 16:27:29 +00:00
drh 8dfbf4addc Test cases for object label matching with escape sequences.
FossilOrigin-Name: c6f2aa38e95b7888650cfa7bb773b18a28e01d883033ac77be6d504ffe417d18
2023-12-06 15:50:13 +00:00
drh 6a8581d828 The rule for the RHS of the ->> and -> operators when the RHS does not begin
with $ is that it must be (1) all digits, or (2) all alphanumerics, or
(3) contained within &#91;..&#93; or else it will become a quoted label.

FossilOrigin-Name: 0e059a546ec11fa5c6d007bd65c249ee2422f1facbdb2792c53e0bc0ccc97e14
2023-12-06 15:35:38 +00:00
drh 91ec00c25a Increased rigor in comparisons between object labels in JSON.
FossilOrigin-Name: 2bc86d145fccc07107b7753cb1a69122676d4096fe59c454497bd81a6142d45e
2023-12-06 14:50:48 +00:00
dan e751596ee8 Add the tokendata=1 option and related APIs to fts5.
FossilOrigin-Name: a76a636b23c0ebd95d47fdf8358de4729e51a5f68f1a730cd4d89b378e94ac0d
2023-12-06 14:36:34 +00:00
dan 39b08db132 Merge trunk changes into this branch.
FossilOrigin-Name: 8f46eace86e7b2e556913575aa3cd6f7987ac0efcc880f0af649d42c253aeb81
2023-12-06 14:30:34 +00:00
stephan 1f2d7c4312 README.md typo fix reported in the forum and update all links from http: to https:.
FossilOrigin-Name: 5c48acdbb44185b352b54911a57a6986d6c7e624bdeba2af48b985d29f0292bf
2023-12-06 12:30:28 +00:00
drh 88208296c2 Rework the JSON functions so that they use the JSONB format internally.
The original JsonNode parse tree design is removed.  All JSON functions
that accept text JSON also accept JSONB.  New functions generate JSONB.

FossilOrigin-Name: 7f0c79b94e8f55e5013e52ba64ba8b32dad1dc4e2224d2099733cbc561de1810
2023-12-05 19:45:09 +00:00
drh 5afd67b3c3 Use extra assert() statement to silence harmless static analyzer warnings.
FossilOrigin-Name: 174c2b2eef5fecd96a5fc89b81032fe81f7801f12097cea10e7e7f0a02114813
2023-12-05 19:24:07 +00:00
dan fb923fc4cc Further tests for the new code on this branch.
FossilOrigin-Name: 59d008b6c23ab900377bc696ee19381feb7614bac80546eae361e401c3620c4e
2023-12-05 18:36:23 +00:00
drh a0de45459e Miscellaneous comment cleanup and typo fixes.
FossilOrigin-Name: 59446dc0bd0091572122a3c8b4653d7a2dc867d16c4a5919f79b81bc3a673ce3
2023-12-05 18:28:15 +00:00
drh 8eac91fab7 Use strspn() to accelerate whitespace bypass in the JSON parser.
FossilOrigin-Name: 843197df08352bdff4b87be91d160e574572aded0d0c66142fd960000c0b4701
2023-12-05 12:52:13 +00:00
drh 590aaff992 Small performance gain by unwinding the string literal delimiter search
loop in the JSON parser by one more level.

FossilOrigin-Name: 4c587feac153e8ebe526559ec3d254f545f81e8d1ed3126f91a5ff25ec4aa72e
2023-12-05 12:22:05 +00:00
drh 78f7d2f4c1 Clean up the JSONB performance test script.
FossilOrigin-Name: 905301075a7fc1010ee7e754867b1b698c9b8576d50e98125def32a5dfb7ee9d
2023-12-05 12:20:58 +00:00
drh fa43e21711 Use an assert() to fix a harmless static analyzer warning.
FossilOrigin-Name: a249ca657e624028bc6b3d2c2bcedd7162d118addb7d62ce519920cecebf1860
2023-12-05 01:44:15 +00:00
drh ae2e972802 Fix OOM and corrupt JSONB handling in json_patch().
FossilOrigin-Name: 1910feb0b7d5cc2b810c3322f6cca281d8730182d30d162bd7bb56800979ea91
2023-12-05 00:17:17 +00:00
drh 3cdb079476 Rename the internal routine jsonMergePatchBlob() to just jsonMergePatch().
FossilOrigin-Name: ebf667b616235bb64b83832008342ba5e7b10b2c170d7cebc431f040fef7ecfb
2023-12-04 23:12:57 +00:00
dan 49bfbc1ef3 Add further tests for xInstToken().
FossilOrigin-Name: 8582707f16133f003a6687f68cbea03d4eb6c2a0e2e07746b7cace0c44e84fa4
2023-12-04 19:48:08 +00:00
drh f26e26afc5 Fixes to error handling in json_array_length().
FossilOrigin-Name: aa85df2d26b74c171c55bde19ef17c4f11f40b8af7181bbf7162f87cdea7e88b
2023-12-04 19:32:17 +00:00
drh 0b8b1c309d Do not make the input JSONB editable in json_remove() if there are no PATH
argument.

FossilOrigin-Name: 66594544f3ba9977475a3e3f74404eb2b2fb845053b28bd24c2b52c7df94e9d7
2023-12-04 19:14:13 +00:00
drh 3fedb7e59e Fix errors in rendering JSON5 escape sequences embedded in JSONB.
FossilOrigin-Name: f1a51ae3863557526a51c6e98e71fcdf4f1ed14a36212b3c90f7408f926345e4
2023-12-04 18:53:10 +00:00
dan 54318b382a Fixes for xInstToken() with tokendata=0 tables. And with prefix queries.
FossilOrigin-Name: 78fbb71598b1ca756acc078253880a1d0f7983a5a26b9efc683e6488122505a1
2023-12-04 18:45:14 +00:00
dan 910c77b049 Fix memory leak in new code on this branch.
FossilOrigin-Name: ebc160b9a05568df66f86e30804399ee29d34b44a60c57e062f98cb92826353f
2023-12-04 17:58:56 +00:00
dan 3dfc063705 Fix a problem with the xInstCount() API and "ORDER BY rank" queries.
FossilOrigin-Name: 317a50563d9e8586fda136e513727241b414e7267d50a06571c8ebd0eae710bc
2023-12-04 17:45:33 +00:00
drh 9c794b9bff Continuing simplifications and code cleanup.
FossilOrigin-Name: ddf92b5059a9106753fd18b82ba8daa269a62af947561c460790107b83416f0b
2023-12-04 17:40:28 +00:00
dan 9d373ca1c5 Fix bug in xInstToken() causing the wrong token to be returned.
FossilOrigin-Name: da78d07e77cbc783fbc725758911c230fd6a1c1885d9576125de955dcc2bd37f
2023-12-04 17:05:37 +00:00
drh 99c41692f1 Remove reachable ALWAYS and NEVER macros.
FossilOrigin-Name: f601de3eeabd85993c1f5ee96b62de6fdabbeae2fe8950e00d08feb48d42c498
2023-12-04 16:01:39 +00:00
drh 732fb64ad3 Two new NEVER macros.
FossilOrigin-Name: 52632c92cb06faf0e804654b3490fd6c199521107bd30c8fcbc3a2a5a488098f
2023-12-04 15:22:42 +00:00
dan 43b4864a98 Add tests for using tokendata=1 and contentless_delete=1 together.
FossilOrigin-Name: a2506b8c9718054912270055638204753c4156bbc115e55194e6df9d7e76cb10
2023-12-04 15:08:21 +00:00
drh dc138cb186 Repair issues and inefficiencies found during testing.
FossilOrigin-Name: ae973cb1515f9d76409c92a2ca2ffd6b71f32b0b490a4886770e7c1b90f12611
2023-12-04 13:12:45 +00:00
drh 2ff73a5f3e Better pre-scan size estimations for objects in the JSON parser resulting
in fewer reallocations and memmove operations.

FossilOrigin-Name: 526b27f90897f5e35dfff7257daf6c4ce4798d649b09b8aecfb02df0449e3c51
2023-12-04 01:14:23 +00:00
drh b7fd951be4 Back off on the use of strlen() for situations where sqlite3_value_bytes()
will work as well, for performance.

FossilOrigin-Name: 79fb54fbb8b9c30f47cdbd437d24a21542716241e822749e5e28c9fbc449bfa8
2023-12-04 00:31:58 +00:00
drh 9af45dc482 Remove dead code. Improved reporting of errors in JSON inputs.
FossilOrigin-Name: 2eaa738e6b5c1b67b3e57c868d9c3a30eea38a0b3b8b02482f06d57a45b10921
2023-12-03 23:38:24 +00:00
drh 16e8a5b2f3 Avoid problems when the path argument to json_tree() contains embedded U+0000
characters.

FossilOrigin-Name: 9f055091af01a5dddba1a7e9868ad030c8f206237e1569215cb161e53e54aa71
2023-12-03 23:30:59 +00:00
drh a3bf077b60 Ensure that OOM conditions in the generation of the "bad JSON path" error
message result in an SQLITE_NOMEM error.

FossilOrigin-Name: aa0e02b5c26a2ef3d6216a0ed8bc01382be43173485f898cb63f2a8c559f2e74
2023-12-03 20:11:35 +00:00
drh 0a18a5807a Enable incorrect JSONB to be rendered into text without hitting an
assertion for a bad whitespace escape in a string.

FossilOrigin-Name: 4d6a9a217df6792b41766b774fb0c0553b45f9104c26a0955bf4a30862d7d7bf
2023-12-03 19:59:45 +00:00
drh 78fa0186b8 Do not let bad hexadecimal digits in malformed JSONB cause an assertion fault.
FossilOrigin-Name: 8dec1ba1e5076ff596756e00c1e2ada0245f168a503dd1cadadf848331acfac3
2023-12-03 11:54:39 +00:00
drh eb18ae3089 Minor code changes for consistency and to simplify testing.
FossilOrigin-Name: df272bd837910ad9e03e222716a1201a601399664365f1dcf73d5932372518ed
2023-12-03 00:51:30 +00:00
drh c78c3c91ae Implement strict JSONB checking in the json_valid() function.
FossilOrigin-Name: 0f26d38880fcbc207abcc94dbc170a7428bab1b4f0b7731aaf5bee0224000994
2023-12-02 21:39:34 +00:00
drh 8f8d481485 Fix harmless compiler warnings reported by MSVC.
FossilOrigin-Name: 419652c0c82980bd043584dcd2976f91dfff7b926b216d597698299850b855c0
2023-12-02 20:37:45 +00:00
dan c22d2b7b7f Fix various compiler warnings and other problems with the new code on this branch.
FossilOrigin-Name: 3a623cfa173b4035c759cb84985d11d8727053beb383648503987d6ab15c0ef0
2023-12-02 20:35:04 +00:00
drh c1e85742da Code and comment cleanup. Everything should work the same.
FossilOrigin-Name: c640754df0d3ffdad994745f0d0e10c8f19f424b87f6a6e6e269491a0350b950
2023-12-02 20:25:36 +00:00
drh 53c2160db0 Fix harmless compiler warnings. Refactor some identifier names for
clearer presentation.

FossilOrigin-Name: 7e3941502789c5afaf19b08112f464abf5e3cba7f92fc9290af2a0f96127ad9a
2023-12-02 18:17:38 +00:00
dan 94c521295a When tokendata=1 queries require multiple segment-cursors, allow those cursors to share a single array of in-memory tombstone pages.
FossilOrigin-Name: e0175d07e4094db5ea4b0378a5ff480dafb6ba9da86a113fa767c4c89c3c866f
2023-12-02 18:14:07 +00:00
drh a11aaff05a Take extra care to ensure that JSONB values that are in cache are actually
owned by the JSON subsystem, and that ownership of such values is not handed
back to the bytecode engine.

FossilOrigin-Name: 1304534001e9ef66c6b12752b69d790bfa3427cc803f87cc48ca22ae12df0fdf
2023-12-02 18:04:27 +00:00
dan c44041e03b Ensure that tokendata=1 queries avoid loading large doclists for queries like "common AND uncommon", just as tokendata=0 queries do.
FossilOrigin-Name: 7bda09ab404a110d57449e149a3281fca8dc4cacf7bd9832ea2a1356ad20fe8e
2023-12-02 17:32:16 +00:00
drh 05db513435 Protect a memcpy() against OOM conditions.
FossilOrigin-Name: 26144d1c25ae0435db568009ba05e485d23d146f2b1f29f3a426c87860316aed
2023-12-02 16:11:22 +00:00
drh 2c26bde4ff Do not allow a JsonParse object to be considered "editable" after an OOM.
FossilOrigin-Name: c6bacf57bd6fe0fee00c9d41163a270b60997c20659949971bbf5c6c62622bfe
2023-12-02 15:59:48 +00:00
drh 4cd397c0d9 Fix potential unsigned integer underflow in jsonAppendString().
FossilOrigin-Name: d2fba2cbdc3870d34228c1a9446eced884325acc183900d7dd0b96132570fb4a
2023-12-02 15:06:43 +00:00
drh 82136d90f8 Minor fix to the header comment on jsonXlateTextToBlob().
FossilOrigin-Name: c3677ba410208c07b711f5f526eb5cf039a8eee49f632c7ae04fa55cdfbb9058
2023-12-02 14:55:46 +00:00
drh f0b8b16317 Performance optimization in jsonAppendString().
FossilOrigin-Name: fdf00e96239c73fb67e2acecc5b95f55a1fc51c3deed4512613c0d6070ce5805
2023-12-02 14:16:47 +00:00
drh 679c90850c Simplification and optimization of the JSON parser.
FossilOrigin-Name: f5ec9485119a2a6cb33eb864c7ca9b41d4a2ed08ab6ad9a6b0dd9358ab253576
2023-12-02 13:36:52 +00:00
drh 768b6e32f6 Remove a NEVER that can be true if a virtual table column is declared to have
a DEFAULT.  See
[forum:/forumpost/3d4de8917627d058|forum post 3d4de8917627d058].

FossilOrigin-Name: 8abc2ccaf8106f20243568cd7fa74174386eb85d7ea381201e97e2fd527033e0
2023-12-02 12:23:34 +00:00
drh 6df61985d4 Unroll a loop in the parser for a performance increase.
FossilOrigin-Name: a6dc29e4d5e13949e0fcd9d5dde575c2670eb10a230ab9df3806fc8c3016c540
2023-12-02 01:38:53 +00:00
drh 5ec9c916ad Fix harmless compiler warnings and enhance performance the parser.
FossilOrigin-Name: 285633da6d188547e52f07779e209c9e5f3dc33ce0668e14858f3337889ef4b8
2023-12-02 01:06:33 +00:00
drh 3af20cf3a0 Performance optimization in the JSON parser.
FossilOrigin-Name: 68d191f40e708962ec88e0c245b4496bc4a671300484b1cc0f3fc7e6d199a6e6
2023-12-01 22:01:26 +00:00
dan f4c2962558 Remove old code for tokendata=1 queries.
FossilOrigin-Name: b0a489e8e1bf0290c2117ab32d78b1cc7d67bcb226b55ec044c8367ebde3815b
2023-12-01 20:37:11 +00:00
dan ebd173a79f Merge latest trunk with this branch.
FossilOrigin-Name: 8258967411d3ff212424b25fec79ded0d8ae83e773cd35a0bbf300c94923f25b
2023-12-01 20:10:20 +00:00
dan b5effc0605 Different approach to querying a tokendata=1 table. Saves cpu and memory.
FossilOrigin-Name: c523f40895866e6fc979a26483dbea8206126b4bbdf4b73b77263c09e13c855e
2023-12-01 20:09:59 +00:00
drh 03075fa868 JSON cache is now more effective.
FossilOrigin-Name: 443a3f3a8e64d81cad8300a30e2cc57c4e39f69b5669ac8b550c590ae9f1134a
2023-12-01 18:49:02 +00:00
drh 063d0d4c3a Fix up the JSON cache to work better.
FossilOrigin-Name: 1fdbc39521f63aedc6f08ecaafa54ea467b8c6316a692a18ad01eecbf22a0977
2023-12-01 18:46:14 +00:00
drh 5bfa7e65d1 Cache is working better, but does not preserve the hasJson5 flag.
FossilOrigin-Name: a12add7ab9f5aee5bb2ede0c4d22e599dd28f7a107dce72b2ea48ef92d233e8a
2023-12-01 13:28:13 +00:00
drh ca1ce7773c First attempt to get the JSON text-to-binary cache working. All test cases
pass, but the cache seems not to help much.

FossilOrigin-Name: 25ed295f300fea6185104a73721076bccd2b2a6e411c78564266fa6dca4ff70c
2023-12-01 12:57:12 +00:00
drh 4b9ed1b256 Remove all trace of JsonNode from the JSON implementation. The JSONB format
is used as the internal binary encoding for searching and editing.

FossilOrigin-Name: 11ebb5f712cc7a515e2e0f2be8c1d71de20c97fe5b74c4f4d72c84fd21182d35
2023-11-30 23:36:14 +00:00
drh 38aeb97f27 Convert json_valid() over to using only JSONB as its internal format.
FossilOrigin-Name: 7b5756fa6d00b093bf083a8d7a5ef5485f7a09e4eac473785c8380688f861a1b
2023-11-30 20:57:48 +00:00
stephan 68003d9f18 Add a basic batch-mode SQL runner for the SAH Pool VFS, for use in comparing it against WebSQL. Bring the WebSQL batch runner up to date, noting that it cannot run without addition of an "origin trial" activation key from Google because that's now the only way to enable WebSQL in Chrome (that part is not checked in because that key is private). Minor code-adjacent cleanups.
FossilOrigin-Name: 883990e7938c1f63906300a6113f0fadce143913b7c384e8aeb5f886f0be7c62
2023-11-30 20:34:24 +00:00
drh 276042bd23 Convert json_type() to use JSONB internally.
FossilOrigin-Name: 83074835b900ce85cf67059e674ce959801505c37592671af25ca0af7ed483f1
2023-11-30 19:29:56 +00:00
drh 0ab1e25468 Convert json_insert(), json_replace(), and json_set() over to using only
JSONB internally.

FossilOrigin-Name: 4e2083e86f19ef7634f0b253fb924e52014b43ed0ce8acc51c36f3c5682180a6
2023-11-30 19:11:14 +00:00
drh 66795962b5 Enhance json_set() and json_insert() so that they create missing
substructure.

FossilOrigin-Name: cc7a641ab5ae739d31c24f0ad0caeb15a481a63fa8f13720718ea922c25862ff
2023-11-30 19:06:27 +00:00
drh 7394a6ef57 Simplification of the new JSON insert/set test cases.
FossilOrigin-Name: 04c0d5644372446c924a2e31a26edf51ddc563a1990d170b0ed4739e3e8b239b
2023-11-30 16:17:09 +00:00
drh 48222bef66 New JSON test cases showing insert or set with missing substructure.
FossilOrigin-Name: 6802b6459d0d16c961ff41d240a6c88287f197d8f609090f79308707490a49c2
2023-11-30 16:16:10 +00:00
drh ad27c437ff New test cases for insert/set/replace with paths that indicate substructure
that does not yet exist.

FossilOrigin-Name: 146c717c51940b2139befc45ac74e7a1c36ef3c32fd3cfe35b334488eebe6298
2023-11-30 12:04:14 +00:00
stephan bac2fe9f44 Update some OPFS-related help text in WASM tests. Minor cleanups in speedtest1-worker.js.
FossilOrigin-Name: 263f6d3a7784ef7d032dbf7a3265aca8dd70bf50797f28f6b2e8ddb6a301f83a
2023-11-30 10:00:25 +00:00
drh bfc7e62be4 Convert json_insert(), json_replace(), json_set() to use JSONB internally.
Mostly working, but some corner cases are still not quite right.

FossilOrigin-Name: 99c8f6bd5c9a31b6d00f92e383bec8a8235ed553916ad59adbb1b7663f6ebff1
2023-11-30 00:52:33 +00:00
drh 694beecf2b Convert the json_error_position() routine to use only JSONB internally.
FossilOrigin-Name: e7a8ba35bff6fde55827f978de5b343b6c134c7fa53827f5c63915a9dc2598ad
2023-11-29 20:06:49 +00:00
drh da2578391d The json_patch() function now operates exclusively on JSONB. This patch
also includes improvements to JSONB debug printing routines.

FossilOrigin-Name: fee19d0098242110d2c44ec7b9620c1210ef3f87913305f66ec85d277dd96ab6
2023-11-29 17:36:54 +00:00
dan 89fcfbb424 Fix signed integer overflow in fts5.
FossilOrigin-Name: 60e46c7ec68fd8caaed960ca06d98fb06855b2d0bb860dd2fb7b5e89a5e9c7b4
2023-11-29 16:22:39 +00:00
drh ed088f5f8d In the recovery extension, if a payload size is unreasonably large, it is
probably corrupt, so truncate it.

FossilOrigin-Name: 988c3179e978a3a6d42541e9c7a2ab98150383671810926503376ed808f150ff
2023-11-29 13:47:46 +00:00
drh 6d7afd8ba8 Merge all the latest trunk enhancements into the jsonb branch.
FossilOrigin-Name: 1a59fcab2179cc3b52ecd3de7d2018db96ac149aaff521959773a517b8d9ac3e
2023-11-29 12:18:02 +00:00
drh 2ba5534101 The assertion change at check-in [7946c79567b0ccd3] is insufficient to fix
the problem of a Table object being deleted out from under the OP_VCheck
opcode.  We need to reference count the Table, which is accomplished here.

FossilOrigin-Name: cad269d5e274443c39203a56603b991accc0399135d436996fc039d1d28ec9db
2023-11-29 02:45:09 +00:00
drh 4b54d6cdaf Convert the json_array_length() function to use JSONB instead of JsonNodes.
FossilOrigin-Name: 5ab790736d943e08f097efcee5cfbf0d83c65b0a53f273060330ba719affa5e5
2023-11-29 01:38:15 +00:00
drh 748178a9fb Fix all known problems with JSONB json_extract().
FossilOrigin-Name: d5f48c57e975ac468cf29a43a5d0b56ef6d06cf35a8b0bddf87ec1c0fc7ae028
2023-11-28 23:26:55 +00:00
drh 1f8c7c7053 Do not set the J subtype when the output is JSONB.
FossilOrigin-Name: 4f106b64fe8988435872806bd0a6c223b61f53af0dd1c47c847bb4eec4e03e27
2023-11-28 23:18:04 +00:00
drh 1ef232c0e1 Preserve flexibility in the format of the RHS of -> and ->> operators found
in legacy.

FossilOrigin-Name: 6231ec43adb7436195eb1497de39a6c13c6b4f1c5032e6ea52515d214e61fdbc
2023-11-28 20:33:20 +00:00
drh 1cab41e290 Attempt to get json_extract() working with pure JSONB only, and without
the use of JsonNode.  Mostly working, but there are some differences from
legacy in corner cases.

FossilOrigin-Name: 8c324af1eca27e86adc45622af4f3b06a67a3f968596ac58aa7434b1f6f05f3c
2023-11-28 20:25:23 +00:00
dan 0d4a888485 Merge latest trunk updates into this branch.
FossilOrigin-Name: 554fc13f2ca5f2ebd9ad0206034c25b556ff40db3106051c5e539f2e142e88ea
2023-11-28 19:43:08 +00:00
drh ef97f8360a The json_remove() function now uses only JSONB, never JsonNodes, internally.
FossilOrigin-Name: b69786e746ae2b927b64d9871fd120b7f8f06cc53739fd46a4da51aa16cf8576
2023-11-28 18:16:02 +00:00
dan 47409209c2 Update this branch with latest changes from trunk.
FossilOrigin-Name: 7f1b61a3ea2cc5132abf91b0f635be4fcaa082de7b33ca131874a75ae11ee576
2023-11-28 17:32:30 +00:00
dan 940b23bc3a In SQLITE_ENABLE_SETLK_TIMEOUT builds, use blocking locks in place of sleep() calls when opening a read-transaction.
FossilOrigin-Name: 4c055b7a6e4533e1e571773456226ca7038ce372df3eedbbbcd9a81e8652a6cf
2023-11-28 17:12:42 +00:00
dan b8950e0f45 Handle an SQLITE_BUSY_TIMEOUT error if one occurs while attempting a shared lock on a read-lock slot.
FossilOrigin-Name: 5fbf3906d272df3eb981f67455eb35f649ad2774cba9fc3f077b28d9bef3f0cb
2023-11-28 15:29:04 +00:00
drh 26cd4b57df Activate the ability of json_patch() to work on JSONB.
FossilOrigin-Name: 11aba347ff7c639500eec904e212eabe889b077351b946cfeac2b74b9703672a
2023-11-28 13:38:22 +00:00
drh ec1f59f0cd All legacy tests are passing.
FossilOrigin-Name: 2c436806b8d5f57de99c00f6154b038454fb9ae427d00d7b4a46ab9c7c69bcb9
2023-11-28 13:35:53 +00:00
drh 5026ddb83d The json_patch() code for JSONB compiles and works sometimes, but there are
still issues.  Incremental check-in.

FossilOrigin-Name: e0099464a0045a04f4ccf29bc2b8325fc8c7f39ccf4847e74818f928c9153588
2023-11-28 12:28:28 +00:00
drh f46f89df97 More aggressive use of jsonBlobEdit(). Improvements to the MergePatch
implementation sketch.

FossilOrigin-Name: fbca9570fd2e1465739e4d3a8d9bb40fad594fd78ab49b2cb34efa27ebdd8361
2023-11-28 00:27:58 +00:00
drh eb04a0bb7b Add untested (#ifdefed-out) code for the MergePatch algorithm against JSONB.
Add (and test) the jsonBlobEdit() routine that is needed by the new MergePatch.

FossilOrigin-Name: 4d353387fc10e1038cfdd86e66007bf728c231a928e588897bbee0fbfe76f225
2023-11-27 23:46:12 +00:00
dan bae2d0f2eb Have SQLITE_ENABLE_SETLK_TIMEOUT builds block when locking a read-lock slot.
FossilOrigin-Name: f797baf47cf7859cfd8ce248f4f3087af4551a7040af990333426e5a7c269504
2023-11-27 20:37:03 +00:00
dan e52854a9e6 In SQLITE_ENABLE_SETLK_TIMEOUT builds, use blocking locks in place of sleep() when opening a read-transaction.
FossilOrigin-Name: a51ef39998e25e86bd0600e71d15011b12e05f4319608018293bdaecb09e8c97
2023-11-27 19:22:50 +00:00
drh 8a3034add8 Enhance the (SQLITE_DEBUG-only) json_parse() routine so that it shows a
decoding of JSONB when given a BLOB argument.

FossilOrigin-Name: af267868562e0799ad691dccad05f17afbc34d609eede8c55f57d209290246ef
2023-11-27 17:13:18 +00:00
drh 821a4c9f8c Give the json_valid() function an optional second argument that determines
what is meant by "valid".

FossilOrigin-Name: a4e19ad43dac81e7655ec03ff69bb99d1d02b0c227034c90fb41415fd4793fe3
2023-11-27 15:57:11 +00:00
drh 85eafe6f3b Convert the json_tree() and json_each() virtual tables over to use JSONB has
their internal representation.

FossilOrigin-Name: ec18caa3f7790b780dde66c1ccbb6eb09d2f1507629cc45955fc1b08380b4017
2023-11-27 12:36:29 +00:00
drh b4e5bc6cdb All tests passing.
FossilOrigin-Name: b5a5660ca22437640c9bf32c44d92c76a7293dafcbaf4fa6a4c171128d64871d
2023-11-27 12:30:55 +00:00
drh 15c0b03c5d Fix corner-case error conditions.
FossilOrigin-Name: ec23d34ab75e1d7e9366e59c633e0d30def8759f6d4717583ebeb4c90aeccf0d
2023-11-26 00:56:40 +00:00
drh 50b37832b2 Same results as the legacy JsonNode implementation on a small set of test cases.
FossilOrigin-Name: c3da4b079a1a15a4c0b1a6e71f876648b1d9eb32eddc67b9946c2475c7b6d085
2023-11-26 00:48:37 +00:00
drh e09a38c2e8 Remove unused elements from the json_tree() cursor.
FossilOrigin-Name: 914a50117d477b2cd30d58388fb8d1b71ff7ff6842ba025f38efc6e9647d06d0
2023-11-25 23:00:50 +00:00
drh 796abda538 Remove the vestigal JsonNode logic from json_tree() and json_each().
FossilOrigin-Name: 66c2ab9ebbf90477742e6be0d30e061d827c409de038f2a5b73479ed9448c4a6
2023-11-25 20:59:03 +00:00
drh aea2d23121 Almost working. Path is still not exactly right when Root is defined on
json_tree().

FossilOrigin-Name: 92258246916a9c0d72785964513113848a850dec78bdade8b3f274e410df4e7e
2023-11-25 19:28:44 +00:00
drh c2474105ca Generate the fullkey and path columns of json_tree().
FossilOrigin-Name: ffaa468ab8871906121df9ee5ef3dc00129a0086ed9c18831ecda69bf7f71455
2023-11-25 18:11:11 +00:00
drh b7d5cb711a Handle the path argument to json_tree() and json_each().
FossilOrigin-Name: fded888469565b2a4687185a926bd23fccfbf167c8bebe6c10696fc7f972f41e
2023-11-25 13:40:19 +00:00
drh 5e6500c81c Continuing work on json_tree() against a JSONB.
FossilOrigin-Name: 3df891cb11feee65e239ee2506eda34a9688341f05210d7c2e25a05338cb71ad
2023-11-24 21:57:38 +00:00
dan 48cca2422e Add the --buildonly and --dryrun options to testrunner.tcl.
FossilOrigin-Name: a0c87ae9d3db914d18e2c8811db0d0ae3ad7b15c63de84fa975efce28bace27e
2023-11-24 20:14:18 +00:00
drh abbdbdfc1f Incremental progress toward getting json_each() and json_tree() to work
directly off of a JSONB blob.

FossilOrigin-Name: f8cab41b3bc65af6ff34b481db693d640ea025d09463d50b8e56d855e2abc913
2023-11-24 18:44:00 +00:00
dan 88a61ff8ba Allow a pattern to filter test scripts to be appended to testrunner.tcl "mdevtest", "sdevtest" and "release" commands. e.g. "tclsh test/testrunner.tcl sdevtest fts5%".
FossilOrigin-Name: f8ea0b58b37f4052ded448e595d6d2992988a33b8ecfe58d68f20532f8cb5a37
2023-11-24 18:33:40 +00:00
larrybr f5232b3a97 Get all CLI print calls which went to stdout in 3.44.0 to continue going to stdout.
FossilOrigin-Name: e9951ede184ce07cf725152723d795f299922460715ab76225cd3071bf0f18ee
2023-11-24 15:58:14 +00:00
drh ab70266698 Fix jsonParseReset() to properly clear the JsonParse.aBlob element.
FossilOrigin-Name: ab2644aacf4757a51cf62e05cff6711a0a3605d60502a3dd310887df1b993545
2023-11-24 14:25:56 +00:00
drh c0f8d58b21 Omit precompiled binaries from the source tree.
FossilOrigin-Name: 7dbc2f496d7a362460bb4c262ecafe5f30e35a8744861163d12c996365c2142f
2023-11-24 14:03:20 +00:00
drh d47719ee02 Omit the precompiled binary from the source tree.
FossilOrigin-Name: 4ff103d294b79cf7734e87e94e9d88c0e9f0b087cbb352e6da2f0a3a6b268f46
2023-11-24 14:01:56 +00:00
drh ac8b6e70c9 Merge the latest trunk enhancements and fixes into the jsonb branch.
FossilOrigin-Name: a838ebcbbd9f554fd84a1d3176fb572faaef7d0ec0675f1a5bf9430865dafae0
2023-11-24 13:41:20 +00:00
larrybr 77044f30a2 CLI .output/.once to not redirect ".timer on" results.
FossilOrigin-Name: ce766ed51f772a960d0b7a52c113b55c7ae90ef35050496d8e2a77547eab1a4d
2023-11-24 13:26:42 +00:00
dan 226aebae07 Fix an fts5 problem that could occur when mixing regular and secure delete operations on a single table.
FossilOrigin-Name: 8eb3f40021994f0d25ef9d246873796c84886e5951bb8ab991e1e2df31998484
2023-11-23 11:18:06 +00:00
larrybr 5d175eb020 Fix output redirect bug reported in [forum:/forumposts/cbf4933cfeee74bb|the forum].
FossilOrigin-Name: ce542fee6f0150bbd13505dcd26b21a9d1d23eb2dc8e78e63f9da6b4298ec028
2023-11-23 07:08:49 +00:00
stephan a328c42724 Minor doc fix in src/betreeInt.h, prompted by an email report. No code changes.
FossilOrigin-Name: 30d49aaed20454fe006e2f89db58f0c0b2433c99b2676a0b9583f56b7adb7fdb
2023-11-22 22:59:28 +00:00
larrybr 92752a31d9 For CLI build with unused function warnings on, #ifdef out a console I/O function, fPutbUtf8().
FossilOrigin-Name: da36f90d94b3b55de71aa7965421ea850ac677e9e63d60744633c4dc59448de4
2023-11-22 21:24:50 +00:00
dan 41c9e0b767 Add documentation for new fts5 auxiliary function APIs.
FossilOrigin-Name: 9be8969edd49e3da96fb8ac2279aff6fe2e215d6ac55162b4734aca1b6316580
2023-11-22 21:15:05 +00:00
dan af54826e4a Defer building xInstToken() hash-table until it is to be used.
FossilOrigin-Name: 9b005085ff4a53cda0a1dff0c836630d6d3b95b9c40658ffd2a886f3e1b37faa
2023-11-22 20:02:55 +00:00
dan 5c268bbf67 Fix tokendata=1 and xInstToken() APIs for detail=none and detail=column tables.
FossilOrigin-Name: 37b271c19d772bd06524db816ded03377b426efed7a7783c8a96f6fb156ecd86
2023-11-22 19:02:54 +00:00
stephan c0314bc01d Correct the URL for loading sqlite3-worker1-bundler-friendly.mjs from sqlite3-worker1-promiser-bundler-friendly.js, so that the promiser can be used in bundler-using environments. Problem reported via email.
FossilOrigin-Name: 753a75218913c3b9c7ec5438387107369c34d1775d68b04d19ae18197e558605
2023-11-22 17:11:39 +00:00
drh cf72606a7d Inserts invalid JSONB should return "malformed JSON", not a json path error.
FossilOrigin-Name: 306ee66fbd0231a9f5b229e5630d5cc66c9cf08b466d2d9204e79e1f44a16374
2023-11-21 22:36:32 +00:00
drh 27fea97e44 Direct editing of JSONB using json_insert() and json_set().
FossilOrigin-Name: fffb7a9538838e26991e6f16ea3138346a30c33ea6c3d3834680ee6d1f6eece2
2023-11-21 20:13:08 +00:00
drh e1df37b947 Correct blob-to-text rendering in some corner cases.
FossilOrigin-Name: 7822e0e59f9b611fe6289cc762b0aff61f9b87c3a82c60de110f447589a2c125
2023-11-21 19:05:22 +00:00
larrybr d9f3d6d753 Merge console I/O changes for Windows CLI.
FossilOrigin-Name: 935a8a8ee76d0014df42c1480e044fd1c2dfc26e78abb587d99d861d2ae5eb27
2023-11-21 18:26:06 +00:00
drh 664fe310b5 Fix the translation of JSON5 numeric values from BLOB into text.
FossilOrigin-Name: 40c4fb441f220982e4d61fd42597cf18546791a302fbcc8eec2eed29ee15ef35
2023-11-21 18:23:43 +00:00
drh e97f295783 Merge all recent trunk fixes and enhancements into the jsonb branch.
FossilOrigin-Name: 6d78d50ed2357e6c943c1ef97b1d2ea0902cbadef90c2c35dccdbdc2bdf8702f
2023-11-21 17:54:55 +00:00
drh 8a4cecaea2 Make edits directly to the JSONB BLOB when the input to json_replace()
is a JSONB.

FossilOrigin-Name: d69c6acef54a81f46a97a05d443fe648635b4b70772069d6705ef829b718e985
2023-11-21 17:51:58 +00:00
larrybr e788255681 Sync w/trunk as pre-merge-to-trunk sanity check.
FossilOrigin-Name: 448d6a1182d29940d5d34be2ce67df5601b688cd902dbbe97e95073f982a49ce
2023-11-21 15:55:31 +00:00
drh a4050f1434 Fix the trace3-4.4 test to be more rebust against timing quirks.
FossilOrigin-Name: 8936daa08243729d8538bb7288bbefb43f3bd842a0d4b2e8019092f5701c2926
2023-11-21 12:02:04 +00:00
drh 4f77a27032 Back out an incorrect change to the sqlite3ExprCompareSkip() function that
was added way back on 2019-08-22 for [44578865fa7baf97|check-in 44578865fa7ba]
and which was only today discovered to be incorrect by
[forum:/forumpost/45ec3d9788|forum post 45ec3d9788].

FossilOrigin-Name: f5b3eb0fc8936ba274a7654ff6dfa7d4654bd8dbca7f3a5ec1134b0b5260d59d
2023-11-20 15:54:00 +00:00
drh 0fcaf16f20 Convert an assert in OP_VCheck into a branch that aborts the opcode, as this
can happen on some very obscure conditions, as discovered by dbsqlfuzz.
Test case in TH3.

FossilOrigin-Name: 7946c79567b0ccd3a00d12390e99896d7dc11407d0b52bc39338a16a493f56f6
2023-11-20 13:06:59 +00:00
dan 56bb07bad6 Fix an assert() in fts5 that could be true following an OOM or IO error in contentless-delete mode.
FossilOrigin-Name: 3fe89238c31ca163d021ef1fecde594328815aaba7053fef9e9c789562a62f45
2023-11-20 11:40:18 +00:00
larrybr 033127af75 Cherrypick shell1.test fix.
FossilOrigin-Name: 91c888eddbcb1b461f7d562c2f0eee57f1a2be7f9959b9d69e80fc7c1da20e9e
2023-11-20 00:21:24 +00:00
larrybr 4c9a02c9f0 Cherrypick shell1.test fix.
FossilOrigin-Name: 39e30c5f9cc6dcac003255734e3ce1ac5b05349ea1a25e1c108b5f6d1d97612b
2023-11-20 00:20:56 +00:00
mistachkin 73e3e0c21f Correct conditional compilation issue seen with MSVC in the Win32 mutex subsystem.
FossilOrigin-Name: 6f8f4bfe607f1405d313bb88a33490621002f63e8c02c980f4c083630ad3a6d2
2023-11-18 18:36:26 +00:00
dan 046a7f9493 When ENABLE_SETLK is defined, avoid ever blocking on the lock mutex in os_unix.c when requesting an exclusive lock.
FossilOrigin-Name: eb36d475e91bfdbf4a18b6fd9751fbcecf15d960dcd1c00d2d18b5bf1d7503fe
2023-11-18 17:20:04 +00:00
drh 52c41ed41b Adjust an assert() in fts5WritePoslistData() so that it only applies if there
have been no prior errors.
dbsqlfuzz 25dca9b2568f67dc78a0e32ff280133fe71994bd.

FossilOrigin-Name: 257cdbab90c6db8ccc9a8fd5df556b69c3a35a329d39cd4642c792d7359a54a5
2023-11-18 12:06:21 +00:00
drh b8688e131e Another assertion fault fix, similar to [a9443dbfbe25e588]. dbsqlfuzz
2d9af4e94aca188e0092900eec711401c5d51687.

FossilOrigin-Name: 3afaeac56dff58db596360bf6f8dca97cb31405f73be8e189e8c0e6a1e5b239d
2023-11-18 11:23:01 +00:00
dan 11c512d631 When SQLITE_ENABLE_SETLK_TIMEOUT is defined, use a separate mutex in os_unix.c for each shm locking slot.
FossilOrigin-Name: 64691df980cdf73ddc00aabe15baa6c018e1d76b0e87f55ab33a9348fd5680bd
2023-11-17 19:01:38 +00:00
drh f9ae48b9ed Add the new "run-fuzzcheck" target on the autoconf unix makefile. Requires
that FUZZDB be set to the name of the fuzz-data database file.

FossilOrigin-Name: 8fa3915dbfdd9a5b0fcf4d695590b3de7e7d19d2917924186ca889cd60bf0eb9
2023-11-17 17:55:03 +00:00
drh 003d419b33 Add an assert() to prove the sqlite3_prepare() always either returns
SQLITE_OK or else leaves *ppStmt set to NULL.  See
[forum:/forumpost/70bb8576c6c084c2|forum post 70bb8576c6c084c2].

FossilOrigin-Name: feadd4024228e578811447c4b2d2b60780ff3d3681f80ca903585aa7b289d758
2023-11-17 17:25:30 +00:00
dan 735e7ee62a When SQLITE_ENABLE_SETLK_TIMEOUT is defined, use a separate mutex in os_unix.c for each shm locking slot.
FossilOrigin-Name: 4098df9652d90f2d22d5591d915d672c5413471f7916223510ba6fd932bfdd36
2023-11-17 17:10:37 +00:00
drh e416251725 Merge the latest trunk enhancements and fixes into the jsonb branch.
FossilOrigin-Name: 162f0509ef27bcd3ec87629640281a71c773e7c3bbd2cd0df76faf481531e7f1
2023-11-17 17:03:45 +00:00
drh f2bb6ab3fb Fix harmless compiler warnings in debugging code.
FossilOrigin-Name: ce6a75622ea5bca517bc6613e738aa670c9e1dd863596220eded5c2379c616c7
2023-11-17 12:22:42 +00:00
drh c15133e82a Fix an incorrect assert() associated with ALTER TABLE where an unknown
aggregate function contains an ORDER BY clause.
dbsqlfuzz e0900262dadd5c78c2226ad6a435c7f0255be2cd.

FossilOrigin-Name: a9443dbfbe25e588b4adddde664ddf482f19f71c704fbf356d49cf3a6135e7fb
2023-11-17 11:58:26 +00:00
dan a0764f63a8 When querying a tokendata=1 fts5 table, do not use a prefix cursor for the case where the term has only one variant.
FossilOrigin-Name: d711c96ba855686d6881a50498418de3492144f005684b5ae55bca24413dce47
2023-11-16 21:11:56 +00:00
larrybr e1edecf5d3 Simplify and make more rational how console I/O package features are selected. (Motivated by Fiddle and other command-line utilities to soon use the package.) No substantive code-execution changes.
FossilOrigin-Name: 1cde05877b7e47cb2ab9de26edd5245eb5ff1d91e69b94f3b24944fd4f6ccd3e
2023-11-16 18:31:05 +00:00
drh 11e8242e2e Both json_remove() jsonb_remove() work on pure JSONB as long as the input
is JSONB.

FossilOrigin-Name: 68d551730be0a3ea9579646ed4836c73554c83ca7f2303b69a18843f1750f1a7
2023-11-15 20:32:06 +00:00
drh 5e3ae1ec76 Some simple test cases for JSONB direct remove.
FossilOrigin-Name: 8cb4d2cbfc1b5f3c612d85138e66da8735d6589c7538fdf494a761c9b8104f18
2023-11-15 19:21:09 +00:00
dan 91c8e65dd4 Changes so that if SQLITE_ENABLE_SETLK_TIMEOUT is defined as 2 instead of 1, all blocking locks are taken for a single millisecond and the default busy-handler invoked as normal.
FossilOrigin-Name: 79e24ec3dd40373bbb93792829b18d9ef40daf19d4606174e36c8e19e61a7529
2023-11-15 19:19:04 +00:00
drh 8c96d3c104 jsonb_remove() now works without having to use a JsonNode parse, assuming
that the input is JSONB.

FossilOrigin-Name: 5207679e929786e577a0553d0d84dda5125456dcde80c0f3156f14f4d8c804cb
2023-11-15 18:55:03 +00:00
drh af0c9ffb4a The jsonb_remove() routine now appears to be working.
FossilOrigin-Name: e76d48137ea823b7810dc8c3b70eb21adabdd6cfbac36050c85d1375e94be1de
2023-11-15 18:47:31 +00:00
drh 0399994759 Work toward getting jsonb_remove() to work directly on JSONB blobs.
FossilOrigin-Name: a79ff8e58fcaf718a6fb78e145117f1d6d40d133f31e9752bb9c6d484850a27b
2023-11-15 16:10:31 +00:00
drh 9eb7694d58 Improvements to the description of sqlite3_errmsg() and sqlite3_errstr().
FossilOrigin-Name: fe5cc4169c5348324e863d3e339de94ed01749a3b9c86aa9adc5e3244bd9d033
2023-11-15 14:09:37 +00:00
drh fb57c8a932 Merge all the latest enhancements and fixes from trunk into the jsonb branch.
FossilOrigin-Name: ba91408f4c044feda003ef93784ccefb619f99ab64379ced481ee8e9e890fd41
2023-11-15 13:23:40 +00:00
dan 50b0e25a55 Add implementation of xInstToken() API.
FossilOrigin-Name: a34b26fe7f60b74e7ae5cf64900920a3d352a20da2496401bcbc27041689cd07
2023-11-15 11:45:19 +00:00
stephan 5cc4ab93f2 Fix an exception misuse in test-opfs-vfs.js.
FossilOrigin-Name: 9264955e6e47aa8fc3a6f8bed192a6c12f43de49f7fba2e0cc080e47abedde14
2023-11-15 11:31:49 +00:00
stephan 1b963102b7 JNI: add more wrapper1 Sqlite.Blob tests.
FossilOrigin-Name: d63f6e1ffda9a88a37acc7862cb7ee8b9cd669b0fdba64c1ec9dd221cf807714
2023-11-15 08:59:53 +00:00
stephan bd06d8672d JNI: clear out the sqlite3_context native pointer after calling UDF callbacks which do not have an argv (as was already done for those which have an argv). Add related tests and code commentary.
FossilOrigin-Name: 138f40543b26b2e02e27d830d92e30b12cfef5a8dc3f0b58b39c68e1b3c91cc6
2023-11-15 08:29:42 +00:00
stephan 4d6b10cdbe JNI doc updates.
FossilOrigin-Name: 1b1f36a206319e99ccaed969893ff95dcf3b8e97ed301544cf3cd3fee2780335
2023-11-15 06:28:51 +00:00
stephan 6b36312e8a JNI: add the @Experimental annotation and mark all java.nio.ByteBuffer-related methods with it.
FossilOrigin-Name: 0f4b223102e5dc9142c9d2cb8892b8d3cc476e579420028b93d4e12f4cf94d3e
2023-11-15 06:10:37 +00:00
stephan 973011432b JNI test code cleanups.
FossilOrigin-Name: 09142ac14347e6f41bbe50bc835920e271713452733a478ede547816cc291ace
2023-11-15 05:08:39 +00:00
stephan d7ef0e8f87 JNI: add CONFIG_LOG and CONFIG_SQLLOG support to wrapper1. Code-adjacent cleanups.
FossilOrigin-Name: 83c49b9e71e5ae8852bab60a6fa630e22164c8efbf074c85450136781d0fffd3
2023-11-15 04:55:38 +00:00
stephan 253727b33d JNI: apply an internal level of API indirection to avoid having to work with massively-mangled names for the various sqlite3_config() overloads.
FossilOrigin-Name: 24f20663b1c4e13ff7d9a89e1df6af2e6f5730e5042ae6479e0a8c1bb846f313
2023-11-15 03:02:11 +00:00
drh 7df044633d Add NEVER() to a branch made unreachable by [6f9eed826f5b3d1c].
FossilOrigin-Name: 26dcaa34e3521783bde687749d590eca54cecb4bb95ef332a4a6473c4141d052
2023-11-14 16:53:52 +00:00
stephan 5d59669711 JNI: reimplement Tester2.execSql() using the high-level API.
FossilOrigin-Name: 3a69679e41d23a760df349b2471a1ed306c087f9a7a7a2e2cba3723071dee6b2
2023-11-14 14:59:41 +00:00
dan d928e78088 Fix another obscure problem with nested aggregates. dbsqlfuzz 04408efc51.
FossilOrigin-Name: 6f9eed826f5b3d1cb08402925b406a7fe9b54259af1ba5eba92d4d37fbad628a
2023-11-14 14:50:34 +00:00
stephan cb3a0b1b67 JNI: add Sqlite.Stmt.step(boolean) overload which returns the result code of sqlite3_step() unless passed true, in which case it throws for any result other than ROW or DONE. The intent is to simplify handling of LOCKED and BUSY errors.
FossilOrigin-Name: f4f1cc201a7dc618a66617778ecf58ba60461ae700cb41f5c4abfa26dd0c0ed4
2023-11-14 14:38:11 +00:00
drh 34ccd18d65 Extra PRAGMA vdbe_addoptrace output showing when OP_Column gets translated
into OP_Copy due to using a coroutine instead of a cursor.

FossilOrigin-Name: eba180e77977acd6f3d4a0e3c99a2ebd4ce2839e9ce30c588929e36d69aa762a
2023-11-14 13:51:19 +00:00
stephan 4ce5bc2836 JNI: use ByteBuffer.limit() instead of ByteBuffer.capacity() when figuring out where the logical end of a ByteBuffer is, for reasons explained at length in new code comments. This is unfortunately slower but is the correct way to do it.
FossilOrigin-Name: 51539419edc08ee6c70d8719d0f4d5ad47dd545a7fd9bf01d03a434aabd41d68
2023-11-14 05:33:44 +00:00
stephan bdfc51dfef JNI: add sqlite3_blob_read_nio_buffer() and iron out the blob/ByteBuffer interface somewhat.
FossilOrigin-Name: 7df317b448a09ae77e2c68cc901fdb6d56a2246c1313f06bebd1f3e53f02c19b
2023-11-14 04:59:57 +00:00
larrybr f695620e46 Add a few asserts into *Put*() functions to catch calls that might cause an address fault.
FossilOrigin-Name: 2341f9b6a48634a94e90e849f579afd07d398c3e0304efccf07d18b41a0e40aa
2023-11-14 03:24:43 +00:00
stephan adcd13d243 JNI: change sqlite3_prepare_multi()'s exception-handling semantics to be more C-like and, to support that, add the package-private sqlite3_jni_db_error() method to set the db error state from package-level Java code.
FossilOrigin-Name: 46656b354311ec0a36832af1c4ccb3b6a244aa55cfb3681e25c3f42b13b387dd
2023-11-14 02:43:30 +00:00
stephan b481413d95 JNI: add sqlite3_blob_write() overload which accepts a java.nio.ByteBuffer. Cleanups in adjacent code.
FossilOrigin-Name: ca32af8542aa2725cc87f54541b19897556f610e4674edf9f22a84e3d4097a82
2023-11-14 01:33:15 +00:00
stephan ce2edab088 JNI: add sqlite3_column_nio_buffer() and sqlite3_value_nio_buffer() using an only-slightly roundabout approach to creating properly-sized ByteBuffer objects.
FossilOrigin-Name: efbc82b218d26b7ca9b881da69d5fd14d22b5211fbd85a835da50e5bfde3d160
2023-11-13 23:11:10 +00:00
stephan 87c407da18 JNI: add sqlite3_result_nio_buffer() and tests. Discover that we cannot create sensible sqlite3_column_nio_buffer() or sqlite3_value_nio_buffer() counterparts because of ByteBuffer interface limitations.
FossilOrigin-Name: 44b4df01ff86841fb85b6295cbada422c6ba8a32a420a2e840e2d607b6c90164
2023-11-13 18:35:37 +00:00
larrybr 5ed4ba0f7d Cure many warnings from gcc, clang and CL.exe.
FossilOrigin-Name: 29ea2a3aadd34facc45a2e9f9a567bac16fca76beb019b879e2611433c85bbb3
2023-11-13 15:59:27 +00:00
stephan 7f7d7bea01 JNI: add sqlite3_bind_nio_buffer() and initial tests for binding ByteBuffer objects as blobs on JVMs which have JNI support for nio buffers.
FossilOrigin-Name: b10ce1ef82d84726fbf6a8f624d6530f84fefb505f7868b4a0ea910fed7a877f
2023-11-13 14:58:37 +00:00
dan e108029332 Add new fts5 API xQueryToken().
FossilOrigin-Name: 828566392b3ea8db603cb1ae5eccbc8ac035efaa284bc7c15ba89874f634aec9
2023-11-13 14:29:12 +00:00
larrybr 41d4b7bdee Fix var-intro after executable code departure from old C.
FossilOrigin-Name: 08996f4fd52f5742e77225ce3da2ac43ea5cc44bb0d8d21126b6d624273842a5
2023-11-13 13:01:01 +00:00
larrybr c270dad95d Slight cleanup of length-limited strpbrk() replacement, and eliminate strpbrk() call.
FossilOrigin-Name: 97e2c9621e3e01048e33da63ca4146ededa33cf6adefa996fa49b8c6a8d555b5
2023-11-13 12:53:47 +00:00
larrybr 14a08730de Enhance console_io to permit emits limited in various ways, such as valid UTF-8, upto control chars, or with counted limits, all getting away from 0-termination as the sole limit. In CLI, use this capability to avoid certain emit-chars-singly procedures that were breaking up UTF-8 characters. This fixes broken json mode output (on Windows) and (maybe) C-literal-like emits.
FossilOrigin-Name: 906c5c4082e30b7a0d07df89a42566461e2113507c5a0b339827ca0822b8fe84
2023-11-13 05:24:00 +00:00
larrybr fdbd9119d4 Add sqlite3_x86.exe unversioned binary.
FossilOrigin-Name: a731cdddbb99dbd3f9d1875cad5094239b78969c84fe4c56ecd63e33a5874e3f
2023-11-12 19:57:23 +00:00
larrybr 782c9f2e9e Use setOutputStream() to designate implicit output for oput{z,f}() emit functions, and use them extensively.
FossilOrigin-Name: 7850fb98a19d0ae3535367de3bca9e50408a2c21504c5772947fc39f287aa830
2023-11-12 03:58:15 +00:00
larrybr cc2b3c2051 Write BOM without fwrite(), using sputz() so that if it goes to the console, it is translated "properly".
FossilOrigin-Name: 06ef96a8233c6b6ca2653a0b780b3a401cab2b77594125f4f1c54b269c22a9f9
2023-11-12 00:43:36 +00:00
larrybr ea80462c10 Remove SHELL_LEGACY_CONSOLE_IO PP symbol and code it made active. (It is in the repo if ever needed/wanted, but it is just inferior or dead code now.)
FossilOrigin-Name: 27c5bf6563bc24ba7b47865e8d2f3e2d439666e19038d86dd8445fcdb9abe97a
2023-11-11 22:53:55 +00:00
larrybr 1bcb7c4902 Complete shell transition to using {f,o,e}put{f,z}() emit functions. This fails test 13.1 in json501.test, but so does trunk in the same way.
FossilOrigin-Name: 923c6b8b3a508c715b816c6bcd2ae9ac519bc37a62afc4ef813085c00f1e7cb6
2023-11-11 20:46:12 +00:00
stephan 56e1610f7a JNI wrapper1: when checking for an out-of-bounds statement column index, perform the is-statement-finalized check before the range check so that the former exception trumps the latter.
FossilOrigin-Name: 0832f9a8e9f574b157c791c5cddc73aff7b2ff403509f5d78f310494d4a7f93d
2023-11-11 14:50:01 +00:00
stephan 307f95c1e4 Do not cache a statement's column count in the JNI wrapper1 API because an ALTER TABLE via another statement may invalidate it, as reported in [forum:6d80efd58d4591c7|forum post 6d80efd58d4591c7].
FossilOrigin-Name: a6ab88e9a67f23ab7885402776282b94033cb48dbe34d4d18356e4dc22aae7cd
2023-11-11 14:43:50 +00:00
larrybr d2c8658024 Fix malf with redirected input due to bad assumption that stdin is a console in the fgets() replacement.
FossilOrigin-Name: 79d1f2c1019964dd154fbdd3f349822cb946a2600883994523ed145047f0a9ea
2023-11-11 13:09:09 +00:00
larrybr 1229f8e6bd Pervasive changes to console_io.{c,h} in support of simplifying ubiquitous emit ops in shell, and to get better control of console streams that might be opened only via .read or .output commands. Changes to shell to use {s,o,e}put{f,z}(...) calls for initial testing, but this check-in has few such conversions so that most will be in a separate check-in. Many renames to better follow recent coding convention. This code seems to be working, but has not been tested on difficult platforms or with multiple console hosts yet. So it is a WIP.
FossilOrigin-Name: 14762a004cdf37d1e12f26aadff8ed3824893278f22ff141de86dd44d9b250f3
2023-11-11 06:20:38 +00:00
dan fb28a93f75 Fix another problem with mixed join types and the RIGHT JOIN strength-reduction optimization. [forum:/forumpost/befdab472d | Forum post befdab472d].
FossilOrigin-Name: f1eae192315335d7e385b0a801a17700a9718d245bda6628518c5df9a1e9d3d6
2023-11-10 20:55:20 +00:00
drh 88dcfe56da Additional debugging information on the tree-dump of the BETWEEN operator.
FossilOrigin-Name: aca31e49d1d25043769544ccf2a07980c5f162a8eb2486e393bf9d9d1a394a60
2023-11-10 20:46:58 +00:00
drh 10c815a55d Remove an incorrect ALWAYS() that was inserted yesterday [1e039b6eb59c0001].
FossilOrigin-Name: 12885e298b9d3f977f1de11a194692dfb5fbb7daeabd958674f884a5575ddd24
2023-11-10 20:35:59 +00:00
drh e8d4fd59e4 Merge recent trunk enhancements and fixes into the jsonb branch.
FossilOrigin-Name: 091a5f058dfe2115fb9213655b34f00bcec80aebb299b571975cfe4ecd5ec206
2023-11-10 18:59:23 +00:00
drh 5a81e6e5ce Ensure 8-byte alignment of data structues in sqlite3_database_file_object().
This should have appeared on trunk originally and then be cherry-picked onto
the branch.  Oh well....

FossilOrigin-Name: ac39800bb2685fa287c7d834faed75f0bc61320ef986de314392d6eadb574d30
2023-11-10 17:49:26 +00:00
drh 53381132be Remove a NEVER() from whereAddIndexedExpr() that is reachable if there
is an unknown indexed function in the schema.

FossilOrigin-Name: a976b7208ff8603d7353ce9a0bdfba8e681cbb2ed3de6cfb5f0e8b07312ab86f
2023-11-10 16:29:02 +00:00
dan 0eed27d38b Fix an obscure problem with the join-strength-reduction optimization that could occur when mixing LEFT and RIGHT joins in the same query. [forum:/forumpost/7f74ce0bee|Forum post 7f74ce0bee].
FossilOrigin-Name: 530d10e93a5f63b71aaa94a2b89102d012a2cda815997066beb0f585fe823536
2023-11-10 15:03:18 +00:00
stephan 9d60c3c256 Expose the missing SQLITE_SUBTYPE to wasm.
FossilOrigin-Name: ac9534b2ceb8185b1fc03282f881cd3e4aea64af75a02ebded1e07d4d8278739
2023-11-10 15:00:26 +00:00
stephan 828a00c287 Expose the missing SQLITE_SUBTYPE to JNI.
FossilOrigin-Name: 0f92f4c90eb9397325f1a86836e356862108e2e850c1801e0bec4a7030dea271
2023-11-10 15:00:11 +00:00
stephan af65e7d21f Expose SQLITE_RESULT_SUBTYPE to JNI.
FossilOrigin-Name: 3bf75875b8be8d2e878681506fb35f2062d59f07fa23f571c58cd92f270bb197
2023-11-10 14:15:07 +00:00
stephan 6db0b11e07 Expose SQLITE_RESULT_SUBTYPE to wasm.
FossilOrigin-Name: 6d2fe9848beb35a8206e49c2ffae29a3eb2fe6411d77f366e962ced3f83e4749
2023-11-09 17:53:44 +00:00
drh 3839dcf973 Add the SQLITE_RESULT_SUBTYPE flag for application-defined functions. Add
the -DSQLITE_STRICT_SUBTYPE=1 compile-time option that raises an error if
any function invokes sqlite3_result_subtype() without the SQLITE_RESULT_SUBTYPE
flag.  SQLITE_RESULT_SUBTYPE prevents an indexed value of that function from
being used to replace an equivalent expression, since the indexed expression
does not carry the subtype.  Fix for the problem described at
[forum:/forumpost/68d284c86b082c3e|forum post 68d284c86b082c3e].

FossilOrigin-Name: ba789a7804ab96d81b15d6ef6fed1f802fa69db47cf91d368933e55289fa1d6e
2023-11-09 17:36:37 +00:00
mistachkin 2009a5acba Fix compilation issue seen with MSVC.
FossilOrigin-Name: 0dfe790d8118ff1bacc9e7c97d4f8ff9e5789f6cda8ec6fd981ea38b4da3905d
2023-11-09 17:28:31 +00:00
drh beb06e6b0a Put an ALWAYS on an true branch.
FossilOrigin-Name: 1e039b6eb59c0001a9efdd2f9928a34d4e9e01972ee76aa04a1279369dc03840
2023-11-09 17:26:39 +00:00
drh cb1f190ed2 Futher documentation refinements.
FossilOrigin-Name: 311c2eba93097bbecfa286bbeaff9bd6fc75a238e20cd2b6f834e594032d8c59
2023-11-09 16:52:44 +00:00
drh b10c3d32e0 Fixes: (1) In the ->> function, instead of setting a subtype and clearing it,
do not set it in the first place, as doing the set would trigger an error
under SQLITE_STRICT_SUBTYPE.  (2) Allow the SQLITE_STRICT_SUBTYPE through
the property filter on sqlite3_create_function().

FossilOrigin-Name: 6195468b14f6f17ea072cf191c9ef1bd0713acd314bc6dc128be7322bfd612cc
2023-11-09 15:01:56 +00:00
stephan 752722e8d1 A .class file build dependencies fix in the JNI build.
FossilOrigin-Name: 8cc32915165efd7c261f008bb6fa4cc6581ee7bd73ea5da47513742b9e6d34e4
2023-11-09 13:00:33 +00:00
drh 6eb381ff4a Add the SQLITE_STRICT_SUBTYPE compile-time option. This change reveals that
the current SQLITE_RESULT_SUBTYPE design does not work unless we tag the ->>
operator with SQLITE_RESULT_SUBTYPE.  But that will disable an important
optimization.

FossilOrigin-Name: e98a9a65dd309f72c240e280c7bebabc58af664fae9ee0d30c3fa1c78db5bae9
2023-11-09 12:58:03 +00:00
stephan b9050dcec7 Add some notes about the JNI pointer-passing approach and convert a couple of potential NullPointerExceptions into appropriate C result codes. Clarify that invocation of undefined behaviour from the Java API does not (due to the addition of defensive code) mean the same thing as it does in C (e.g. no NULL pointer dereferences).
FossilOrigin-Name: 19c4778f45261006368b2d9460350fed1e55fed314c8b3e1af34cd8c3c73b7d8
2023-11-09 12:48:54 +00:00
drh b18bb822dc Do not replace expressions that return subtypes with values taken from an
index.

FossilOrigin-Name: a35d13db09e32ee339f3983fe36b073714753ee3d39f577ae8d20596d7adc3eb
2023-11-09 12:17:57 +00:00
drh 194b8d514b Omit the new SQLITE_VALUE_SUBTYPE name. Stay with legacy SQLTIE_SUBTYPE.
Add extra documentation to sqlite3_value_subtype() and sqlite3_result_subtype()
indicating that the SQLITE_SUBTYPE and SQLITE_RESULT_SUBTYPE properties are
required on functions that use those interfaces.

FossilOrigin-Name: 563ad3be60d22c45f1c5b9a3e67738593f8b38f137147c56514166fbabf95365
2023-11-09 12:08:16 +00:00
stephan ec427813ac Two more JNI build fixes for Windows/MinGW, reported in [forum:4f949edc312d2a75|forum post 4f949edc312d2a75].
FossilOrigin-Name: a3f9c39086e582e16ca15647961956b3c28d038655d3b43d4b94bd306fbec1a4
2023-11-09 12:01:02 +00:00
drh c060b5f3a8 JSON5 bug fix: Escape double-quotes that occur inside of single-quoted strings.
[forum:/forumpost/ddcad3e884|Forum post ddcad3e884].

FossilOrigin-Name: 1c98d46d60ef1494bd8b7561c7d0cd5aafc178201a6f1f0da25dea6140b91cd0
2023-11-09 01:54:26 +00:00
drh 243f2ec6a1 Make a distinction between functions that consume subtypes and functions
that generate subtypes.

FossilOrigin-Name: 48a92e3ad855227188a4c5afe4abbb7171761cf6fc930660084d9abeecfd91d9
2023-11-08 21:38:30 +00:00
drh 2cbe14098b Do not cover expressions using an indexed expression if the indexed expression
is a function that might set a subtype.

FossilOrigin-Name: e908b26a990929996b3c16f0429e8313cd8fcefe7c883c77f66ea69f4059d6e2
2023-11-08 18:08:07 +00:00
drh a4cf38ca9b Merge recent trunk enhancements into the jsonb branch, and especially the
finer-grain characterization of JSON function properties.

FossilOrigin-Name: 72393b003f9f8675e4a124dddd09607b5b34ddefe56716b283c68c0982fb3d96
2023-11-08 17:11:13 +00:00
drh b494366370 More precise characterization of JSON functions. Indicate when functions might
return JSON (subtype 'J') and when they make use of the function argument cache.

FossilOrigin-Name: b2b62546c4a5e9dccb8aa0cb8eda228d662c69159e320b01a377317bc909e89f
2023-11-08 16:37:12 +00:00
dan 6234b33478 Have the shell tool emit a warning if the user attempts to use ".scanstats vm" in a non-SQLITE_ENABLE_BYTECODE_VTAB build.
FossilOrigin-Name: 3978c084a509c3c739fbe87e20feec9ddf1325e35170329987af197ca9fd731a
2023-11-08 15:56:41 +00:00
dan e445586630 Remove old files related to wapptest.tcl from test/ directory.
FossilOrigin-Name: dd3e7b5bcad122ac1e7e19ec547f4486ce90a6a2aa89a64e36bea13a216492fe
2023-11-08 15:51:42 +00:00
dan 0ad5301378 Avoid blocking as part of passive checkpoint operations, even if SQLITE_ENABLE_SETLK_TIMEOUT is defined.
FossilOrigin-Name: e5ecc404cae1ce8b639d0263fa07571c066f11bfc62f5ba331ad7ae138e78572
2023-11-08 15:49:57 +00:00
stephan 8657eddcdb Remove an unused/invalid test from the wasm suite.
FossilOrigin-Name: 916ae898743a969295a48ae2a6e9e366586834b32d77d3fa281bbaf7f2818502
2023-11-08 15:34:03 +00:00
dan 43d05ccc31 Add declarations for new API functions.
FossilOrigin-Name: b8a48cc18c94d15017f898c820fdd784efbaac20d7a45c4d97269333e8f2ec60
2023-11-08 14:55:20 +00:00
stephan 86b1898cb4 JNI build fixes for platforms where the jint type is not the same as int, as reported in [forum:9089d2049a|forum post 9089d2049a].
FossilOrigin-Name: b32b0873274bfe472114da8a308a04bee76ba26a5830d8d04fc921f9c1544f9d
2023-11-08 12:56:23 +00:00
drh c78d3b5963 Suppress harmless UBSAN warnings about memory overflow in OP_AddImm. The
exact same machine code is generated by GCC.

FossilOrigin-Name: b0594383b9fa021a8713d640a4606b9053f8e21d64b4ec8ea60a0b6cddfca306
2023-11-08 00:45:14 +00:00
drh ec8f893e06 Changes a no-op call to freeP4() into an assert().
FossilOrigin-Name: 32a7b1bd4d88a6839c2c5061a38b4a28f31b22aa8fa08c743633ff96fc4bf88d
2023-11-08 00:12:38 +00:00
dan d7b0b5ec6f Add -DSQLITE_ENABLE_SETLK_TIMEOUT=1 to a release-test configuration.
FossilOrigin-Name: cd3e38fb0f2f7e134bb6cfbe86b6621017344e4361dc6a09ec1367860ba8773b
2023-11-07 20:56:29 +00:00
dan 275234e320 Fix an assert() that could fail within calls to sqlite3_snapshot_open() in SQLITE_ENABLE_SETLK_TIMEOUT builds.
FossilOrigin-Name: 84634bc268e5c80146f3f3b2e13118f239c9a7e4e4e9dfcaccef2b17252ce53b
2023-11-07 20:11:49 +00:00
stephan 344255e5aa An attempt to work around compilation errors on MinGW reported in [forum:9089d2049a|forum post 9089d2049a].
FossilOrigin-Name: d5658a3ee4f585cc1e96d84425aad2e1f56b3ae507c9e17b71e030f83f0b70e8
2023-11-07 19:39:23 +00:00
larrybr 73f03276d9 Refactor MBCS/UTF-8 translation to avoid extra allocations, supporting non-formatted (faster) output. Some code cleanup. Wrap .system/.shell command exection with restoration of startup console mode and renewing mode setup. Changes to make legacy MBCS build work better (than legacy did, even with --no-utf8.)
FossilOrigin-Name: d5e88fcde53ca7ba05bb164943a9f57bd92080bb7e5eebbbed64b9886ac97338
2023-11-07 19:30:14 +00:00
drh 17a3295387 Update the documentation to the sqlite3_set_auxdata() and sqlite3_get_auxdata()
routines to make it clear that they do not work as one might expect when they
are called during query planning, instead of during query execution.  The JSON
routines misuse those interfaces, so add a special flag to JSON routines that
prevents them from being invoked during query planning.  Fix for the problem
in [forum:/forumpost/a655ee159eca1ea5|forum post a655ee159eca1ea5].

FossilOrigin-Name: 796a23f9ee33da0803844a2f40c1733db894cc4ef7fbaa1fa94af6af2d3b873b
2023-11-07 19:03:13 +00:00
stephan 637922acac JNI: during static init record whether the current JVM supports JNI-level access to java.nio.ByteBuffer raw memory, and add sqlite3_jni_supports_nio() to query that.
FossilOrigin-Name: fb8dbb77a4d8efafd6772333824b4ab589828cf155a63ca6a26730314d0a4bd9
2023-11-07 17:15:55 +00:00
stephan 488125d4ce Add Sqlite.prepareMulti() to JNI wrapper1, for preparing multiple statements from a single input.
FossilOrigin-Name: e4670d68b52233ab376a1725983e148aaf2a2c3658a41f5768e37a0f1f87428a
2023-11-07 15:56:39 +00:00
stephan 6dc368e632 Diverse minor cleanups in the JNI pieces.
FossilOrigin-Name: 35233dd900632b997b5e532170a3b2af0ca7f1dccb8407555b93f2b395b0f7b4
2023-11-07 13:44:29 +00:00
stephan 6ca0cfd87a Flesh out [7a63b5b65a79] to be able to build JNI with or without SQLITE_ENABLE_COLUMN_METADATA.
FossilOrigin-Name: fcee41b3d4d2558299ead28cc17f290b9ff1957a84c3feaa0a24872feeb22901
2023-11-07 13:22:49 +00:00
larrybr 4c78cb50bf Get dependencies into make recipes. Get legacy console I/O (-DSHELL_LEGACY_CONSOLE_IO) working. Due to movement of MBCS/UTF-8 translation into traditional stream I/O simulacra, the input translation does not happen the same way. (It works the same, but fails differently and a bit better.) Added printf() and fputs() look-alikes, and made CLI use them.
FossilOrigin-Name: 1721dc6a434361c4e2b87c6e677b6dc223432b3cdd5b9eecabaa258889fb2d2a
2023-11-07 02:41:46 +00:00
stephan 23056b608e Add -DSQLITE_ENABLE_COLUMN_METADATA to the JNI build, as per [forum:9205518c0568fdf0|forum post 9205518c0568fdf0]. Add tests for the functions that flag enables so that the build will fail if that flag is missing.
FossilOrigin-Name: 7a63b5b65a79d15658a160d0878c7371941c67e9b48a7442762c68c60b77288a
2023-11-06 21:57:15 +00:00
dan b4fc827c0f Merge latest trunk into this branch.
FossilOrigin-Name: 3a869cf1f84b0e9bdcc4de53685430ab41eafacbba1ca7b87e727aa98811c6c5
2023-11-06 19:16:38 +00:00
dan a3e8822c4b Fix an fts5 problem where a transaction consisting of (a) a DELETE on rowid X, (b) a prefix query, and (c) an INSERT on rowid X, could corrupt the index.
FossilOrigin-Name: c2058a045b57571b2b5d342adb212fe606717c633a0422755691ae6bf5725d25
2023-11-06 18:40:25 +00:00
dan 4cad31912e Merge trunk changes into this branch.
FossilOrigin-Name: ac5570614ebb77feff4943c7a78d877508e31087b550f337c11b45f1016e3c55
2023-11-06 16:32:50 +00:00
larrybr d0046388a0 Move console_io lib to its own subdirectory, etc/consio .
FossilOrigin-Name: 1d0583f2eb69fdca1cbc55763c0e86a7e32cb7771bfbc2cdf02da4e3fedbfa23
2023-11-06 15:31:08 +00:00
larrybr eb1898d6f3 Some renaming, warnings cured, and more coding convention conformance.
FossilOrigin-Name: 58815f0ad259599a674e7d0fe0d4676a1c4fb8e118c08b619580ff655fb3be40
2023-11-06 15:15:58 +00:00
larrybr 56fba47850 Add an fputs() equivalent to console I/O lib, and use in CLI.
FossilOrigin-Name: d661f90724b1bd31948169841bacb80ba4ae91b52191a0b0021a89a22d932a02
2023-11-06 13:24:07 +00:00
larrybr e3a6a60901 Cause CLI to use console_io library.
FossilOrigin-Name: bf66a7c1d330b04fd3c3e50f43ebe1e41307db59d7189798acafb3de77e0c8b2
2023-11-06 03:09:10 +00:00
larrybr a0cd392979 Make MSVC accept it, too. (Cannot use static const int as part of a constant expression?)
FossilOrigin-Name: bb278d2496b27d2e2ee3cedd6fc54394e71ab2ba5d3d51593f97e897b8b306a4
2023-11-06 00:15:14 +00:00
larrybr 8dd4697e48 Make it buildable. Pull test program from testee. Zap stray fprintf().
FossilOrigin-Name: 2b850aca1e76805a0358064318a765fa66ce394d015936fd47683d74ca4c187e
2023-11-05 23:55:41 +00:00
larrybr ff39634625 Input working. No line-editor provisions yet. (WIP, but suitable for testing.)
FossilOrigin-Name: e8568b1d925c2118eb08394dd8aa50cfb521240f87668f535ec4a03e67dc9a09
2023-11-05 19:42:00 +00:00
stephan ed99e7493a Add incremental blob I/O support to JNI wrapper1.
FossilOrigin-Name: 7f1c76fe930d69a0274f70fa7b7e68e0db6226b731a065fa57d0936c8400ffb0
2023-11-05 04:20:04 +00:00
stephan 546db3f14a JNI wrapper1 normalizeSql() now throws UnsupportedOperationException, instead of returning null, if built without SQLITE_ENABLE_NORMALIZE. Remove SQLITE_PREPARE_NORMALIZE from the JNI interface because it's a legacy no-op.
FossilOrigin-Name: d081a126697e214082f3b203f23ea63510080e5c2aac1d8badc9e6e4bfea7c95
2023-11-05 03:37:33 +00:00
stephan 88c90a793e Add authorizer support to JNI wrapper1.
FossilOrigin-Name: 773f9873865b5277a6a682c4695f216bfe1ec05ed5e5a2a70aaa451934ba2dc0
2023-11-05 01:55:20 +00:00
stephan 9a26589940 Add progress-handler support to JNI wrapper1. Correct the return type of the extended_result_codes() JNI binding and expose it to wrapper1.
FossilOrigin-Name: 6c0acfdce2160d8db261a59677cec571b6abc333481525b1ec975d98e88bec88
2023-11-05 01:39:29 +00:00
larrybr 557297ae77 Setup, takedown, mode set and output working. No input yet. (WIP)
FossilOrigin-Name: dfea85be1fb927ea446c9d98bae42ba1197bdab098aa6d95aa512a37d07a1e52
2023-11-05 01:21:14 +00:00
stephan 0ee7ae72a1 Add update-hook support to JNI wrapper1.
FossilOrigin-Name: 6c584cf27179d16deee84e9699493cf29bebef123fa2a7493aad0324bead1618
2023-11-05 01:14:07 +00:00
stephan 2c930e3b86 Add commit/rollback hook support to JNI wrapper1.
FossilOrigin-Name: ff3d44fe42528d96533d22c7807472df89bca18f1def23b018e2f407318143f8
2023-11-05 00:48:43 +00:00
stephan b02ca781ad Add busy-handler support to JNI wrapper1.
FossilOrigin-Name: dcf579ab2de4a3d3a437cde59b2fd60f1729c0bde31df1865117e6a5ea4bab20
2023-11-05 00:02:47 +00:00
stephan 15d38c0dde Bind collation and collation-needed to JNI wrapper1 and correct the callback return type for collation-needed callbacks in the lower-level JNI binding.
FossilOrigin-Name: 0f673140681685ab390ecd7326a8b80d060b7ab23c31a2cfc28ba76fd5096afe
2023-11-04 23:37:11 +00:00
stephan dc8a684c11 Wrap the sqlite3_backup API in the JNI wrapper1 API.
FossilOrigin-Name: 3ee6cc29d2111e7ad90860827c0ea808fdf07bc71defdade7e6794ec4a2a3ce2
2023-11-04 22:47:40 +00:00
stephan ffdb479e7c Reimplement auto-extensions in Java for use with the JNI wrapper1 API.
FossilOrigin-Name: 14ed4c64533622e5faf1aaa59c24885885aad43f1c0d4717773e79440e8e1468
2023-11-04 21:51:34 +00:00
drh d4677f192f Ensure that the YYYY-MM-DD input to date and time functions has been normalized
prior to returning a result.
[forum:/forumpost/6bb476897e|Forum post 6bb476897e].

FossilOrigin-Name: b692eb8ccb2d0645599ad73a8bdacf5df499114244aadeb38aabc580fc4dc7c5
2023-11-04 21:44:00 +00:00
stephan 1b199243cb Expose context_db_handle() to the JNI wrapper1 API and clean up some related tests.
FossilOrigin-Name: c23123af7d40dea24a0848dff987fd58a6703ce04165060533544db85983d566
2023-11-04 16:01:13 +00:00
stephan 348e192ea8 Rework the JNI wrapper1 variants of status() and db_status() to be more Java-esque.
FossilOrigin-Name: 40ad3920673561a06edf0b70a50a40be6cd20817fe22b87b63a9ac80cb2c9df8
2023-11-04 13:37:42 +00:00
stephan 5f3b13d136 Bind the bool-flag sqlite3_db_config() variants to the JNI wrapper1 API.
FossilOrigin-Name: b5cdcb9279d9276f24b67083839f463beecd731f46f2e8bf68fff716df0a3921
2023-11-04 13:16:49 +00:00
stephan aad6808efc Bind the trace API to the JNI wrapper1 API and add a way to map the native-level db/stmt types to their high-level counterparts (required for translating callbacks such as tracers).
FossilOrigin-Name: 702910e0d1cfc897a269b4fb36b255165958edf529ac9553ebc5155e404d4cd3
2023-11-04 12:53:00 +00:00
larrybr a501791d59 Define interface between project command-line apps and a console I/O "library".
FossilOrigin-Name: 64abef8314b8544fdc7b71317d61a4641dc607a1ae42b8ff21543226fd338ba2
2023-11-04 02:22:04 +00:00
drh 5189ef98d3 Back out the ALWAYS inserted late yesterday. The fuzzer discovered a
counter-example.

FossilOrigin-Name: 570635575cc5fbffe910ed992b58393e214117ef3b5370a66f115cd0ee202913
2023-11-03 18:45:26 +00:00
dan ab8a4b2bb0 Fix a case where adjacent tokens are handled incorrectly by the fts5 snippet() function.
FossilOrigin-Name: 8f5e9c192ff2820d8cfb076ab28f30697d10c22710583d6c7fd7019c4a0ea795
2023-11-03 17:20:20 +00:00
stephan b945183195 Wrap more of the stmt API behind the JNI wrapper1 API.
FossilOrigin-Name: 8fea23dc3af023ccf2909f1b4c6f91e7df0ffaac875b15f1fb3e264fba169b6a
2023-11-03 13:00:51 +00:00
drh 7b5123c796 Enhance the JSONB lookup routine with logic to apply edits. The new logic is
currently unused and hence untested but does not create any regressions.

FossilOrigin-Name: b12110276fc15d1b6b0cc455f89747ace7a09650d5ba433d8bb431bb4e5bc951
2023-11-03 12:09:22 +00:00
stephan 073f8f5458 Bind the sqlite3_keyword_...() and sqlite3_compileoption_...() family of functions to the JNI wrapper1 API.
FossilOrigin-Name: b27242414d6023eac7e62cf6120e1f02b0ddc7b8f0a1e4c48111cfe88d197cbd
2023-11-03 11:53:13 +00:00
drh 1d5f2af063 Update the tool/srctree-check.tcl script so that it verifies that autoconf
has been run following edits to VERSION.

FossilOrigin-Name: 52ab3deba4d26ca0e9a84a6dff254195b4a0e2cc6cf948cf7a66bb11117e7002
2023-11-03 11:41:24 +00:00
drh 24f7f5923d Merge all the latest trunk fixes and enhancements into the jsonb branch.
FossilOrigin-Name: b089bf46374b374d02d7654c65eb3e75d1777638b398061e47644af0dab48c9b
2023-11-03 11:35:33 +00:00
drh d706adba48 Update the version number to 3.45.0 to begin the next development cycle.
FossilOrigin-Name: 15b618e92a2708cc83256947736de8c494a9985a77e38bd5efc7e51e72cba344
2023-11-03 10:15:16 +00:00
drh dcd6d21fe9 Mark a branch made always-true by the penultimate check-in with ALWAYS().
FossilOrigin-Name: 268b5984a4263bee245a9bb47ac927bde56cdf4af8795b851dada5622224076f
2023-11-02 22:39:11 +00:00
drh 792103a0f2 In the previous check-in, use a u32 instead of an int, to make it easier to
prove that the integer will never overflow.

FossilOrigin-Name: 51002079e0b499e116415189a913a238e95ed9766a8af9d042928f4263861a87
2023-11-02 22:11:35 +00:00
dan 5e4233a9e4 Fix a spurious "misuse of aggregate function" error that could occur when an aggregate function was used within the FROM clause of a sub-select of the select that owns the aggregate. e.g. "SELECT (SELECT x FROM (SELECT sum(t1.a) AS x)) FROM t1". [forum:/forumpost/c9970a37ed | Forum post c9970a37ed].
FossilOrigin-Name: 4470f657d2069972d02a00983252dec1f814d90c0d8d0906e320e955111e8c11
2023-11-02 21:02:53 +00:00
dan 78fee3f738 Add the "remove_diacritics" option to the fts5 trigram tokenizer.
FossilOrigin-Name: 0d50172477064dce3f7c61641d88aa15d8a1edea9872adaffcc1a7a1c9ab0e93
2023-11-02 18:22:35 +00:00
dan 7b0fd0c564 Fix a problem with amalgamation builds on this branch.
FossilOrigin-Name: 8f046c82c9cf51fc349674577c68d3d2499ee37009deacbf937d711d9930fd49
2023-11-02 18:10:22 +00:00
stephan 06af40b2e1 Merge latest JNI pieces into trunk.
FossilOrigin-Name: e8c97faec339430fa2d1a7e915050d9b363bdba606a4419397cb099141da254d
2023-11-02 18:08:49 +00:00
dan e186fe20f5 Add the "remove_diacritics" option to the fts5 trigram tokenizer.
FossilOrigin-Name: 83da80135b6105f47d1de560232449562ae8ac176c8011a6f75589f62bc9b1db
2023-11-02 17:31:06 +00:00
drh 845a2e0cc2 One more tweak to tool/srctree-check.tcl so that a complete build can be
accomplished from a read-only source tree.

FossilOrigin-Name: cba9f0601ca995ac6952aa3a83f7264f6d25aaaa2ea36b19e90cbf591077de72
2023-11-02 13:10:16 +00:00
drh 8a93ce7458 Update the srctree-check.tcl script and child script so that they can be run
on a read-only source tree and so that if any inconsistencies are found, the
script returns a non-zero exit code and thus halts the build.

FossilOrigin-Name: 58eb5440d75fb13c1a089cb935de6fb94b4322e10e349db8f11f37b9a2fda2fc
2023-11-02 12:05:55 +00:00
larrybr 57a7fdd814 CLI to compile with older MSVC compiler.
FossilOrigin-Name: c89f75f8eef449f6869bda423593f7e14080e805f8ed192aff9463460f11b1a9
2023-11-02 11:04:37 +00:00
stephan 7d52262889 Remove some outdated JNI docs.
FossilOrigin-Name: 52aef3d015bad12c4bd9334ec795276a68a0dabdc6fd0ff2a1068f640171eb77
2023-11-01 20:09:02 +00:00
drh 0577cdec5f Update the windows build documentation to provide the exact command used
to build the DLLs that appear on the download page.

FossilOrigin-Name: 3524a8914c62f7987d948e5ef50cfdcb07254489e80e2257f38d0050c4baf450
2023-11-01 19:35:15 +00:00
dan 54e7a0f360 Fix testrunner.tcl so that it accepts a relative path as an argument to the --zipvfs switch.
FossilOrigin-Name: c1ed7ed02269e5b26142eb307e7889fc636672f101ea7fc192a2445e16a7dd6f
2023-11-01 14:33:44 +00:00
drh 2158864585 Bring the jsonb branch up-to-date with version 3.44.0 changes.
FossilOrigin-Name: a420a4f7ff76b0e9cf5f6d515ccfa31e526d58f4001a4015a367e2aa3c82091f
2023-11-01 14:12:07 +00:00
drh 89b4fcfb0a Version 3.44.0 for the bedrock branch.
FossilOrigin-Name: 827cf67fb37e2413069bf3c76aa054cc3b183845afbdfa118de948d3cf847054
2023-11-01 12:45:01 +00:00
drh 07a7cd3714 Version 3.44.0 for the wal2 branch
FossilOrigin-Name: 470152fd2847296e0fc325fdf098e49c888e99a2332fb45533f976ad85a67d8d
2023-11-01 12:41:25 +00:00
drh 66476771b7 Version 3.44.0
FossilOrigin-Name: 17129ba1ff7f0daf37100ee82d507aef7827cf38de1866e2633096ae6ad81301
2023-11-01 11:23:50 +00:00
drh 5ba99da869 Fix harmless compiler warnings in test code.
FossilOrigin-Name: d1895dd8f5757a339f619f22b29c8a739398ded673bb9c93f1b8eb8a4b38f510
2023-10-31 17:40:30 +00:00
dan ddd1c97500 Fix a problem with SQLITE_ENABLE_FTS5 builds of the non-amalgamation testfixture.exe in Makefile.msc.
FossilOrigin-Name: 544091cc0f5b7395a89226629267c858ec53f18095aadb3e9307ef8ea718176e
2023-10-31 17:39:10 +00:00
larrybr 739edb99ee Remedy CLI non-UTF8 handling detection flaw [forum:/forumpost/fa0e00d36db5fe0a|noted in a forum post].
FossilOrigin-Name: a4c7954b9380dfd3599e521f7fd40b8b556a7d0268198b302146311d20c3b162
2023-10-30 23:20:45 +00:00
larrybr 7545b0eab7 Remedy CLI non-UTF8 handling detection flaw [forum:/forumpost/fa0e00d36db5fe0a|noted in a forum post].
FossilOrigin-Name: 2666b80d2cf64a14d0482ee1ce0e8b2ff9e0c5bb21cd626470241a73a6555a3a
2023-10-30 23:04:27 +00:00
drh c667489332 Merge the latest 3.44.0 release candidate patches into the bedrock branch.
FossilOrigin-Name: 956b03a4627ea2b9ab7596515aaf076a9895262d9ee23e5248c18d1a75f035a6
2023-10-30 22:57:30 +00:00
drh 47e42cd143 Merge the latest 3.44.0 changes into wal2 branch.
FossilOrigin-Name: 4f65ae2df071d2064ff1494b918c1dee2147e174f79060cd031c25f1aff70a7e
2023-10-30 21:46:46 +00:00
drh 23f5f62d41 Update the autoconf/Makefile.msc so that it aligns with Makefile.msc.
FossilOrigin-Name: c4f724de279dae1afd10d435ad61ed07c1e4658288457c50e4f9b2f4521bbe12
2023-10-30 20:35:33 +00:00
larrybr cc2af7e68a Dodge a nuisance warning from MSVC in CLI use of Prepared Statement Scan Status. (Yes, C rules are fine.)
FossilOrigin-Name: 166c96d91dcc8c500a084d5703db103176ed5ec7d5f1334050cd66b3f1cd0e4f
2023-10-30 19:50:20 +00:00
drh f3f0bd2155 Add a cast to i64 for an integer in an sqlite3OsWrite() offset
calculation.  The cast is not strictly necessary, but it helps human readers
see that the code is correct.

FossilOrigin-Name: 7564ff1ba2c2fba89106d1aa06cc5379e752f119f22370f2f155f24cc698dec6
2023-10-30 19:03:17 +00:00
drh 383295eb17 Fix another comment typo. No code changes.
FossilOrigin-Name: 3203256c0285ef2fea810d59978603bfd7e38151cbc42e1baddf84b57c56f638
2023-10-30 18:48:03 +00:00
dan 2d0dd57dd5 Add FTS5 to default Makefile.msc builds. And UNKNOWN_SQL_FUNCTION and STMT_SCANSTATUS to default Makefile.msc shell builds.
FossilOrigin-Name: e41fde2fdb999b430561ac6d6ba150666ad1858e99c973f8dcd04a6934991032
2023-10-30 18:17:29 +00:00
drh aa36dcf074 Make the new sqlite3_get_clientdata() and sqlite3_set_clientdata() interfaces
available to run-time loaded extensions.

FossilOrigin-Name: 8fc9aa0f896a2676b73c41cefbff74c822a7106b56324558862dd0a2a1877eaf
2023-10-30 18:09:14 +00:00
dan a238c57eb8 Fix problems with previous commit on Makefile.msc. Also some win32-specific test failures in fts5misc.test.
FossilOrigin-Name: 527f2237a6fb5e2ac97afddd7c65c0afa971efc616cb3b738e7245ee0b8d9570
2023-10-30 18:07:18 +00:00
drh 30c9aca83e Fix typo in a comment. No code changes.
FossilOrigin-Name: 424ef9550abaab32f745dc955e8996d51836eaed737e24c4b4db811aa118ce5d
2023-10-30 17:56:27 +00:00
dan 23e8f976dd Add FTS5 to default Makefile.msc builds. And UNKNOWN_SQL_FUNCTION and STMT_SCANSTATUS to default Makefile.msc shell builds.
FossilOrigin-Name: bf99ebba23b6a72a0c02c5542574f409828343e89501cbe2112596ced063c6ab
2023-10-30 17:13:12 +00:00
larrybr 88d3c148c8 Predicate Windows CLI UTF-8 console I/O on a runtime capability check rather than an OS version check.
FossilOrigin-Name: 0058ea6a69103c1442bf5090553ce9bf77e06011a4960df754eb96144ced497d
2023-10-30 16:38:05 +00:00
larrybr 8a95e92af9 Predicate Windows CLI UTF-8 console I/O on a runtime capability check rather than an OS version check.
FossilOrigin-Name: f89d062f8890fffc957a354e966784031d561d0f8f5c174c1ccdcf77e66c32bd
2023-10-30 13:56:50 +00:00
drh d2147bd32b With SQLITE_ENABLE_BLOCK_ATOMIC_WRITE enabled, if a transaction is committing
and there is a new freelist page at the end of the database file which would
cause the database file size to grow, ensure that page is written and the
file size grows before the block-atomic-write commits.  Fix for the
problem identified by [forum:/forumpost/3bd8d497b2|forum post 3bd8d497b2]

FossilOrigin-Name: c9fdd6805df04f05ef347e5a43506fd37a729c5924abb6e1103e871c4ac2d6dc
2023-10-30 12:09:48 +00:00
larrybr 7c2d3e8a06 For Windows CLI, institute a version check to determine default MBCS or UTF-8 translation on console I/O. (Default to UTF-8 where known possible.)
FossilOrigin-Name: ddc6ead6453e0f98943bd07aedd90d47bc2e9e9e27b008d493491168bea2b3f1
2023-10-29 20:05:18 +00:00
larrybr 84eab13df9 Simplify code slightly. Improve comments on added code and its use.
FossilOrigin-Name: 046c84296627382ee416f64b02b77a937b368e30b32e6b800de5a854810766f6
2023-10-29 19:55:22 +00:00
larrybr 060c097e0d Properly close a handle. Use a putatively effective runtime test for UTF-8 console I/O capability. (This makes the version test useful mainly for avoiding a warning that UTF-8 console I/O could not be setup.)
FossilOrigin-Name: dc91eb91725f3db65c73725f1fbcf18a711cafb65b4fea3277aa0905a24df353
2023-10-29 16:26:12 +00:00
larrybr 3017a6851f Condition default UTF-8 console I/O for Windows builds on OS version 10 or more. This is to accomodate an IsValidCodePage() API which may happily report CP_UTF8 as a valid code page when the stock console cannot, in fact, do UTF-8 I/O.
FossilOrigin-Name: 6b9b2a886fd4d239c2e87c3f3809c011f77c0f60e0c279bbe4e1d1b53c609e2d
2023-10-29 00:24:22 +00:00
drh 5269e846dc Limit the range of integer unix timestamps in date4.test, since some systems
(RaspberryPI) cannot deal with timestamp values that exceed pow(2,31).

FossilOrigin-Name: 765290663b28e90a0494997baf023f9610a4ed32f0ff0099bf9fc3d485733fca
2023-10-28 16:19:21 +00:00
drh 50862112b9 Update documentation for sqlite3_errmsg() to try to make it clear that if
the schema of a database contains table names or similar with invalid UTF,
then the error message generated by SQLite might also contain invalid UTF.
No changes to code.

FossilOrigin-Name: 33ba13c7c4b6f9c5e64ea30c819718f2caea214afca945c9ed7075864f4aaa40
2023-10-28 11:53:14 +00:00
drh 1b772cae2d Since SQLite considers NaN to be like NULL and NULL sorts before any integer,
make sure the sqlite3IntFloatCompare() routine reports that the integer
argument is larger if the floating-point argument is NaN.

FossilOrigin-Name: de1cf31a34b0f21288e7e30434a06baf25ee579929107c22e65c57236577fc4a
2023-10-28 11:40:33 +00:00
stephan 793e872803 Merge trunk into jni-post-3.44 branch.
FossilOrigin-Name: 9670eb2496b4005cf718d9fd12dbd11733f7cf6704a967ebb316504ea6a51e82
2023-10-28 03:56:14 +00:00
stephan a9fea28761 Remove the SQLITE_MAX_ALLOCATION_SIZE flag (set to 536mb) from the WASM and JNI builds because it can unduly limit db exports via sqlite3_serialize(), as reported in [forum:75524f7342c1ba45|forum post 75524f7342c1ba45]. It now defaults to whatever sqlite3.c uses, which is currently just shy of 2gb.
FossilOrigin-Name: f6e1137919243c5ce86725df64b40b7e12e82cbceaff210ca41616d620f0dd1b
2023-10-28 03:54:28 +00:00
stephan 95c11b1158 Add missing docs for the JS Worker1 export method, as pointed out in [forum:75524f7342|forum post 75524f7342].
FossilOrigin-Name: a7b267bd11216ee990cdd855044fbc18c300dff07cf25b317f27c1bbbc340dcc
2023-10-27 23:02:01 +00:00
larrybr 4b8010a8ee Fix cosmetic flaw in SQLITE_TXN_* doc. (no code change)
FossilOrigin-Name: 72d7c18f80f41529811f74855ac198681a0cfe7634225233ec4b8df219c2e73b
2023-10-27 21:44:53 +00:00
stephan 26422f823f Export sqlite3_get_autocommit() to WASM and add tests for it.
FossilOrigin-Name: 7cb6d3cd3926882240c91a4b90fdf237b931653295d8d94b05ed885dcae59608
2023-10-27 21:05:50 +00:00
drh 6572f7799c Disable two rtree test cases for Linux-i686. We do not have that platform
available for testing.  The test cases work on all other 64-bit and 32-bit
platforms that we have at hand.  So there is no way for us to debug this
problem.  Better to disable the test.
[forum:/forumpost/ffcbf789b5386573|forum post ffcbf789b5386573].

FossilOrigin-Name: fed4d6e90baecca716429f95e1fab7930297afa2a191a87a0a87cb93910d1bce
2023-10-27 18:16:15 +00:00
drh 2e0d7b28c1 Add makefile targets for fuzzcheck-asan and fuzzcheck-ubsan.
FossilOrigin-Name: 27fc14303a0c6942e8cf5017d66c7ff5fcbc700a6109a93768263370f608397e
2023-10-27 17:40:46 +00:00
dan a03932a790 Fix to Makefile.in so that the "tool-zip" target can be run from outside the source tree.
FossilOrigin-Name: cd118da9db0680fab8761ff44384639b9d564b67a5dfb1280ab72e44ff57714d
2023-10-27 15:31:44 +00:00
larrybr 1b2a93ea36 Cleanup shell.c comments. Hide -utf8 option, as it is undocumented and accepted only for backward compatibility. No functional changes except for -help content.
FossilOrigin-Name: 3a87995560b5acbebeb5af407aa9eddf4c37c6a27ec9429ece0ea931918f9d9c
2023-10-27 13:59:05 +00:00
drh c42276f63e Back out [2904fcbeebba9189], mostly. The page bitmap is needed even for a
partial integrity_check in order to avoid an infinite loops while walking the
database.

FossilOrigin-Name: dd6e03b37356e2961abd97f636402c09744c98c3033f3435503958c6219f9121
2023-10-27 10:49:17 +00:00
larrybr 1ecf531fb0 In CLI for Windows builds, do MBCS/UTF-8 translation independently for input and output. (And cure an MSVC warning.)
FossilOrigin-Name: 80f0b6f9c76c049edaaaeedc1745978b9d5b02a9c34bbf64209266a154b2b0e3
2023-10-26 18:41:13 +00:00
larrybr d7969e5274 Handle utf8 mode restore better for redirected input or output. Move utf8 mode setup to before any invocation command processing. Sync w/trunk.
FossilOrigin-Name: 59452e7c72de02c2b6d40974c91173d87fd5f6f2e28c0b1f6f53eb1a09693fb6
2023-10-26 18:24:41 +00:00
drh 2473180bb1 Enhance the "tool-zip" makefile target on Windows so that it uses more
universally available commands to build the ZIP archive.  Fix other minor
issues with that makefile target at the same time.

FossilOrigin-Name: d2f2f9126562c24966b0e9bc3629b85950c20199c41250478ba1da68eb7a0046
2023-10-26 17:30:54 +00:00
drh 113e15fbbe Further enhancements to make sure the correct returnning trigger is run when
there are nested INSERT RETURNING statements.

FossilOrigin-Name: d83405e870b071540b56e76291bdce1388db80b860afd68a34e4233f6ff2a3ab
2023-10-26 16:59:22 +00:00
dan 94331d406a Handle the case where a virtual table xBestIndex method called while coding a trigger fired by a top-level statement with a RETURNING clause prepares a statement that also contains a RETURNING clause.
FossilOrigin-Name: ec6bffa3b0fbf7cdee852cb0208de1f5869dafd88e2ee64c23cec89634219adc
2023-10-26 16:05:57 +00:00
larrybr 69d40f8e1f Ready for merge, but could use testing on some dusty old Windows machines.
FossilOrigin-Name: 20220615621e01ff83fd26133840e2553a6087bcb56bbb8e9bf1408dd881cd9b
2023-10-26 15:12:41 +00:00
drh 3c75a03615 Deal with the case of a reentrant INSERT on a virtual table where the
outer INSERT has a RETURNING clause but the inner does not.
dbsqlfuzz 3ac9a1e33f676254e02c0f297263b0a7aeb0c1a5.

FossilOrigin-Name: 8aba78e4dbf5c196aa50a28579f4bcd7c96103661c5968d62b2fded075bc73e4
2023-10-26 12:59:41 +00:00
larrybr 634b037fa3 Cure a harmless MSVC warning. (no other changes)
FossilOrigin-Name: 02234ef85ed9349ddf6225f67b7e941b0b68cbd2e58c9d7d84a8324393349853
2023-10-25 20:44:01 +00:00
larrybr 63aff62e69 In CLI for Windows builds, do MBCS/UTF-8 translation independently for input and output. (WIP)
FossilOrigin-Name: 47b13e66e07be3a26cf7fa0b7b65ad8576c3b4be50fc422be8fe93be93d681ac
2023-10-25 20:27:18 +00:00
dan 799e743333 Updates to the mktoolzip.tcl to work on win32.
FossilOrigin-Name: 166b775bb523362ffe17722ce175278726303c319fbe7e4a1e9291736db38a5d
2023-10-25 20:08:21 +00:00
drh 208f5c65d4 Fix harmless compiler warnings.
FossilOrigin-Name: cc8efe0494b8fe0df18aa67b1675779bf704d1ac53647fe1f7f55d8048041680
2023-10-25 19:06:23 +00:00
drh d91884586a Provide dummy parameter names in the prototype for the new xIntegrity method
of the sqlite3_module object.

FossilOrigin-Name: 131374d720a365cbbe3031a2bc0a34ba1c74e751e39c66e484dbce07ab49fb62
2023-10-25 18:53:00 +00:00
drh 1d91f8619b Fix minor problems with the "showwal" utility program. Add a missing header
to the zipfile.c extension.

FossilOrigin-Name: 7b3f866e619c2b781b613493727cd111335ceec690d984408725c756feff26e6
2023-10-25 18:35:48 +00:00
drh b7086c48b6 Test module altermalloc3.test requires DQS_DDL to be turned on.
FossilOrigin-Name: c4d02f4b0761a2836aa34f204e725f49478a19b02160e694e01042e6e7766167
2023-10-25 17:26:11 +00:00
drh 47a484d8ed Add an ALWAYS on branch that was made always-true by the DISTINCT ORDER BY
fix.

FossilOrigin-Name: c29ebcb0be8e969c359c077bd4ce752a9c952d2659062868f12efdd2cc77186d
2023-10-25 15:30:11 +00:00
drh 8f99cb9ad6 When doing a DISTINCT aggregate that contains an ORDER BY, only the
arguments to the aggregate need to be distinct, not the ORDER BY terms.

FossilOrigin-Name: d2dbbdf7194bab4e5e3b74d3dbffb012a335829824c775c72dd7347c013d2125
2023-10-25 14:54:16 +00:00
drh a9a7d118f6 Fix the amalgamation generator so that when it is reporting the Fossil
version number in the header comment of the amalgamation, it does not
elide the first hex digit of the version hash.

FossilOrigin-Name: cc66f526bd58c110cac027583f636099fa05b2b56d7ea9807b554cd950c1bf3c
2023-10-25 12:58:06 +00:00
drh 9f20bde65a Enhance the new xIntegrity method of the sqlite3_module object with new
parameters that provide the name of the table being checked and
a flag to indicate a "quick_check".  Based on feedback in
[forum:/forumpost/965c0d02ea|forum post 965c0d02ea].

FossilOrigin-Name: bc8afa3f15954bb35f65dbf940bf069de5e14d333036676c24430cf17b658d05
2023-10-25 10:37:11 +00:00
drh f4154879eb Implement check-in [477577120b897bf1] differently, so as not to disrupt
non-standard build configuration. See
[forum:/forumpost/c11523ca2df50293|forum post c11523ca2df50293].

FossilOrigin-Name: 50448fe4fdc8fd93303fe26bdcd885ecc606080c8e66e69d5be8dac28a77492b
2023-10-24 19:56:57 +00:00
dan 68666f36f9 Fix a test case in fts3corrupt4.test by bypassing the fast-prng.
FossilOrigin-Name: 540ce72b5089413ed5bf2f0fbdf2f9874aee6cf3e42f24b4e8bfb440816e734b
2023-10-24 19:05:55 +00:00
drh 85b1f5c2f6 Test cases for COLLATE and NULLS FIRST/LAST in the ORDER BY for an
aggregate function.

FossilOrigin-Name: 641f928feb8b6cfd64cb03992bc18c1653960b6eaeb35145d17df494727c7a11
2023-10-24 17:45:11 +00:00
drh 9d486435f1 New #ifdefs to fix certain compile-time options.
FossilOrigin-Name: 688c6279ef1d4ff8d58e83a1c73ca0221ab8570a7cd89459946a86afdad6ec1c
2023-10-24 16:16:27 +00:00
dan 3da982635c Fix a problem with the fts5 highlight() and snippet() functions when used with tokenizers like "trigram" that output overlapping tokens. Forum post [forum:/forumpost/63735293ec|63735293ec].
FossilOrigin-Name: e952db86faaafd2ef8558175ebcae6834d0b0002813cc78c7ae7da713604cbec
2023-10-24 16:06:56 +00:00
dan d548f74024 Fix a problem with the fts5 highlight() and snippet() functions when used with tokenizers like "trigram" that output overlapping tokens. Forum post [forum:/forumpost/63735293ec|63735293ec].
FossilOrigin-Name: d570aa02f79b1d7d3889e33f9eebab1b7edcf5231b1357451eed9a538607de54
2023-10-24 15:53:02 +00:00
larrybr bdcae89358 Spelling improvements, in comments only
FossilOrigin-Name: 9ee25eeaf17124dbe34b961384e74c1ae239ee1798bb01907938600ed6b4066f
2023-10-24 15:11:55 +00:00
larrybr da67619d97 Spelling improvements, in comments only
FossilOrigin-Name: 65bcc583784476f8cc52dfe72ecc52df93d0ec5817c313ad7a83905c0047dd8c
2023-10-24 15:07:28 +00:00
drh c0ba6a97ec Fix various harmless scan-build warnings.
FossilOrigin-Name: 54be9af4469d7e31ee852f67e5aa32996557c10de654a60103fd165d2fedf311
2023-10-24 11:06:44 +00:00
drh eb9882d7d1 Fix a harmless compiler warning in the expert extension.
FossilOrigin-Name: 0c4907ddf9abd1ebfef31c1a53d702c4dcaa015c0032b8b52774c9e260b6cfd7
2023-10-24 09:57:54 +00:00
drh 13e69440fb Fix a minor typo in the previous.
FossilOrigin-Name: 9d388267e4e6724e2df333fe09d509e87defcfe984c5c2ebe031152d320812d0
2023-10-23 23:34:53 +00:00
drh 043768ce5a Add notes on how to build a statically linked version of sqlite3_analyzer.exe.
FossilOrigin-Name: 99ce9939c861ecc07877e640f75d395afb6502130bf4b5ccea8b98a80a1569ce
2023-10-23 23:34:24 +00:00
drh 811bece3f7 Add the sqlite3res file to the Windows build of sqldiff.exe so that the
resulting binary shows the SQLite icon.

FossilOrigin-Name: c696a4963bdd010c8161acbf7601078313e7da5ebb3f60381a489a70e5728dea
2023-10-23 19:51:16 +00:00
dan 1d2a05fc10 Merge latest changes from wal2 into this branch.
FossilOrigin-Name: c9c9bc097a622bfb6e094b1178cd44da19418cf19cb2d6ad54b9bb868de51f10
2023-10-23 16:08:37 +00:00
dan 757372ab56 Merge latest trunk changes into this branch.
FossilOrigin-Name: 3a6479d3f9d929c63e0caed6288d24ef004edee669d242285dea3fc60345038a
2023-10-23 16:00:26 +00:00
dan 4fecc81e20 Avoid attempting to delete a file while it is open in test script sessionalter.test.
FossilOrigin-Name: f20b4a96f747adeb2ed486f06dd7d911f0f24b37df87dcd32d2c3e4b687da9a6
2023-10-23 15:30:38 +00:00
dan 2db02ec245 Revert an earlier change that considered passing a NULL callback to sqlite3_preupdate_hook() to be a misuse. This is required to clear the preupdate hook altogether.
FossilOrigin-Name: 1bf6f0f3bb54933e4e6c730554144934f73a7fac87c97a916b275afd7379f397
2023-10-23 15:24:44 +00:00
drh 8c86f56e90 New test cases for PRAGMA integrity_check against FTS4 and FTS5 tables.
FossilOrigin-Name: 3950d20c3a4433543546ce4495ae07aab3d6cefb163b1f30813a9776c5b962e8
2023-10-23 15:23:10 +00:00
dan fd05978a1e Fix mutex1.test to take into account that the prng mutex is not (as of [c84e4483cb4]) taken when writing a header to an in-memory journal.
FossilOrigin-Name: c9e37417fcc41f1e975290bf583113ff8f4144c056e8fcd9fafcb432c1a26b18
2023-10-23 14:24:09 +00:00
drh 45b2e946cc Improved error messages for the xIntegrity method in FTS3/4 and FTS5.
FossilOrigin-Name: df69abc0823f4dd9c051f8d5993d2f447ee0dadab5cef73d670107b1071db255
2023-10-23 13:46:22 +00:00
drh bd8941a4f8 Fix sqlite3ExprDup() alignment assertions so that they work on 32-bit
platforms.

FossilOrigin-Name: 143f95efc7114455b1920f1d2f9a867392740f608d1dda4306005f01238103f8
2023-10-23 12:16:27 +00:00
drh f49d52d642 Temporarily disable the ext/expert/expert1-6.0 test, which does not work
on my configurations.

FossilOrigin-Name: a41b903d0fe489d20c4a7118a2c91f7c8a78568ce552225f64c2945bb811ea16
2023-10-23 11:41:41 +00:00
dan 3f81c3264a Add an SQLITE_OMIT_VIRTUALTABLE block around the new vtab code in "PRAGMA integrity_check".
FossilOrigin-Name: ab99eb2b6c50f4f0bb42918b9bdda37e2ef9316bed07de18fa4bd373c49a8b00
2023-10-23 11:34:24 +00:00
dan 06700d77c2 Add missing test file fts3fault3.test. Should have been part of [7f41d700].
FossilOrigin-Name: 92e64ab63a78445fe20863f9679c55a630a2295ff10c4ce33eab6dfaeef09622
2023-10-23 11:18:13 +00:00
larrybr 47b496868c Merge fixes for expert handling of UDFs and other UD-whatevers.
FossilOrigin-Name: b5d7d07df58bb8e4013b4047e4019b96e506e8ef03ca0abfc9aae06a122b353a
2023-10-23 02:08:34 +00:00
stephan 334b6fe418 Add missing JNI sqlite3_result_subtype() binding.
FossilOrigin-Name: ac920b5386db525c794f4cae332dedcc709cac356f6fb85f3e92bc0a34602511
2023-10-23 02:06:27 +00:00
drh aa040d891f Fix the shell1.test test so to align with the new behavior imposed by
[bce807cd48763273] - that backslash escapes only work without quoted arguments
in dot-commands of the CLI.

FossilOrigin-Name: 0db82b4281a0e0d5e365553df11e0347f60c00c861c0fb96227059edff3a0ef6
2023-10-23 02:01:14 +00:00
larrybr 81be0d26bb Clear some picky warnings, sync w/trunk.
FossilOrigin-Name: ee58425904b36319e016dc69bb5f141bb3565b1723a97490a4b2cfa16b89fbdf
2023-10-23 01:55:35 +00:00
stephan 90a5617b55 Add many more high-level wrappers to the JNI wrapper1 API. Correct the JNI bindings of sqlite3_strglob/strlike() to compare as the core lib does if their glob argument is NULL and the other is not.
FossilOrigin-Name: 55c4b1dc402b358d53d65fa1f6ec063e9e38e95c81a05d98dae3cb58c52ef55c
2023-10-23 01:34:17 +00:00
drh 5e6dde3b35 Fix [f5c01676fd281e93] so that it always preserves 8-byte alignment for Expr
objects.  Add new assert() statement to verify this.

FossilOrigin-Name: 678a9728dc6b88d8ef924c86603056df18204bc9a9c4776b9baffd7c5b10c5f2
2023-10-22 23:44:32 +00:00
stephan 96aa4d344d Add high-level window function wrapper to the JNI wrapper1 interface.
FossilOrigin-Name: a27e7471231a24864cbd04b77cbc4b336ce180d738a36ce4318543e2666ed708
2023-10-22 23:36:16 +00:00
drh a63539143d Do not do backslash excape processing on any unquoted strings in dot-commands
in the CLI - on Windows or on posix-like systems either one.  This brings the
processing into alignment with the documentation, allows backslash-delimited
filenames on Windows (as long as they are unquoted), and causes the CLI to work
the same with regard to backslash escapes on both Windows and posix.

FossilOrigin-Name: bce807cd4876327396b4ffcdf77f6931dd3bbd3314336eedf38bcf01d17af32c
2023-10-22 17:27:55 +00:00
stephan 166c8d0067 JNI: flesh out and simplify the APIs for binding and fetching arbitrary Java objects.
FossilOrigin-Name: 89fecf1dd8b97941f9b45130a3c8a67af36ec65cc6f70f5026c569c058a4963f
2023-10-22 14:25:37 +00:00
stephan 32a0d6129f JNI: add column-get bindings to the wrapper1 Stmt class and extend the AggregateFunction tests to ensure that the aggregate context is honored.
FossilOrigin-Name: 60a0e82db26270af9d0a5f55c6173e4fd0bdc90a885e838480ed75f8ef193287
2023-10-22 13:54:26 +00:00
stephan 4598b6e5ae Add API_ARMOR to sqlite3_clear_bindings().
FossilOrigin-Name: f3fb4d345bbf5ae4a35d8076043df601b1bf7dfd68760a416440139eb3e5eb9a
2023-10-22 13:09:37 +00:00
stephan 4bcde614d8 Add JNI wrapper1.SqliteStmt.bindXyz() APIs.
FossilOrigin-Name: 54fce9bf04a7517cdc8e96fe2efec66f03b0d03983c3a45d0ae7e1f16aa5a6c9
2023-10-22 12:43:30 +00:00
stephan 28f45b1ab1 JNI: improve UB protections in sqlite3_bind_blob/text/text16().
FossilOrigin-Name: 5c8383210a87d7f9d37a27053b5b1b6f41794fa8612826c68c1ca49c495cbd97
2023-10-22 12:33:05 +00:00
stephan 6e4ef06ca1 Add missing ScalarFunction.java to JNI build.
FossilOrigin-Name: b8258103fb433d9f5cfa15661b5edf4e60128fb4161d8a18e5cc3253e5aed72b
2023-10-22 11:11:54 +00:00
drh 06f4dc7edb In the CLI, do not translate unquoted backslash escapes on dot-commands
as that can damage filename arguments.

FossilOrigin-Name: 4a0e2c6e17eefb296b7e91a13305b2260d6eb869a37bc5e5b79edaf3c19c810a
2023-10-21 22:31:00 +00:00
drh d2c737f2e4 Remove a branch that cannot affect the outcome from FkNoAction processing in
fkey.c.  Replace it with an assert().

FossilOrigin-Name: 023a9dbe83c0042e9d500e3ae6c0592a468982e4ac278d08c9201967506c7555
2023-10-21 20:34:57 +00:00
drh 76bd238a66 Replace the patch at [5cb61c6788d7c017] with a better way to avoid failing
PRAGMA integrity_check if a module implementation is not loaded, as the use
of suppressErr can lead to problems.

FossilOrigin-Name: be5737ee4f4f1d5f8b3126f6eb70210bf3d8b1cdaef51aae818eaaba43ccf722
2023-10-21 20:03:44 +00:00
dan ef6bf1bbe5 Fix a problem allowing a COMMIT following an OOM to cause fts3/4 corruption.
FossilOrigin-Name: 7f41d7006db4225cf9b3d197d3a76842778669ac079e76361214a8023c9976e6
2023-10-21 18:12:07 +00:00
drh da4cfde031 Add the SQLITE_TESTCTRL_FK_NO_ACTION test control. Make it available in the
CLI.  Fix a minor problems with SQLITE_CHANGESETAPPLY_FKNOACTION in sessions.

FossilOrigin-Name: 91b64c6a70744704fc3285be7d8d46ba679ea5f0f69bd77b0e093eeb7447947d
2023-10-21 16:33:20 +00:00
drh 5c8cfe9617 Extra comment on the implementation of SQLITE_TESTCTRL_FK_NO_ACTION,
warning about the need to reset the schema in order for the setting change
to take full effect.

FossilOrigin-Name: a50a333ae11ba5d92f432108308ac0bec9afb00f466b78c8d3f3aa7e2851ef21
2023-10-21 16:25:44 +00:00
dan fcef73a787 Add missing "finish_test" to sessionnoact.test.
FossilOrigin-Name: 9a1d37f5479adedb93f40e8189ad0efa891a438805d80774ee992d4dfd71150f
2023-10-21 15:21:06 +00:00
dan ca9dc173a2 Fix a problem with SQLITE_CHANGESETAPPLY_FKNOACTION and the foreign-key-trigger-cache.
FossilOrigin-Name: 95304ec6c9654b4366dc58ae263270cb42861cb5017174936f816922b5eb61e6
2023-10-21 15:04:24 +00:00
drh 51b358e5e5 Add SQLITE_TESTCTRL_FK_NO_ACTION.
FossilOrigin-Name: 563cf5f782cdddbbd7f727c65118edfd109aeb731c8aaf0d6ee5ee7030e61ea9
2023-10-21 12:54:37 +00:00
drh a35a54b824 Adjust an EXPLAIN QUERY PLAN output in scanstatus2.test to account for a
different loop order taken due to the removal of the view-scan optimization.

FossilOrigin-Name: 9deb8b6915e794bf9c5de88f2727ef9857351eaf905f8ab56754bc4d399c88b4
2023-10-21 11:43:54 +00:00
drh 813159e2ea Compress the content of the status line in testrunning.tcl so that all the
same status information appears but with less punctuation and with "zero"
values omitted, so that the status line will fit comfortably on an 80-column
terminal window.

FossilOrigin-Name: 966351311682a319ec796c07d407ce90cd1aa0cea9e904ba1085a4ebadbdf925
2023-10-21 11:34:59 +00:00
drh 531ca6058e Create the new string_agg(X,Y) aggregate function which is an alias for
the two-argument group_concat(X,Y) function, for compatibility with
SQL-Server and PostgreSQL.

FossilOrigin-Name: a3b3df8a9e43f1880a904b76027db56ed504a84622831c96ba1962a19bbed762
2023-10-21 11:13:47 +00:00
drh 266e5d034b Avoid an unnecessary malloc() for the page usage bitmap when running
a partial integrity_check.

FossilOrigin-Name: 2904fcbeebba9189cebc48e58d12fc46f78ee23d8c4d46644606691f8cabb80c
2023-10-21 11:06:03 +00:00
drh 51bb5dec6c Fix a test case broken by the previous check-in.
FossilOrigin-Name: 1e61864c3bffec133632f4575d59d11936e8bafc089c4d84a129dcaf7e9b6d7d
2023-10-21 11:00:57 +00:00
drh d5e040b6f0 Add string_agg(X,Y) as an alias for group_concat(X,Y), for compatibility
with SQLServer and PG.

FossilOrigin-Name: b91c19bf2680f60d7826ab5d9e7902e2dc2a55d847bbea565a6489d47f2cc8f1
2023-10-20 20:19:30 +00:00
dan 71bae9f19f Fix a problem allowing a COMMIT following an OOM to cause fts5 corruption.
FossilOrigin-Name: fba3129d1362dd3ee04d61cb3d3c48518cd98dde3febc85298f411776d60073e
2023-10-20 19:59:12 +00:00
drh 81656b15c9 Fix PRAGMA integrity_check so that it does not raise an error if the
schema contains a CREATE VIRTUAL TABLE that does not have its module
loaded.

FossilOrigin-Name: 5cb61c6788d7c0170b587e4667ee57d588c991a9901951762e33b97c6a135a82
2023-10-20 19:06:04 +00:00
drh e98f62fb3f Omit some redundant calls to strlen() used to find the size of
Expr.u.zToken in sqliteExprDup().  This inefficiency was seen while working
on the previous check-in, and I thought it best to fix it while it was
fresh in mind.

FossilOrigin-Name: b7a84eff5dcdf5b2ff81331097cdc64d0c5d16a4ae224320f39b64541ec72dc7
2023-10-20 18:09:08 +00:00
drh ab3eb5b770 Improvements to the sqlite3ExprDup() logic for faster performance and better
run-time error detection.  This check-in fixes the 5x oversize memory allocation
bug from [f371e4c0f8ea73ae] as well as all other known issues that result from
handing the ORDER BY clause of an aggregate function off of the pLeft pointer
of the Expr object.

FossilOrigin-Name: f5c01676fd281e938181b846dd2024d050f597dc6a7a91928beab9d8553dfdb5
2023-10-20 17:15:15 +00:00
dan 17c3408e0e Add the SQLITE_CHANGESETAPPLY_FKNOACTION flag to sqlite3session.h, for passing to sqlite3changeset_apply_v2() to cause all foreign key constraints to behave as if they were declared NO ACTION.
FossilOrigin-Name: fc9f82ea084159eaf3dd1757b96d17d1201b00c4e06455a7dcd8067172b25f28
2023-10-20 17:06:39 +00:00
drh 90e4a3b7fc Simplifications and optimizations to the Expr object duplication logic.
The 5x multiplier crutch from [f371e4c0f8ea73ae] is still present.  More
fixes are still needed.

FossilOrigin-Name: 56142a78163b755f16afc05201f623a7a19d9a4b0620a67f7fa20d2a965a288d
2023-10-20 15:47:30 +00:00
drh 0a261eb4bb Omit an unused constant from sessionfuzz.c to prevent a compiler warning.
FossilOrigin-Name: a0cf7e24f928183866ac54f0d6cd83c859d487a19dc87572ab0188d90d2ff87d
2023-10-20 14:05:26 +00:00
stephan 11223e0cff Correct the opfs-sahpool VFS's xGetLastError() method to return the previous error code, not 0, on success.
FossilOrigin-Name: 95a1dde63117d696323c775580b9c04f044a5b8d609e9174b739ac03ecc1336c
2023-10-20 12:23:49 +00:00
drh d707e012f0 Use sqlite3ParserAddCleanup() instead of calling sqlite3ExprListDelete()
directly when disposing of an unused ORDER BY in an aggregate function,
to avoid disrupting ALTER TABLE data structures.

FossilOrigin-Name: d083e42086733ecd79aba8c268e020b01782bfe1cfa9684ce1c277af9c8bf92a
2023-10-20 10:18:03 +00:00
drh f5e487d168 Merge recent trunk enhancements into the jsonb branch.
FossilOrigin-Name: f47a8d0a207a8442a7f621b070ce9dd1d6013ce26bcf68165d20bb379bd478a0
2023-10-19 20:46:22 +00:00
drh dcf76a8bbd Remove a faulty assert() that was added just a few check-ins ago by
[8b6fffb552c30f9c].

FossilOrigin-Name: 18e7c826f08bce51719ef045daa60200b33790f2fe312c6853f9ef6c7e9d5030
2023-10-19 20:06:20 +00:00
drh 5f53f85e22 For TK_ORDER expression nodes to always be full-size.
FossilOrigin-Name: a5c73b46f4772f214ffbfa31cb87dce82ebd690addabef539bd09def26b7744d
2023-10-19 19:57:57 +00:00
drh 5080397e67 Fix an adverse interaction between the new aggregate ORDER BY logic and the
expression compressor.

FossilOrigin-Name: f371e4c0f8ea73aee9ea0645f396e3da20f1eb97be34c83de9d94c9cbb959934
2023-10-19 19:19:59 +00:00
drh d8b840a4f4 In the TreeView logic (used for debugging only) do not show unnecessary fields
of the Window object when a function has a FILTER.

FossilOrigin-Name: d2c6b82c346dbf39e07f09d187e4a8216a56caa7df3dcfafc82e32f676a1db8a
2023-10-19 19:08:33 +00:00
dan ee3c55471c Fix an fts3 problem caused by reducing the page size using the undocumented "nodesize" option, then running an incremental-merge.
FossilOrigin-Name: 2875dcb1ac86db2704ad377d719f308c8837e196b49c98b20ceda1324f2d27ea
2023-10-19 18:23:17 +00:00
drh 4e254641ff Simplify the Expr compression logic slightly by adding the new EP_FullSize
property to expressions that are exceptions to the rule and should not be
compressed.

FossilOrigin-Name: d5ae82ec52eafed5e3dc8c9d99685f6523fce7d973ef7e8d9d75ed9b8912426a
2023-10-19 18:07:58 +00:00
drh 0588385799 Simplification to sqlite3GetVarint32() to avoid confusing gcov.
FossilOrigin-Name: 89862c51ad9715bedf5b029db484602e740f0db7404970d482ce503c1b9a0ed2
2023-10-19 13:35:22 +00:00
drh 11877c6ba4 Fix an error in a debugging routine used inside of testcase(). Does not
appear in production.  Problem discovered by scan-build.

FossilOrigin-Name: c6c9d7ed6dbc71b998aeaaa1bdeb36b1bb9b902f9d6088ad36db29f8ea8b7ef6
2023-10-19 13:00:41 +00:00
drh 266178f369 New assert() statements to help verify correctness of the ORDER BY aggregate
code.

FossilOrigin-Name: 8b6fffb552c30f9cbf7102c2e26a7b486c4c9edd9f494d803de7004db3018277
2023-10-19 12:36:59 +00:00
drh 519017f25a Fix the sqlite3ReferencesSrcList() routine so that it recognizes columns
in the ORDER BY clause of an aggregate.  Fixes a problem with
[634286828dad873d] discoverd by dbsqlfuzz.

FossilOrigin-Name: 3d26f1aaa4876f21f2c3abf13bbc37933c8f32471153e29019880cc9530cb011
2023-10-19 12:12:49 +00:00
drh d9451051f1 Add support for ORDER BY on the argument list of aggregate functions.
FossilOrigin-Name: 634286828dad873dba244751441aa729cd37b6ed5899fa5875643d4af03c006c
2023-10-19 01:26:34 +00:00
drh 0b633640fa Merge the latest changes from trunk.
FossilOrigin-Name: d18bc400146812c8c5048af7eaf4a5371ddd80fc9ce895d253b6114b9eae7fc3
2023-10-19 01:09:49 +00:00
drh 69ff726d4d Fix a false-positive in run-time error checking.
FossilOrigin-Name: cd63eec0758960d9ee63d7b964ec62e2bd622f8c94a58bd9556046381ffa18d0
2023-10-19 00:25:35 +00:00
drh d05e54efff Changes for test coverage.
FossilOrigin-Name: ddfa09c6031afd4391a9888381bf09214cd542a826b431eeb1537a070f65c5f9
2023-10-18 23:48:24 +00:00
drh 42f5ea354e Test case for alter column that is used in the ORDER BY of an aggregate.
FossilOrigin-Name: 648ddb3a8eb358fc6c95aaa561ae68476c77996abda23890054ac07166278cef
2023-10-18 22:53:22 +00:00
drh ce51b31367 Deal with an OOM condition while processing aggregate ORDER BYs.
FossilOrigin-Name: b6d44427ec0184880ad14919667e2f526777c5085ffa636f6734cb5951a31225
2023-10-18 22:27:59 +00:00
drh 20b95f8d3f Make sure all terms of the ORDER BY within an aggregate go through aggregate
analysis.  Do not attach an aggregate ORDER BY to a window function.

FossilOrigin-Name: 16f3805514a741405f70e0ee3b5a6b67720bc75719372e82daa4136fe411ea2b
2023-10-18 22:03:48 +00:00
drh 49dc1be96f Basic test cases.
FossilOrigin-Name: 6cccf86c362631ada0c6cbaf661520648f8bbf7ae2b137b3d7d959647387e5c2
2023-10-18 19:44:59 +00:00
drh 0021079ceb Fix one minor error in order to get all legacy tests to pass.
FossilOrigin-Name: 5a885139c433a143302ab43dc5aaac1fed9db61548b4ff7d4b19b3a6a46c8fcc
2023-10-18 19:07:45 +00:00
drh 59a0d0bbf9 ORDER BY on aggregates seem to work, at least for simple smoke tests. Lots
more testing is needed though.  Surely there are many bugs.

FossilOrigin-Name: 64c12a835b6f1df8f2f5f4a41de083f6b3fc7f8030042c6aac0082382cd9cc4d
2023-10-18 18:11:11 +00:00
drh db19f48b69 Basic error checking. Resolve symbols in the aggregate ORDER BY expressions.
FossilOrigin-Name: c83a53a574d312130d1238c05ffa449d8bed2535d5ef5b5d9cf02f894494cca4
2023-10-18 13:58:31 +00:00
drh f8202f1ff3 Enhance the parser so that it can accept an ORDER BY clause on a function
invocation.  For this incremental check-in, the ORDER BY clause is currently
ignored.

FossilOrigin-Name: 3a98ff24bf468ed42d410a9a12d9f9b2ca154c7babe99fd6bc6f7b0565e0d132
2023-10-18 13:18:52 +00:00
stephan 0b4de1acac JS: replace one errant reference to 'self' with 'globalThis' and remove a separate dead-code 'self' reference.
FossilOrigin-Name: da1a47932ea96755b1e11fc2547ce11780b87846af6885857318ff18f1e62c31
2023-10-17 23:35:47 +00:00
drh 43dc31cf1f Fix a harmless compiler warning about variant types for a pointer function.
FossilOrigin-Name: 37ff0d8e7f91c32e8c53bb015280be47c66bf599281b640a8d3fd41335b55289
2023-10-17 19:33:52 +00:00
drh 6db4e5ed4c Make a new branch in the SqlExec opcode reachable by tests.
FossilOrigin-Name: 023ad470b68b7c09e159244493a6d0285d706efdecd8719a71e6885509d4750c
2023-10-17 18:59:13 +00:00
drh 42b49a3138 Omit an unreachable branch when SQLITE_ENABLE_API_ARMOR is not defined.
FossilOrigin-Name: 5f8b807b568f4f9ec65f1c924e0e1ac2f996f405e045204145ad7116be7580ee
2023-10-17 18:28:27 +00:00
drh 4c827feba1 Changes to sqlite3IntFloatCompare() in an attempt to better measure
branch coverage in the face of aggressive compiler optimization.

FossilOrigin-Name: 5781d043ffeccda03357ee530564987443dc1deb6111a4028701b4a98fdfe16d
2023-10-17 17:53:46 +00:00
drh 52d934e316 Fix a JSON bug introduced by the optimization of [df099ad713011b67] and
first appearing in 3.43.0.  The problem occurs when doing a JSON_EXTRACT()
on an array element that was added by JSON_SET() without first reparsing.
Reported by [forum:/forumpost/fc0e3f1e2a|forum post fc0e3f1e2a].

FossilOrigin-Name: e5099c549a1d8959d4015516f090b8e6438e517a64b20651175bf5413d94fb58
2023-10-17 13:41:41 +00:00
drh 5e0b11bda7 Enhance the documentation of sqlite3_deserialize() to make it clear that
the input database may not be in WAL mode.
[forum:/forumpost/a7e272cee9ac469f|Forum post a7e272cee9ac469f]

FossilOrigin-Name: e754789971e019647e7068f76dc6f9778675e4849fe51d8b25b39d3edf2379e5
2023-10-17 11:57:36 +00:00
drh 11095bcb89 When an I/O or similar error occurs during a transaction with
journal_mode=MEMORY, attempt to rollback before closing the journal, as
all rollback information is forgotten when a memory rollback journal is
closed.

FossilOrigin-Name: 1d67f75de259e5a26b751a50432822a268ebe367cda6510891ab81a15e5daa1c
2023-10-17 10:47:42 +00:00
stephan b866f98346 Harden API_ARMOR checks for sqlite3_stmt_scanstatus_v2().
FossilOrigin-Name: d3f38e813ba3b887e973af034713cd58fdfbe07da375b154accc7b5790cfaeda
2023-10-17 02:15:49 +00:00
stephan 08747d44a2 JNI: add aggregate function support to the wrapper1 API.
FossilOrigin-Name: 15b28b340a5c5efdbfe3fbed16ee0b699561edaeebb77446addf2374bdf9357e
2023-10-16 16:04:23 +00:00
stephan 626d0a9fda JNI: add scalar UDF support to the wrapper1 API.
FossilOrigin-Name: a850535766d2243d9475e1523c753615875a2da9c9d82a41a9fb61b141c6334a
2023-10-16 14:31:13 +00:00
stephan f2d7e961d9 JNI: initial draft (untested - requires more infrastructure first) of a UDF argument/result-handling interface which completely hides the C-style API from the client.
FossilOrigin-Name: 43b10a5cf9cb8be53d62914f340d533e60a70bf4caa8b9b91c0f867fa0f70493
2023-10-16 13:04:42 +00:00
stephan 7e540e5a2c JNI: cleanups in Tester2 and update the jar makefile target to account for [9fcdf96adca2].
FossilOrigin-Name: abc82bf4b800dde1b6e6172c7be816edb391fdbed5dbd2749f54623fdf3bf8e6
2023-10-16 10:38:34 +00:00
stephan 22fd153e43 JNI: move the C-style API parts into the capi subpackage and the higher-level wrapper into (tentatively) wrapper1, so that CApi.java can support multiple independent higher-level wrappers without name collisions, and CApi can be made public but have the option to be elided from wrapper-level javadocs for wrappers which do not wish to expose it.
FossilOrigin-Name: 9fcdf96adca25cc2d2f4b75ec4eea94254fb9671c5ba63b88213d7f62dedff1b
2023-10-16 10:27:19 +00:00
stephan 19179722d6 JNI: after calling a Java-side UDF, zero-out the pointer of the Java-side sqlite3_context and sqlite3_value array entries to avoid misbehavior if a client makes the mistake of holding a reference to one of those objects.
FossilOrigin-Name: 9fc3104f76a83d600beb11d91feb97bcea8bc7f7cda8cd73e7a6b81fbba879df
2023-10-16 08:10:11 +00:00
stephan f83a4d850c JNI: do not expose SQLITE_OPEN_... flags which are specific to VFSes.
FossilOrigin-Name: 2b4e53d8be42a3bc098317abd8bb58b8ddc25094d80787f784bbc896f4f7b976
2023-10-16 08:05:51 +00:00
stephan 24f6baca16 Change 3 instance of #if SQLITE_ENABLE_API_ARMOR to #ifdef for consistency with how it is normally used.
FossilOrigin-Name: dd766eeb59fec71627dd8ad8f120875b96fda455c6401e5671e086b785e2b2bc
2023-10-15 13:36:21 +00:00
stephan 5698e62f9e JNI: enable all optional components in the default build and fix a test broken by an option toggle.
FossilOrigin-Name: 5586a9df58a400c20a6815e8b0fbc7a7c49e5ef89daf92c452f1990586ca78ae
2023-10-15 13:32:38 +00:00
stephan 29aeb620e7 Correct non-void return from sqlite3_preupdate_hook() when API_ARMOR is enabled. Broken by [6cb77503484e].
FossilOrigin-Name: 40f5906f48d3f1f44d54e5b271fe04e7082166e50e3ed250bb57a4760f112e8a
2023-10-15 13:25:39 +00:00
stephan 6089e5bfcb Add coverage of more functions to SQLITE_ENABLE_API_ARMOR builds.
FossilOrigin-Name: 6cb77503484e104a51f08690974d6e19c336ab5467d327f84be1d042615f008c
2023-10-15 12:11:10 +00:00
stephan 8a273bb6e8 JNI: extend [baf220e78a46246c47] to include macro-generated sqlite3_value_...() bindings.
FossilOrigin-Name: ad34b2542e330c22fcf331810ab43e66346b1bad8de1a3989e6208cfa0fc2f92
2023-10-14 20:44:13 +00:00
stephan 4b1e42ef94 JNI: make the sqlite3_value_...() family of bindings resistent to NULL arguments.
FossilOrigin-Name: baf220e78a46246c476cd429ac131698db5fa66f62fb5725d4f736fe5246283c
2023-10-14 20:34:40 +00:00
drh d718bde6da Do not allow an ALTER TABLE ADD COLUMN on a STRICT table if the added column
contains a DEFAULT clause that would violate the type of the added column.

FossilOrigin-Name: 75b075863eaa56e36635a1d27740d37de8600ba92099b3fad9378a1e6ce12c0e
2023-10-14 20:24:52 +00:00
stephan eb62ccdab5 Add API_ARMOR support to the scanstatus family of functions.
FossilOrigin-Name: 2f3101f016d5f8d954a262befdf5e0f938a223e444bc534e1d3655b733f4663d
2023-10-14 20:01:55 +00:00
drh 153790d9ae The rtreecheck() SQL function should not invoke BEGIN or COMMIT as this
causes issues for statement transactions.

FossilOrigin-Name: b7b2e30b570efda338c62e88446dedb6667640f33855b2422ad20c7b569b025a
2023-10-14 17:14:53 +00:00
stephan 4213cc78cd JNI: add a missing result code check to sqlite3_bind_value(stmt, null).
FossilOrigin-Name: f1a357f42de58927c33e7047dc4417e3b9549d7529625f30c583e66b7d603b00
2023-10-14 16:57:48 +00:00
stephan a9000d424e JNI: add missing sqlite3_bind_value() and minor memory-safety-related cleanups.
FossilOrigin-Name: 3900031763abf4b2aab7b57e886cc407571d0d5928b74f24dcea8fb7371655af
2023-10-14 16:56:12 +00:00
stephan c76e450908 Add API_ARMOR support to the sqlite3_result_...() family of functions and sqlite3_bind_zeroblob64().
FossilOrigin-Name: afabe3e35a66625527e2881749cdb6e13300888ab57bc0f05889d0e3ee203d73
2023-10-14 16:29:36 +00:00
stephan 7a54b12695 More API_ARMOR additions.
FossilOrigin-Name: 78ebf838f645742f87733665cd72af736df345683b27377a2c8310c893b1769d
2023-10-14 14:53:18 +00:00
stephan a17f63290c Add column name to API_ARMOR check in sqlite3_blob_open() to avoid a null-pointer deref.
FossilOrigin-Name: 0114a6622afc4588c47e98d804340449417b603dc4831513eab4d8e4ccb15d42
2023-10-14 13:24:30 +00:00
stephan 067a13c395 Revert [f6cd88e6b234] - the NULL callback case is perfectly legal.
FossilOrigin-Name: 718ab67607895176e529eb7469832d262a347d030e83e7ee66d3b4704bf933de
2023-10-14 12:45:11 +00:00
stephan 6d035d481f Bring JNI-side sqlite3_last_insert_rowid() and sqlite3_table_column_metadata() in line with the core's NULL handling.
FossilOrigin-Name: c2afc1c2a2fc9bdf3d7b7701004e0fa40965cf8c6b7cf44b46f2ca37cfa1d2aa
2023-10-14 12:40:57 +00:00
stephan eaebd7fb6a Extend API_ARMOR checks on sqlite3_commit/rollback_hook() to include a check for the callback pointer.
FossilOrigin-Name: f6cd88e6b234560f729ff00da86144a0121ad96d5a07d227c9ffa3f43c22f72d
2023-10-14 12:20:55 +00:00
stephan 2ba1acce81 Add missing JNI bindings for sqlite3_db_readonly() and sqlite3_db_name(). Code-adjacent tweaks for the API_ARMOR audit.
FossilOrigin-Name: e5eace747ae656f1e05a346e53a2827242e57107567d2c3470e292cf0b40858d
2023-10-14 12:19:59 +00:00
drh c84a0b2195 Bug fix in sqlite3_analyzer: for databases larger than 1GiB, take into
account the lock-byte page when calculating the number of freelist pages.

FossilOrigin-Name: 26a909cdd32afee0f15968ca6f611eb259373b2a2195d0b3d47cfba103f1e6d6
2023-10-14 10:54:37 +00:00
drh 9132b8816a Earlier detection of a host of errors in CREATE TABLE, such the CREATE TABLE
statement itself fails, rather than generating an error on the first attempted
use of the created table.

FossilOrigin-Name: 348fa7aaf7958b3fb689ed023d946064ae8d92718a497a346e95114a2410cbf5
2023-10-13 22:19:23 +00:00
drh cc67a62fa0 Apply the correct affinity to DEFAULT values that are TRUE or FALSE.
FossilOrigin-Name: 4958db70c8103e249748fdeffc35f1c385e2e550e8a48697b82fcd797f3016be
2023-10-13 19:41:20 +00:00
drh 6f12e51313 Do not allow triggers on shadow tables under defensive mode.
FossilOrigin-Name: 44b932ff16e35814febd842d11215cf243393f173aade33ceded473598221e88
2023-10-13 18:29:18 +00:00
dan d7cdfe74f6 Fix an fts5 problem caused by a 'rebuild' followed by a DELETE in secure-delete mode.
FossilOrigin-Name: 18be505c628d9b13431ca6cfe822d4aeae119c53ae08aef26a67f310a8bd7bd3
2023-10-13 15:59:11 +00:00
drh af527231c1 Immediately fail a CREATE TABLE statement that attempts to create a
table that has a generated column loop.  Legacy allows the table to be
created but the table would not be usable for anything.

FossilOrigin-Name: 3237bf964117c1ef71143042837ef21872bb3d04bfd682075672e768953ec802
2023-10-13 13:49:46 +00:00
drh 9817065340 Actually prevent PRAGMA writable_schema=ON from being set in defensive mode,
rather than just preventing it from functioning.

FossilOrigin-Name: 2b574d9e7e922d0961ea27bfc0148d3098157ff8d4feefba469489c81d43f3e0
2023-10-13 12:57:23 +00:00
stephan 7dc0cc45f3 Round one of an audit for SQLITE_ENABLE_API_ARMOR for functions exposed by JNI and those functions missing armor, as [forum:5e3fc453a69b49ca|reported in several forum posts].
FossilOrigin-Name: 8c25c4b18ad07861bf0e47f99f3db04b569b9b859ad0690602f748ddf3576939
2023-10-13 12:48:35 +00:00
larrybr 99d43979d3 Avoid potential overflow in hex(). [forum:/forumpost/7ac0c9c5ea|See forum post 7ac0c9c5ea.]
FossilOrigin-Name: 2e968114fd0e60eebcc088dec02684e10b06ac1ec42d6bedb5287362f5cbf032
2023-10-13 01:39:47 +00:00
drh 5dc4a405c5 Make sure virtual tables have been connected before trying to invoke
the xIntegrity method during PRAGMA integrity_check.

FossilOrigin-Name: 4a4eccb681741917c6771101d4e329f91eae9b035f4d345218e522bb55fff6da
2023-10-12 20:51:17 +00:00
dan 3046039309 Fix a problem with an fts5 secure-delete on a rowid/term pair that follows a legacy delete of the same pair.
FossilOrigin-Name: 579aea0c28e01a79620ac758edc02db3a01baaa073e7773b8f0b6f610479520b
2023-10-12 19:46:58 +00:00
drh 4b136ed70a Enable SQLITE_VTAB_INNOCUOUS for FTS3, FTS5, and RTREE.
FossilOrigin-Name: f34c533b6c1f8ef3f69c75c1039406c12751cdde1fbdcb4d9776b24455facf8a
2023-10-12 18:46:27 +00:00
dan a11499a156 Add test case for [29937081].
FossilOrigin-Name: d3b983edf7164d30ddb8b4b745a3feff9737149d85b1c428d6ad488ac6a1960a
2023-10-12 18:09:27 +00:00
drh c7ad9f65fa Merge fixes from trunk into the jsonb branch.
FossilOrigin-Name: a760bd7bcc63aac41c989bb5f4fbc927f9a93fe9c0aa18da2f0131483ec3f189
2023-10-12 17:47:50 +00:00
drh 5d9a6c6734 Make sure the journal file is closed when transitioning into MEMORY journal
mode, to avoid an assertion fault in the new sqlite3_randomness() avoidance
code added by [c84e4483cb44f827].

FossilOrigin-Name: 29937081a986d88f495ad48748c35ff5829f0ac31dd4ad3e48d180ae2fcb9a0c
2023-10-12 17:41:18 +00:00
larrybr 35d6c24fd5 CLI to check for rare prepare failures. [forum:/forumpost/5be6395182|See forum post 5be6395182.]
FossilOrigin-Name: 286d1e8012a5f5d9e1fe6dce1a732f725b8a1837f66daf6aefb792e19e96f2d4
2023-10-12 15:12:38 +00:00
dan 03204e9106 Add the tokendata=1 option to ignore trailing token-data when querying an fts5 table.
FossilOrigin-Name: 122935182ad5869ce3a4c6d796c38a0509f6f3384dd1b3e60a3f2f0f366cc5f5
2023-10-11 21:08:12 +00:00
stephan 6a0f297da9 Unconditionally force use of SQLITE_ENABLE_API_ARMOR in the WASM and JNI builds. Their corresponding test suites still work.
FossilOrigin-Name: c11394b50d0687b6d6abad9c80d5cd3cdce77e9b9c278f40601c460f463744a1
2023-10-11 20:37:42 +00:00
drh ac4aea5102 Fix the use of an uninitialized value that occurs when doing a json_insert()
of a string value that contains embedded U+0000 characters.

FossilOrigin-Name: d3c0cbb90966316be9cd25e3edb501da42731e8a83c13227b90ce17d3975a2c3
2023-10-11 17:24:31 +00:00
drh 7d1c9da62d Fix the use of an uninitialized value that occurs when doing a json_insert()
of a string value that contains embedded U+0000 characters.

FossilOrigin-Name: fc5ee9e51ad4556af526a6cefca5ae5a3b1b7affc4edf09832491d6b4f4ba366
2023-10-11 17:21:16 +00:00
stephan 582d65cce3 Add (prepare, step, reset, finalize) parts of the JNI level-2 stmt wrapper and associated tests.
FossilOrigin-Name: a7082f186f2b9b6666bbc65f2eadeb74d91fa0a681e3b2468b261ffd322bd249
2023-10-11 13:52:05 +00:00
drh 58c7b770de Simplification to sqlite3ApiExit(). Generates identical machine code, but
easier for humans to read.

FossilOrigin-Name: 793bbfa5af9721bc3a61e8e5eda46dfce2f5ff3f223a7564c9e1b09f11e53cb3
2023-10-11 13:34:18 +00:00
drh 86db4555ca Fix a missing zero-terminator on a string when processing
JSON aggregates into JSONB.

FossilOrigin-Name: fb81d570a3d9b8f55e9f278d5240605c5d0f3c5abde51550797999d03cf193a7
2023-10-11 13:19:37 +00:00
drh 2b7a1f5926 Fix an issue that can arise when processing corrupt JSONB.
FossilOrigin-Name: e50045c22296be84c6bea82bb8b310f07bca820c84d4a7349b16da0cf5d2657c
2023-10-11 12:44:17 +00:00
drh 33b56217cb Improved robustness against corrupt JSONB.
FossilOrigin-Name: 0fbda92bb0eeb40f95c83f717e4e8f5bff1ac82f1c899e9f6d400d67df67214e
2023-10-11 12:21:29 +00:00
drh 2a96a1584d Fix an assertion fault in json_patch() that can occur if the patch JSON
is malformed JSONB.  Report the malformed JSONB instead.

FossilOrigin-Name: a72d54645ca0dd80c60a5ed586049dead3ea7f5fa9ad05c6610a506242a7032a
2023-10-11 11:42:06 +00:00
drh 71dc3c714b Improved robustness in the decoding of JSON5 text escape sequences found
in malformed JSONB.

FossilOrigin-Name: 35e0108af2bdd830375c31c525f8ed0e8df64959d89649a88402dc1a5c376612
2023-10-10 23:02:31 +00:00
drh 6b7b23c580 Fix an off-by-one error in the changes from the previous check-in on
the jsonb branch.

FossilOrigin-Name: 216191b113da43516d31301fb133173add66358d503677ab997bc82cc88bfea4
2023-10-10 18:55:29 +00:00
drh 59ded6b5f1 Improved robustness when translating corrupt JSONB into JSON text.
FossilOrigin-Name: 0caa320d9099adbaf98e3719003dbdc4d158abcb3d8a1af20fbcd4c08c970f4a
2023-10-10 18:42:08 +00:00
drh d88f378c7d Fix a potential buffer overrun due to corrupt JSONB.
FossilOrigin-Name: 5cbb861fc6cb7421a81c957ba9d576e0ae92cb4ed725cdb539b364d66f4ee145
2023-10-10 18:32:14 +00:00
drh 064c1688da Fix bugs uncovered by the fuzzer.
FossilOrigin-Name: c96eb7fb618dc0a5aeec8a5e85076475b77dcd56309438aba1f9bddfc8921e3c
2023-10-10 18:04:40 +00:00
drh cc5e612db4 Merge the latest trunk enhancements into the jsonb branch.
FossilOrigin-Name: f93f16c94d0a58dfa90939e84aba048be958c7b085b45a6f25c2243fdf79b3aa
2023-10-10 17:34:41 +00:00
drh 853fb5e723 Fix a bug in the mkautoconfamal.sh script, introduced by check-in [3308fdda4b81c110]
so that the "snapshot-tarball" and "amalgamation-tarball" makefile targets
work again when run from outside of the source tree.

FossilOrigin-Name: 65ccf5fef812d43aed9e00af36c90e1a499d197e30148753790445e25ee1324c
2023-10-10 15:33:05 +00:00
stephan 0b29d37ef9 Start adding tests for the second JNI layer.
FossilOrigin-Name: 0177f4dfe8a5c03c1d5dd279c6babe5a87b596c2351373f845b89fc6abf7947b
2023-10-10 12:16:04 +00:00
drh aaf1341dcd Use snprintf() in place of sprintf() in a debugging function in the CLI.
FossilOrigin-Name: 5e5e4d2b4958c329578d1e5af4ed25c1b774f0ed2d15fa6747d7d7fbbba4fa46
2023-10-09 22:45:10 +00:00
drh f32aa34346 Systematize the names of some of the translation function in the JSON
implementation.

FossilOrigin-Name: db44bd1d62084ef69c808f7d07e0a25d5a089dcb8b98b1b5d026777591bbdefc
2023-10-09 18:33:01 +00:00
dan 6c96bf2c3f Avoid calling sqlite3_randomness and taking the PRNG mutex when writing a journal header in "journal_mode=memory" mode.
FossilOrigin-Name: c84e4483cb44f827416d8caafa22f076b2f31b2024fe8c5b5bcb0c9955149d11
2023-10-09 17:54:34 +00:00
drh 01a5a19919 Add a TCL script that does various verification checks on the source tree
to make sure that generated code has been updated correctly.

FossilOrigin-Name: 1f1a358af77f4386f98010eeae8487e6d39548a6dfe58c2664552490e7661122
2023-10-09 14:56:15 +00:00
drh 7e60f6dbf1 Fix a compiler warning caused by the previous check-in.
FossilOrigin-Name: 7e8768bf8b4002b1c287f2bc95262548e2ae81b437936154f2bb1ea1f739a904
2023-10-09 14:47:25 +00:00
drh 50da20d5b1 Turn on SEH by default when building using MSVC.
FossilOrigin-Name: f57e8275cf1ab3ad8027044cda0893282d1c403a5aa9fabbb1c7de280597b5cd
2023-10-09 14:05:21 +00:00
drh 0dead8d3d1 Merge the latest trunk fixes and enhancements into the jsonb branch, and
especially the JSON cache spill UAF fix.

FossilOrigin-Name: 9422c24f4a8b290dcae61e50ec81be5b314b22c61a2bca1e194e47da1316b6e6
2023-10-09 12:57:03 +00:00
drh faef28e6bd Fix a potential UAF caused by JSON parser cache spill.
[forum:/forumpost/b25edc1d46|Forum post b25edc1d46].

FossilOrigin-Name: a163fecca90cab9d1b7bf8ebac78d498775eed7b6d81e7920e3401633c3a4b60
2023-10-09 12:46:53 +00:00
stephan 66dacae4c3 Flesh out the error state captured by SqliteException.java. Doc additions.
FossilOrigin-Name: 5c5397ff15543f4b3620244d9e57e15708eafcab1d42c9f87b4a60f0c01e8858
2023-10-09 12:45:28 +00:00
drh 4a11142072 Fix harmless compiler warning in the test module for sessions.
FossilOrigin-Name: 3d5ec99397c183cfb8c5d31e1967e0df8a4d6edbea92bbdf728571e3a6a36d68
2023-10-09 12:39:59 +00:00
stephan c8f829cdac Add JNI Sqlite and SqliteException classes. Add Tester2.java as the main test app for the high-level API.
FossilOrigin-Name: 6acf52be7abce8dcf434c5ebf0d5e88859b033e6418077846247ecf00ccf9381
2023-10-09 11:46:32 +00:00
stephan fd013682d6 Minor JNI doc and public/private cleanups.
FossilOrigin-Name: c49d36ece283274963ce2e5a4db1e8f586dffa22e47f4adb93c625f918c3fd5d
2023-10-09 10:44:10 +00:00
stephan 68585ea342 Rename SQLite3Jni to CApi to (A) reduce name prefix collisions with incoming classes and (B) align with its counterpart in the JS build. Remove the Canonical annotation because (A) the new code separation will inherently make that distinction and (B) the line between truly canonical and semi-canonical (e.g. differing in signature overloads) is blurry enough that consistent use of that annocation is becoming an unnecessary burden.
FossilOrigin-Name: ca216b4486aff7d206ebfc3a5e84d48919c282425d4313396bc19313ffca8a0e
2023-10-09 10:30:54 +00:00
drh a7e9386e88 Remove some unnecessary code. Report errors for invalid JSONB input on
an extract.

FossilOrigin-Name: cbea16c29eb0507f39b5a1cf744a3bb9bb7c71ac156e84a19d03a37cb1816891
2023-10-07 23:35:07 +00:00
drh de8ccf00fb The return from sqlite3_value_blob() in jsonFuncArgMightBeBinary() might
be a NULL pointer.  Check for that case.

FossilOrigin-Name: 7b52b266b066f1385144c1103a3a411306db5f44568366ae1e93cd8cce799bbc
2023-10-07 19:46:22 +00:00
drh 3efb2c4791 Improved detection of malformed JSONB when parsing it into a JsonNode array.
FossilOrigin-Name: ed99a788415e1f8375bd5ec004dd18b1cd0fae4aa94558170882ca487f6dff93
2023-10-07 19:40:20 +00:00
drh f26833d783 Fix the jsonbChangePayloadSize() routine so that it shifts the payload in order
to always render the most compact encoding of the payload size.  This is
necessary as sometimes (as discovered by dbsqlfuzz) the payload size can
grow significantly due to json_insert() or json_replace().

FossilOrigin-Name: 8d6d04ca975ec55c419d40d8463c433b0db698c9fb4812ab9f16d4ee5bee460e
2023-10-07 19:05:10 +00:00
drh 1244b6cbf6 Improvements to error messages returned when the ->> operator fails.
FossilOrigin-Name: 2f3388f14c843f1c02926e8b929365c06c1f1f4ea6fe6316092c3799c14549d3
2023-10-07 17:50:06 +00:00
dan 3b381b4afc Fix a problem with sqlite3changegroup_schema() and patchsets.
FossilOrigin-Name: c4ab4200dc2538e1132d87d70fd309b26b0da8a918fede1cb09b567ea40ec889
2023-10-07 17:29:10 +00:00
dan 4bdcfc6666 Add the sqlite3changegroup_schema() API. To allow changegroup objects to upgrade the schema of a changeset/patchset under some circumstances.
FossilOrigin-Name: 03f9e50b15c9c2fb4636dee24c1491c45322578b9fed08c6f4360be07c5c3830
2023-10-07 16:32:02 +00:00
dan 18efe17553 Fix a problem with the changes on this branch and tables that use an implicit primary key.
FossilOrigin-Name: 1f9d10235ff32e72074be695049567135b02e681068ce117b66adc14a6214449
2023-10-07 16:07:02 +00:00
drh 502b11c6d3 Merge the memory leak fix from trunk.
FossilOrigin-Name: 358de1b09f3d5ec0fe459775b0a2a99dfa235817327016b472aaa1ed56d952e6
2023-10-07 11:37:00 +00:00
drh bae197c175 Improved error messages from search on JSONB.
FossilOrigin-Name: 96cfdc31e305406123f6ef1b920423f71902dc4b34381c3efb720274343d37cc
2023-10-07 11:36:16 +00:00
drh 522e880bc2 Report unknown JSONB element type when parsing JSONB into JsonNode.
FossilOrigin-Name: 1f4d3268257bda234c66c3767451b3d39d7b4c7c73a779ad396d9dfc5315905c
2023-10-07 09:13:59 +00:00
drh 8f0634549c Add a test case to prevent a regression of the leak fixed in the previous
trunk check-in.

FossilOrigin-Name: f99ff655d09763c4a22d065041644ece793d84c82c644931e89ccf50c4f4564a
2023-10-07 08:15:47 +00:00
drh 43e232dc47 Fix a resource leak: sqlite3_blob_reopen() was failing to invoke the
xDel() distructor on a register when it sets that register to the new
rowid.  This was never a problem prior to the large-column cache of
[771fe35074b50b8d] as that destructor was always a no-op.
dbsqlfuzz cd96368deaece480fb94d42427dde053737a650e.

FossilOrigin-Name: 545edd5ecf20a7feb99dc5ce6f338fa1f172c2e17b240cbede497f0729dbd3d7
2023-10-07 08:00:54 +00:00
drh 11aa2adca3 Correct handling of "raw" strings in JSON. This requires three test-case
changes in TH3 to add double-quotes to the path outputs from json_tree().
The new behavior is correct, I believe.

FossilOrigin-Name: ab2bf3e3596dfa7566f4a130adeccbef4056c7321181112a875477193583f30c
2023-10-06 23:02:06 +00:00
drh 6b8aa95c3c Improved error detection for JSONB inputs.
FossilOrigin-Name: 6945e11aa441ace52b93cdbb81ac45b6f809987f3dbc522251e63f8cb948fddd
2023-10-06 22:16:09 +00:00
dan 83e43dc757 Add missing source code comments and fix other issues with the new code on this branch.
FossilOrigin-Name: df39fbe9ab87937beb77af353cd55602290b185e222e0a92a3ebf9a9a0b9e2e7
2023-10-06 20:39:42 +00:00
dan 9fc208aada Update this branch with latest changes from the trunk.
FossilOrigin-Name: 897e926a5eaa0eab7e09e5003e4c15c974897be298a18ae85345962dc3f410a1
2023-10-06 19:46:19 +00:00
dan 53e91a5e22 Add tests for the sqlite3changegroup_schema() API.
FossilOrigin-Name: 5dab481c101b1523b1cfde92678cbc654ea26d946bf29da372d71c1f89cbaf46
2023-10-06 19:01:43 +00:00
drh 7b4349735f Incremental improvements to the JSONB logic.
FossilOrigin-Name: fe326829c27715e249f727ba797c7df6491e874ec205a0a82ee09c78d61c6e1f
2023-10-06 18:21:47 +00:00
drh 98dbfa69d3 Fixes to rendering JSON5 extensions encoded as JSONB.
FossilOrigin-Name: 5a17e4479aad2d8313170e5de83a1c52f30b55d9d4fb776024fa6622e175c63b
2023-10-06 15:35:42 +00:00
drh e690d5ad80 Fix compiler warnings.
FossilOrigin-Name: 5227add3c8d509de2e081249163fafdf30ac3173a6d710957f3c3b6f03e7017e
2023-10-06 14:59:40 +00:00
drh e1e2f2dcc5 Improvements to error handling.
FossilOrigin-Name: b41dd237fb6c0dd7daedeaaf81dbc4fdb31cf511fd32cd3d4ee69db34e389915
2023-10-06 14:52:49 +00:00
drh 91f3cd9874 Merge compiler warning fixes from trunk into the jsonb branch.
FossilOrigin-Name: 6409d307915f8969f12df2d5ffa961645bd4db7ccfbd6f52237a217b6a867252
2023-10-06 13:05:42 +00:00
drh 00eee7a786 Remove an unused parameter from the recomputeColumnsNotIndexed() routine in
order to squash a harmless compiler warning.

FossilOrigin-Name: 9bf4bfd68080367b58594e0d44b110b3ee9766420f648537fd7bc638dacefb72
2023-10-06 12:55:53 +00:00
drh 1935887a68 Ensure that all fields of static sqlite3_module objects are explicitly
initialized, in order to hush-up nuisance compiler warnings.

FossilOrigin-Name: f3b3d712d6e58b1cb8fdebd2b6b3125080b6b3ac8c7c849a8cc1e5e778d62fe7
2023-10-06 12:51:05 +00:00
drh 03f7e42538 Remove and unused static variable from shell.c.in in order to silence a
harmless compiler warning.

FossilOrigin-Name: 477577120b897bf15edc2a85e220d5c64a1d5a245354939269f6753d8140ac95
2023-10-06 12:22:14 +00:00
drh 568643fd15 Increase the size of some variables associated with the PMA sorter in order
to avoid any possibility of a signed integer overflow.
[https://bugs.chromium.org/p/chromium/issues/detail?id=1489025|Chromium fuzzer issue 1489025].

FossilOrigin-Name: dc3be3af471d90dd810c1e2cc59e83cf6a57f01971b1258ea0da402e4d577ef2
2023-10-06 12:15:01 +00:00
drh dd7f09e6c0 Improvements to coping with malformed JSONB.
FossilOrigin-Name: 563cde404cec4c6559ae0c5fc5edbc878fee874b36562ce2fac5049cc8349343
2023-10-06 00:06:26 +00:00
drh 27401d38d8 Better error detection when doing a lookup on a JSONB.
FossilOrigin-Name: 6e8e0eedbf06e175af6ef230f1ce9a5ad84698b1c6dcd2794d878c746e7b6be4
2023-10-05 23:05:35 +00:00
drh f7af8f3270 Improvements to comments. No changes to code.
FossilOrigin-Name: ac74d7877685006e43684edd6a2d22be8c9857f9f9eb52fc5b3c182d5e2fdb8d
2023-10-05 22:52:43 +00:00
drh dac2707152 Turn an unreachable branch into an assert().
FossilOrigin-Name: 0f75199160e48fa8c44f986f1af777adf19e40fbd114a6f58e24d5e6dede779d
2023-10-05 20:17:01 +00:00
dan 16381d062a Add the sqlite3changegroup_schema() API. To allow changegroups to handle differences in schema created by ALTER TABLE ADD COLUMN.
FossilOrigin-Name: 309deee2dd8dd07623fce79f6bb62d5279d140dd0be3b34bc42af20b0507726b
2023-10-05 19:09:23 +00:00
drh 51d507d432 Fix the parsing of non-standard "Infinity" and "NaN" values from text into
JSONB.

FossilOrigin-Name: df1fbbeb83a2b4a496c9de0d86c7c8c677f8fe3bc770da163dcc1d338a17b58b
2023-10-05 18:33:19 +00:00
drh 5b6cfb7906 Change the json_valid(X) routine to return true whenever X is a blob that
could plausibly be a valid JSONB.

FossilOrigin-Name: 425f0b85a6f8ad0604c4a5faa18efb90436fedb1fe2612a559147c62cff8b7e7
2023-10-05 18:09:12 +00:00
drh 14fce24a9b Fix the text-to-JSONB parser so that it handles some JSON5 floating point
literals correctly.

FossilOrigin-Name: 564edb3b6dd752e09e445e3a25c2f5394ceede2a2cdff251d6433dadc0b3644f
2023-10-05 17:52:39 +00:00
drh 0585d9f08f Slightly stricter testing for when a BLOB is valid JSONB: If the element
is a null, true, or false, its payload size must be zero.

FossilOrigin-Name: 487781be8a93d9b8d870efb9f25822bb4bbcf9d39d571d1017e9c29043ea515d
2023-10-05 16:33:00 +00:00
drh 01acfd5805 Fix a memory leak following a syntax error in jsonb().
FossilOrigin-Name: bf4b36eda8e200b67041ebdf60fc31d0325bd84185272f8d390c56a91d6ac1ee
2023-10-05 15:05:33 +00:00
drh 9061e22a05 Allow the PG-style syntax for the PATH operand on the right-hand side of
the ->> and -> operators.

FossilOrigin-Name: bae5071b0834aa3b7fd4bd871f863e1b148c6558989c8f6cdd02dc4da4770953
2023-10-05 15:02:00 +00:00
drh 0f921e2bac Merge trunk enhancements into the jsonb branch.
FossilOrigin-Name: be5907b648386e05530d1e321e2a3e563e07e99f08373aaf9a3452adc9228dc3
2023-10-05 11:22:16 +00:00
drh 5d9446d100 Improved detection of corrupt databases in balance_nonroot().
dbsqlfuzz d1cf013c50a620b68c2a5bd240d29afd65e5f58e.

FossilOrigin-Name: a2464bbb825b5976ef974a2e6c17ea150f5e6fcd0dd0f144b9f9c1c22a9c9c82
2023-10-05 11:06:10 +00:00
stephan a357bdbfc4 Do not squelch exceptions from SAHPool VFS importDb() - rethrow them. Problem reported in [forum:c80fc578809b80a3|forum post c80fc578809b80a3].
FossilOrigin-Name: 325bcdea4a436e514d83095d3893caf07a714b718e1b643507179b101c108e3b
2023-10-05 11:04:49 +00:00
drh 3a3912f21c Remove a stray blank line from the beginning of btree.c. No functional
code changes.

FossilOrigin-Name: c89983dcfc208faa86b31717144c069572598e9770228264d69a044a84bde277
2023-10-05 10:35:51 +00:00
dan 6d8e91be9d Allow a session object to generate a changeset, even if columns were added to one of the tables using ALTER TABLE ADD COLUMN while the changeset was being collected.
FossilOrigin-Name: a3f435eccf3a2aa11cb7420e94af5efcdfa04e9c169c5aaf61fa5cdcb165ceef
2023-10-04 21:15:24 +00:00
drh 10e751937c The Makefile distinguishes between tcl8.4 and tcl8.5. Some makefile targets
require tcl8.5, but others (ex: "sqlite3.c", "shell.c", and "sqlite3") require
only tcl8.4.

FossilOrigin-Name: 770308db9776b8c0a70b8807463e89a9eddfe5552e25e67324cd303dc974f50e
2023-10-04 12:49:08 +00:00
drh 57996f53e4 Modify the main makefile so that it fails with an error if the tclsh found
by ./configure is not tclsh 8.5 or later.
[forum:/forumpost/4380363682708ece|Forum post 4380363682708ece].

FossilOrigin-Name: 1765f3b5a00a8ca20a7b9e18ac7f9d7de0679470b234b83aea83aa5b4d4d34e6
2023-10-04 11:52:44 +00:00
drh 4500d87e76 Fix a memory leak in JSON group-aggregates when the output is JSONB.
FossilOrigin-Name: 08e7db138b636890cb29a76d65c2069c6e2ff44470fa4bca14f4526fe5f195ae
2023-10-03 22:40:22 +00:00
drh f362731c1c Fix a bug in the jsonReturnFromBlob() function that causes a positive result
for a negative value for when a JSON integer is too large and needs to be
converted into double.

FossilOrigin-Name: dca684da0c29ec78460362f972ea7747be42c13c4d1325da9d62c1ea58022e39
2023-10-03 21:54:09 +00:00
drh 51cc3041fe Fix problems in the "json" output column of the json_tree() virtual table
for the case when the input is JSONB.

FossilOrigin-Name: fefa4475c496aab32fe7b9cc203747ce4faf7448da4f29b807e5486c7686238d
2023-10-03 20:58:39 +00:00
drh 6e737b9252 Improved handling of OOM while translating the JsonNode representing into
the BLOB representation.

FossilOrigin-Name: ef5956710bb542d6045c82937d02218a7ed45af94cf3959b0c180268e04d14e1
2023-10-03 19:37:19 +00:00
dan 1846de49a4 Fixes for fts5 expression parser module to allow embedded 0x00 bytes in tokens.
FossilOrigin-Name: 342c8d0783f449817d3f565ff6b9f010a6c690beeea32f1861640810490a8b5f
2023-10-03 19:06:52 +00:00
dan eb28787b5f Update fts5_decode() to allow for embedded 0x00 bytes in tokens.
FossilOrigin-Name: e051120067fd87f57b498e505e3960cf4d14e8e33bad940618cc0823253254f7
2023-10-03 17:07:54 +00:00
drh f0c129f247 Merge the latest trunk enhancements into the jsonb branch.
FossilOrigin-Name: e6406a9865b75dea2f26d3ee4f4c206958400059c7f92ced88edc8507dd3c82f
2023-10-03 11:36:47 +00:00
drh ee0f6777f9 Fix a problem in the JSONB parser that comes up following an OOM.
FossilOrigin-Name: 355acfb18897254f6b6444a21d781b5e10e930b81952850dd2a40d88bbf2f3db
2023-10-03 10:59:01 +00:00
drh 8405e1140a Fix an issue with the use of jsonb_group_array() and jsonb_group_object()
when used by window functions.

FossilOrigin-Name: 808bd349ba587fbcdc4aea3f9c08e0df01ba08dec181c5af5ea157e89d86ff7b
2023-10-03 10:43:30 +00:00
drh 313336f90b New test cases for JSONB.
FossilOrigin-Name: 6d4aeff5751722c83ebb0e1f21072b72be418c64dcf28ee032d3a548e1b3a951
2023-10-02 23:56:46 +00:00
drh e5c384e9a5 Fix jsonb_insert() so that it does not behave like jsonb_set().
New test cases added.

FossilOrigin-Name: 54197149b811d30b6c4487eedf5692b164ed0f90cfcc541aa3157094f5f17f6a
2023-10-02 20:16:06 +00:00
stephan 4f65a3cec9 JNI: when fetching text-or-blob/length pairs, fetch the text-or-blob first, then the length, as the API docs suggest.
FossilOrigin-Name: e4f9d3946fa570bccea2de17c5536901bdabd46cfe25678fdd37ba0e2bfd99b1
2023-10-02 17:12:16 +00:00
drh 51e3f7a1c3 Fix the length of the string in the new concat() and concat_ws() functions.
FossilOrigin-Name: 325243226520376f4a142864ae3fb98866f25bff4dc50a6b648ef98428a7169f
2023-10-02 17:06:28 +00:00
drh 43d71eb892 Remove an ALWAYS() from a branch that can in fact be false following an OOM
fault.

FossilOrigin-Name: 3a1feb1c469d3dd50e2558581199b16bfd59b9d5bcb0de5f1a8f0ec30a17b4d1
2023-10-02 15:56:37 +00:00
drh dee29e8c7a New test cases for JSONB.
FossilOrigin-Name: d624c31e5c49e1ce63b4b72caa42a61c5167866f47d842fbcfe4e826fd079d7c
2023-10-02 14:51:28 +00:00
stephan 041fb16631 Pointer-casting-related fixes for JNI on 32-bit ARM (pi4 armv7l).
FossilOrigin-Name: 54441039ae3ebe39e42f008f77ce1be117cbda6daeed8b01739ad42a3c1f0754
2023-10-02 14:44:28 +00:00
drh c97407fdba Allow json_replace() to accept JSONB arguments as the new value.
FossilOrigin-Name: 95eb7b37fab29931924311f541d52173ef77a448efc8771b1a1783ccd786d23d
2023-10-02 13:35:06 +00:00
drh 8eeafb7544 Improvements to error handling for BLOB inputs on JSON.
FossilOrigin-Name: 14f20ecbfab44934e86f1ac7a3f745b989aa8190c6df119ff5aa8100fa248d93
2023-10-02 13:20:43 +00:00
drh 5624b0b4cd Gather forward references into a single place for the JSON code. Allow
JSONB arguments to json_array() and json_object() and similar.

FossilOrigin-Name: c352201b8c299c330d9abbff6dbcbcbcf00ada53183d3cd91020ec772e066357
2023-10-02 12:40:04 +00:00
drh 9b176f863b Fix problems following OOM in JSONB parsing.
FossilOrigin-Name: 0d8cd6b5fb592f88f593ceaad9cdfa95dcfdf14169c3a56a5becbd66d62fd02b
2023-10-01 18:59:34 +00:00
stephan dc13d74140 Add JNI binding for sqlite3_normalized_sql().
FossilOrigin-Name: 2532e5662a2bbb3e6b4c4be3fad035734b684bdf59b178cb64b444ca6d8b956b
2023-10-01 12:15:38 +00:00
stephan ec15e551f9 Make JNI binding of sqlite3_value_frombind() return boolean instead of int and add tests for it.
FossilOrigin-Name: 906e2ed3e9647f5c2355e9fea2beee141767cae7016da9fa1fe88283726b9369
2023-10-01 11:53:40 +00:00
dan a35ae44150 Changes so that fts5 can handle tokens with embedded '\0' bytes.
FossilOrigin-Name: c027c092c4af53bd6ae3cc6e2b4439167d9eeb0f9de549b6a2c2a72a67ee886c
2023-09-30 18:13:35 +00:00
stephan f046b82324 Fix JNI binding to compile without SQLITE_ENABLE_PREUPDATE_HOOK. Add build option to disable all optional ENABLE flags.
FossilOrigin-Name: c04022b7407f77eaf0175e831ebcd6bbdc0af1cef0d42c5c11102aa8484f24ca
2023-09-30 17:08:29 +00:00
drh d8f26350a0 Restore the trunk version of sqlite3_user_data(). Fix the xColumn for virtual
tables so that the sqlite3_context contains a valid but NULL user data pointer.

FossilOrigin-Name: 15ffd932fecc82a5791b2024f55d2ec80887e8eb7345de68d6f5cac4912cfbe8
2023-09-30 16:50:17 +00:00
stephan d1878bed86 Make all SQLite3Jni methods which take a void-pointer-as-long-int argument explicitly package-private, as they're always for internal use only even if SQLite3Jni is eventually made public.
FossilOrigin-Name: 5e387275f69ab2d3159b4b67b8cbfc6270410b61e5ac1f988616e8d051f6572e
2023-09-30 16:19:57 +00:00
stephan 41c6232e54 JNI: rename sqlite3_errmsg16() to sqlite3_errmsg(), as the 16 suffix is superfluous there.
FossilOrigin-Name: df25b21689de84b7b8c940df39461d3d88f592f0ba2dddcfcbb0bebcea6ee7da
2023-09-30 14:48:21 +00:00
stephan 0018ef15a3 Expose sqlite3_stmt_busy() to JNI.
FossilOrigin-Name: 5733aa5fe341d50b84baebd8803a86e381bb95959a8bc733b8fff6fd4e732867
2023-09-30 14:34:55 +00:00
drh 7e86d3fc69 Finish adding jsonb_ versions for all JSON routines that return JSON text.
FossilOrigin-Name: 6daa7b69695e1b68dba317abbcad4d0205c91963d4a9eb2d595a3ec10fa0fdf4
2023-09-30 14:34:39 +00:00
stephan 8f43696c8a Correct an undefined-value deref in the JS bindings, triggered by passing invalid args and then tripping over an undefined value while trying to report that. Reported in [forum:dd42df144651d779|forum post dd42df144651d779].
FossilOrigin-Name: fd4632540717c4771ac9ae1fe4400091a2cd0553639bf809abaf4ecbc54de4f0
2023-09-30 13:13:23 +00:00
stephan 9ac7e2d0fc Make (almost) all SQLite3Jni.sqlite3_... members public but make the SQLite3Jni class package-private, which has the same foot-protection effect but makes it much simpler to patch SQLite3Jni to publicize the interface later. Remove some of the extraneous foot-shooting protection from indidivual functions, in lieu moving it to the still-pending higher-level interfaces.
FossilOrigin-Name: 082b86683e77b2d4a30e1515552f52511324ef97c43b622b4dab7975ae9fbd7d
2023-09-30 10:59:18 +00:00
stephan 008b62f1ae Merge in and close jni-ptr-passing branch. The remaining pointer-passing adaptation can be done incrementally without affecting the interface.
FossilOrigin-Name: 7927c42410ba0d5fc8e6e08caaa5cbaaeb359869f17d1afaa082f1aa47fdb2fc
2023-09-30 10:37:36 +00:00
stephan d1ce2755f0 Make all native JNI sqlite3_...() bindings package-private as a foot-shooting protective measure (higher-level pre-native-call argument validation is largely pending). Move SQLTester.java and TesterFts5.java into the org.sqlite.jni package so that they can access the being-tested methods.
FossilOrigin-Name: ec82f7251acab7df40755ef5f456f36fe49b59e63a20be59bd610fc4280ba8cd
2023-09-30 10:31:56 +00:00
stephan ebce46f02f More work towards the new pointer-passing mechanism in JNI, and code-adjacent cleanups.
FossilOrigin-Name: 6c63987e893357dc8b10decaa96c30fb37b75481640a303e77a0d8224354491e
2023-09-30 09:41:58 +00:00
drh 5736ba5ac3 Update json_each() and json_tree() so that they work with JSONB inputs.
FossilOrigin-Name: bb5e50ff56dff95d954aacdd4c5461790f953cef8d7b89da000d8d587fcdf9b8
2023-09-29 22:37:18 +00:00
drh ee50aad865 The u.zJContent field of JsonNode for a string or label should NOT include
the quotation mark delimiters.  Ever.  This is an inefficiency that really
ought to be fixed on trunk, but that can wait until this branch lands.

FossilOrigin-Name: 96f545f6f839dab4829861361ee3d7a56840217c5f954f334616e77d23c5fe29
2023-09-29 19:47:25 +00:00
drh a99e2fb576 Additional refactoring and cleanup.
FossilOrigin-Name: 45dd1760875b1ad899a10189c6f5a0a9a0677903794fb5a06ffacd39952a7882
2023-09-29 16:37:22 +00:00
dan bee5657042 Add test case to ensure the assert() removed by [f090af77] no longer fails.
FossilOrigin-Name: 615ab71761754b072439f92e73fdb98c916b820937bcca8904f4be34936d5f9c
2023-09-29 15:56:40 +00:00
drh 8be48440b6 Remove an incorrect assert().
FossilOrigin-Name: f090af775f6d2ce8b50850f5c9e33546bcb9cff5467d99ca210530cb7ca83d5b
2023-09-29 15:53:55 +00:00
drh ae5f55e227 Improvements to comments and procedure names for clarity in the JSON
implementation.

FossilOrigin-Name: 9b620d813ef483f1277c1683c5e926a882f07f3b90804dea0c91b325ff8e45a4
2023-09-29 12:45:14 +00:00
drh ecce6022d5 Describe the JSONB encoding in a header comment to the json.c source file.
FossilOrigin-Name: 1c0cba3461d6111b3aeb77726880221f1240355f0b57e060febbdeb12fb688c0
2023-09-29 11:17:43 +00:00
stephan 83f62816f8 JNI: correct a NullPointerException triggered via SQLTester.
FossilOrigin-Name: 0a873de76c0cbcd8e2eda3f0508e427f1dcb32b01798687c0545acfe10102179
2023-09-28 20:34:28 +00:00
drh 8cdab14286 Fix some minor memory issues so that all legacy tests now pass.
FossilOrigin-Name: 1744bfc669346ff221f28d45fd978863e876a2d2f0b82bcf0e5ee6f0326900cc
2023-09-28 19:11:36 +00:00
drh 0a19bb078d Allow the sqlite3_user_data() function to be invoked with a NULL argument
or with an sqlite3_context pointer from a virtual table.  It returns NULL
in both cases.

FossilOrigin-Name: 2f49687371ada65fef374336c28b352c48ab98dc31282ac82397035efe04ba11
2023-09-28 18:23:52 +00:00
drh edbc44c5a1 Merge trunk changes into the jsonb branch for the compiler warning fixes
from two days ago.

FossilOrigin-Name: cee113cc315b04fd75ccc172cf4529bf15b2050bf274433496c31a282e281ab8
2023-09-28 17:41:45 +00:00
drh 77e00f24d7 The json_extract() function now works on BLOBs without the need to translate
into an array of JsonNode objects.  This is much faster.

FossilOrigin-Name: 5853065b7ee326c04ddfcde64c178f487af04cd3adc3cc99f559907484ec169d
2023-09-28 17:27:24 +00:00
drh 6b1db92228 Extract directly from BLOB is now complete and appears to work.
FossilOrigin-Name: 3de58ec99444b16dfcda1e226420e2343450b77abd3faf33a88b6d18339ef17c
2023-09-28 17:23:46 +00:00
drh 59862e6d22 Miscellaneous bugs fixed.
FossilOrigin-Name: 5c0815fa2e422d81198a43a2c04a022e319fcbcadfd4be4437f2e663892ca26b
2023-09-28 17:07:43 +00:00
drh 2dc60ec57f Improvements to json_extract() to better support JSONB. Still not 100% working.
FossilOrigin-Name: 8c82576176539c4d132b14d46adbf31366c4bcaa59a61dd639dc9cc308fe8825
2023-09-28 15:56:35 +00:00
larrybr d93cfa2831 Clarify serialization API doc w.r.t. buffer validity and usage, instigated by [forum:/forumpost/8922e2ad2ad41205|forum post 8922e2ad2ad41205]. No code changes.
FossilOrigin-Name: b488b9fb71652eca90d2bf73d32f3d748badf517859dc833c58e021b0e017194
2023-09-28 15:00:25 +00:00
stephan 0dabcd139d Adapted the JNI bindings for the sqlite3_value_...() and (pre)update hook APIs to the new pointer-passing mechanism.
FossilOrigin-Name: 4182f0275d5d65e04a130eeef4d44642a5ffeeb4b84430d240ea2605345f1404
2023-09-28 11:19:37 +00:00
stephan 5d1448d08a Convert the remaining macro-generated JNI bindings to the new pointer-passing mechanism.
FossilOrigin-Name: 250fd6ae806cf705c0f29ad30ad8fb885b12590848e7adae63bc21d874c6d3bd
2023-09-28 10:50:26 +00:00
stephan 50b2a41330 Convert the macro-generated JNI bindings which take a db pointer to the new pointer-passing mechanism.
FossilOrigin-Name: 5f47fb77db4ee77afc541e680559ad88e66ba7fd04b830e70f8be92cf8d0a60c
2023-09-28 10:27:01 +00:00
drh 1854837b5a Work toward getting json_extract() to operate directly on the BLOB, omitting
the translation into a JsonNode array.

FossilOrigin-Name: c1feba70f55a8e5f4696d48e4706855415d173ac8ac3c2656787c242a883b4f5
2023-09-28 10:20:56 +00:00
drh 733da8d31a Performance tests for JSONB added.
FossilOrigin-Name: 7c1be8e361db87458ac9d8fcee080c2b558936539c852bb80f0f7941d61bf15d
2023-09-27 16:55:13 +00:00
stephan 1d74432fef Adapted JNI sqlite3_blob_...() and sqlite3_close...() to the new pointer-passing mechanism.
FossilOrigin-Name: 0b22c8ef93e5ccd45316099fb8575e27620158b1992c0c70fe0348cfc10147f8
2023-09-27 14:41:49 +00:00
stephan a917ee7056 Adapt JNI sqlite3_bind_...() bindings to the new pointer-passing method and correct the mapping of sqlite3_bind_zeroblob64() to use zeroblob64() instead of zeroblob(). Related internal API renaming.
FossilOrigin-Name: 980d559fff6e55d1e2ef39f242a8a09313a936cfb141807db475bcceff924869
2023-09-27 11:01:32 +00:00
stephan 9afd67cce7 Add missing license headers to JNI annotation classes and extend the definition of @NotNull to include 0L as the equivalent of null to account for the current rework of how pointers are passed from Java to C.
FossilOrigin-Name: bccdfeb9efde20410bde545893fb98ce5c0c4d04a8e7797b868ba8994120e1db
2023-09-27 10:29:07 +00:00
stephan 4632f91403 Rework the JNI sqlite3_backup APIs to the new pointer-passing approach.
FossilOrigin-Name: 48aeb1e2cdeb4aec85c4f63a1f809215cd4b473791169e72b5ddf6d6bdc2f7b7
2023-09-27 09:58:36 +00:00
stephan 36f60e6c1a Start reworking JNI methods such that they pass void pointers from Java to C instead of passing their strongly-typed wrappers, as that is reportedly significantly faster than passing the wrapper objects to C and extracting the pointers there. There are still many, many functions left to rework for this.
FossilOrigin-Name: 66c814dd473731703ee00e1ff610acfdccf09d1b87dd2355795ed697d4ed5d3e
2023-09-26 21:37:52 +00:00
drh 61df55d0be Fix harmless compiler warnings in the testfixture testing utility.
FossilOrigin-Name: 5a39a02d2dfd9ed6171cd0bd434b2bc268d0ed8ead6e1a396d1603266d9493ef
2023-09-26 21:22:46 +00:00
stephan e81d229fb7 JNI: implement AutoCloseable for the sqlite3 and sqlite3_stmt classes, and adjust a few tests to use it. Override Object.finalize() for sqlite3 class but not sqlite3_stmt (where it triggers a JVM crash for as-yet-unknown reasons).
FossilOrigin-Name: b633d8bbdbd43464ee03058c273e5188117b9538ed5d5148a1a3851e4adf6410
2023-09-26 19:49:35 +00:00
drh 676bcf7457 Merge partial-index constant value fixes from trunk into the jsonb branch.
FossilOrigin-Name: 700bdbd7383f66a0da675c197204da4e7b6ed757155145ee98d572de32a5d0ae
2023-09-26 19:46:38 +00:00
drh 0e224d93ea Fix yet another assert() in the partial-index constant value optimization.
FossilOrigin-Name: f459d0806cf044fd07743e4c91d0a5a6ddf45b3b41004bde4278f190d99a4cf5
2023-09-26 19:38:24 +00:00
drh 42156fd90c Add in many jsonb_xxxx() interfaces. Still uses the internal JsonNode
representation for transformations and search, but it does at least conform
to the desired API design.  Largely untested.

FossilOrigin-Name: e6045b4e1bf3a8e33926fc12b3c039f5e1002eaecbe277ffa82b0ec271a29d17
2023-09-26 19:30:46 +00:00
dan 1c67749166 Remove a NEVER() from a condition in the partial-index constant value optimization that can sometimes be true.
FossilOrigin-Name: 63aea8c3e0605edcf49ad1e5ce1a56d2690dcb8c4857043997545daab46ca035
2023-09-26 19:26:47 +00:00
drh 219e7f7986 Merge all the latest trunk enhancements into jsonb branch.
FossilOrigin-Name: ac242c4d47ec36aab1c2fa5e65e7b595e686f49473b75bd63708d05c59ce3f0f
2023-09-26 15:13:04 +00:00
drh a6e7202677 Remove unreachable code from the partial-index constant value optimization
([8d4160910d651246]).

FossilOrigin-Name: 7ad38254c37153efa72291d09800693ca60894359548eda877d59defa8c70d49
2023-09-26 14:57:30 +00:00
stephan 6f737371c0 Remove some outdated JNI docs.
FossilOrigin-Name: 2ca5cbbb2a8ee94fe9f4888dea695a4429aefa17611fd9ebe59bf4866290ab0c
2023-09-26 09:46:04 +00:00
stephan efa8369214 Minor JNI doc updates. No code changes.
FossilOrigin-Name: 63687cc659f06a1f6aa9d4496ef01d9f0adbee6184b8e265fb8feec1ad30449b
2023-09-26 07:43:51 +00:00
dan 47b427423d In partial index scans, if the WHERE clause implies a constant value for a table column, replace occurences of that table column with the constant. This increases the likelihood of the partial index being a covering index.
FossilOrigin-Name: 8d4160910d6512469cb0a060b89c2509dde1c8e33d9358034d7fb39cd494eabe
2023-09-25 17:46:02 +00:00
dan 1b96f9167f Another very minor tweak to code on this branch to save cycles.
FossilOrigin-Name: 2c805dc68112db7f06aea56a731a33e046e876851bdda3e725ba96f6da1bf2ee
2023-09-25 17:21:53 +00:00
drh 09f23d2a36 When parsing JSON text into the BLOB format, only use node type JSONB_TEXTJ
for an unquoted object label if the object label contains escape sequences.

FossilOrigin-Name: a82ebbac3c542ec7f86d1e8414d7fd166db48450115ee3b26d12b5bb445f5896
2023-09-25 17:14:17 +00:00
dan cf0906575d Change the order of an if..else... in new code to make it slightly faster.
FossilOrigin-Name: f80dc64483a54a488e507da140836261ae1b864577cc95d8e869919d5a25e73c
2023-09-25 15:25:49 +00:00
drh cc11526e22 Enable -Wall for the compiler on --enable-debug builds.
FossilOrigin-Name: 63a7b521390001939909d43d908af78f7df7cb92272d2b74f038da918c2bef05
2023-09-25 14:20:07 +00:00
drh 5933581cf0 As a temporary measure, try to translate the BLOB JSON format into the
legacy node format for processing.

FossilOrigin-Name: 14f2e95a9e531ef0d3fa7f1249f23c073a50c31b2109eefc2f258cada635ac2f
2023-09-25 13:23:29 +00:00
dan 873849b81e Fix a bad interaction between LEFT JOIN, bloom filters and partial indexes.
FossilOrigin-Name: 5775f92b31f0539c66d37e50689bec17a82c0e857a781331c03e34bc18157087
2023-09-25 11:12:39 +00:00
larrybr 1da3215d65 Cause sqlite3_exper_new() to replicate UDFs and custom collations early enough to appear in virtual column expressions during schema copy. [forum:/forumpost/e030aa4b3a|forum post e030aa4b3a]
FossilOrigin-Name: 8fc2c45558a826a84e6f7e98564573da125a5e2d5c635a85bdeba2abe29fbfa1
2023-09-25 00:39:43 +00:00
larrybr 16096e6bd1 Make expert changes acceptable to C89.
FossilOrigin-Name: a4f03c22ec13d2f9769ac655ad77ccd906ab8c0ee32df37139bd998c234a7a0f
2023-09-24 22:51:56 +00:00
larrybr e5adfb1a5f Merge expert changes to accept UDFs and custom collations.
FossilOrigin-Name: 8ead1b5ed6693c0e3c30428033049c95a1fe636d1d1eff1734d0a896b5ff1f5a
2023-09-24 19:47:37 +00:00
larrybr 5918adb374 Add minimal exercise of expert acceptance of UDFs and custom collations.
FossilOrigin-Name: 0f6bbb79cbb52f5ddf38840898ab6789a4e108353918296df20b332117e6a148
2023-09-24 19:38:22 +00:00
larrybr 7ecef617ee Sync w/trunk prior to merge back.
FossilOrigin-Name: 2d290854cd9b6cef70b53c28745e1ccee5b5443378407bfc726e452e7e7a0a95
2023-09-24 19:28:07 +00:00
stephan a2ead23868 Resolve the JNI FTS5 test5() failure and remove some nearby dead code.
FossilOrigin-Name: 5655c51f103c6dfade799ffb5ca2eb21b4360ada598741f373c952ae732535aa
2023-09-24 19:13:31 +00:00
larrybr a25d478ea8 Give expert ability to deal with custom collations.
FossilOrigin-Name: ac1dc1b6ded0e43acb5b2b707110f767f7bdb5cecdb6b28a982db174ae233d37
2023-09-24 19:00:00 +00:00
dan af7cce98c5 Add extra tests for changes on this branch.
FossilOrigin-Name: cb45f5b070cfdbb4358e8e909b0b34d658b5925db06e282b38046298a159c545
2023-09-23 21:31:40 +00:00
dan 7a9bbfe1fa Avoid an error when parsing a schema that contains indexes with WHERE clauses containing unknown collation sequences.
FossilOrigin-Name: d5dd39e37ab994ce5d75688a40c6494c17807431529e85eb2b11d58f367fe232
2023-09-23 19:25:03 +00:00
dan c18c7523a4 Fix a failing assert() caused by changes on this branch.
FossilOrigin-Name: d0e21f20bd8643d21c4b88fb75b83939d68e9b7118f7289a164656f11d2ee555
2023-09-23 18:52:22 +00:00
drh 9f811a35b8 Fix a harmless compiler warning.
FossilOrigin-Name: 7c4210253b660b385d14de3ae7ab30f038036308e1164ec17b40e2805b9b2235
2023-09-23 18:49:30 +00:00
dan 2d3c5385bf Add extra tests for the fts5 xPhraseQuery API.
FossilOrigin-Name: b5edea16317ccc896394011f45442af44d5ea1041b6abfe152c90412c78bfc5b
2023-09-23 15:34:02 +00:00
stephan 6c248201b1 Resolve the JNI FTS5 test5() crash, caused by two unrelated typos in JNI callback signature strings. That test now fails with an expected-vs-got mismatch but no longer dies in the native code.
FossilOrigin-Name: 0233f310bf6bbb84f76b25205af160951e0bf6d8b2f1bca4537384af4fe47e58
2023-09-23 06:50:19 +00:00
stephan 8a67a46340 Remove an invalid optimization from JNI code and add a missing check check for Java-side exception.
FossilOrigin-Name: b8151f879718f7e30520f7e5ae04bd6724ae98af067be097b1a00df2175a038a
2023-09-23 06:37:18 +00:00
dan bd42642431 In partial index scans, if the WHERE clause implies a constant value for a table column, replace occurences of that table column with the constant. This increases the likelihood of the partial index being a covering index.
FossilOrigin-Name: 66ed7abdfa228abde2052e3988589371f0e49b11582b1b4a83255d2df3a0aefa
2023-09-22 20:21:27 +00:00
drh e367e453e1 Fix minor parse-to-BLOB bugs.
FossilOrigin-Name: 8b53b2e6600c324ff7864840d98a3f03896b9792fcb60b70cc1f6227b3bd4ca1
2023-09-22 16:20:48 +00:00
drh c1c1d4d4a6 Add the ability to render a binary BLOB back into valid canonical JSON.
FossilOrigin-Name: 0b70cb77a4c8e3f17932f1ecca3942e0b0b03de637fb9656a130fe045f7ef826
2023-09-22 14:33:39 +00:00
larrybr 163af02aca Give expert ability to deal with UDFs.
FossilOrigin-Name: 3406b05b4f57901f64f9b5fc83fe0250b827ae7f342c2026818dab0840aafd23
2023-09-22 14:20:45 +00:00
drh 90189be7ce Begin adding code to render binary JSON back into text. Very incomplete.
This is an incremental check-in.

FossilOrigin-Name: b817dd865ed60fc4da0b662a9edec0fceb8921b02ce98133bdd565988939fd0f
2023-09-22 12:16:56 +00:00
drh 8c55945220 Improvements to the JSON binary BLOB format design.
FossilOrigin-Name: 2c89ae5d02f6a40ef869e2a162e2c72871df60572b27959fd1d7171f495ce881
2023-09-22 11:20:35 +00:00
drh 769a8dea81 Reorganize the code to put the new JSONB routines together, for easier editing.
FossilOrigin-Name: dc23e783d4147d363856abe109586fc79a5b535b492beee0cf7a0234c0210667
2023-09-21 18:16:35 +00:00
drh 6831cad03b Initial development code for an experimental binary BLOB encoding for JSON.
FossilOrigin-Name: 8131b3c272f47db2618886046a9713285ce120cb87d721484ee7444273290681
2023-09-21 17:51:39 +00:00
dan 8aaf63c6ac Add extra tests for java Fts5ExtensionApi API.
FossilOrigin-Name: f9d62b853ce8bfbfdc9f137e984e7a1b51d70e88c38b136b4fad1e8ae6ee8913
2023-09-18 20:42:06 +00:00
dan 876f09b560 Extra tests for java Fts5ExtensionApi interface.
FossilOrigin-Name: c923893f3604b278277de1bb919ef713bf7a4296b7ff71451cfe19bc2ff03190
2023-09-18 18:12:03 +00:00
drh 38119058aa Remove a NEVER() in btree.c that could actually be reached, with the approriate
database corruption.

FossilOrigin-Name: d1fd00e0875f881cacdc581a75215ec1872a81969ac07b8ebe3aaa707a9a3640
2023-09-17 16:36:22 +00:00
dan f99a27fab9 Fix resolution of "rowid" and similar identifiers in queries that use nested joins.
FossilOrigin-Name: 37ec43d92bde13efc71fa57ff5e59c4a95b9054c298f844aefeb06d4a91ad0d4
2023-09-16 18:18:57 +00:00
dan 2599705105 Enable "OR IGNORE" support via SQLITE_VTAB_CONSTRAINT_SUPPORT for internal-content (not contentless or external-content) fts5 tables.
FossilOrigin-Name: c362bde4f4b8489947f080154d7fddcfd6e8e21d742a483c496fb7fbe59969d2
2023-09-16 17:11:44 +00:00
drh a91fe45339 Add a NEVER() to an unreachable branch.
FossilOrigin-Name: 6b6eb38979d68c06e382620c8813d6b67a3de02c4a7a029c84f924b9a2e380c6
2023-09-16 16:39:27 +00:00
drh 5bd96b2446 Fix minor code indentation inconsistencies. No function changes.
FossilOrigin-Name: 05107a0ae1456b746d3119af68d39532fad23a7eef88c09a9ef46ab7f8da4b9d
2023-09-16 16:23:20 +00:00
drh 5ddf0cbe25 Fix a harmless compiler warning.
FossilOrigin-Name: 9e9ff4ca3a205d0b599634782a5ad389fbe4823663550f48c54ca35dbc660dc7
2023-09-16 16:12:10 +00:00
dan 2e8edc1d45 Further tests and assert() statements for the change on this branch.
FossilOrigin-Name: 1c202d540ac362bfc747a9f8472e83c9d7614e38467f8b48787a669fb34664ba
2023-09-16 14:42:18 +00:00
dan 63702bccaa Fix resolution of unqualified "rowid" identifiers in queries with nested joins.
FossilOrigin-Name: bbcbd3d537d6790373d97f59386b8ce7fa2177db572af0f1babe058a76e25cc3
2023-09-15 20:57:05 +00:00
drh 124a6aa7d5 Simplifications and performance optimizations for the RTree extension.
FossilOrigin-Name: 04a333f5faf6b90592f1f69889ac6c28949955e186f39037cd639480b06feae8
2023-09-15 20:28:27 +00:00
drh ef2056e533 Drop support for the view-scan optimization (check-in [609fbb94b8f01d67]) as
it was causing multiple performance regressions.  In its place, reduce the
estimated row count for DISTINCT subsqueries by a factor of 8.

FossilOrigin-Name: f911f1c4977fbcae041243955cf2b98d8cc8baa337885a69be0f2b9bd2efa6f3
2023-09-15 19:51:18 +00:00
drh 58c10d9469 Minor simplification to the DISTINCT output row count change.
FossilOrigin-Name: 0738386d7c505a98768ddaff7ff1fc76f115882d92d6f84bf136b4aa2caad8b8
2023-09-15 19:27:36 +00:00
drh c09701db9b Do not reduce subquery output row count estimates due to DISTINCT until
after the decision of whether or not to use an index for ORDER BY has been
made.

FossilOrigin-Name: 27390051e86ad86fb35219329d359be9e83073f59782631af7fc519225e10565
2023-09-15 19:00:47 +00:00
dan 81b70d97eb Allow expressions like "<tbl>.rowid" to refer to implicit rowid columns of tables in nested FROM clauses.
FossilOrigin-Name: 59a1bbc69f5dbb33418fa4b383393fb13a46bc1e531577da8ad54ae2fad5a10e
2023-09-15 18:36:51 +00:00
drh 20220d4682 Experimental: Assume that a DISTINCT clause on a subquery reduces the number
of rows returned by that subquery by a factor of 8.

FossilOrigin-Name: 5a940e47d501f23347eaf084f4820e064e93665d19c8938c80715b71006ede8b
2023-09-15 16:15:33 +00:00
drh 658096e124 The view-scan optimization was added to enhance the performance of one specific
query, but it causes performance regressions on a host of others.  Disable it
(at least temporarily) in order to try to find an alternative way of fixing the
one specific query that it was created to fix - an alternative way that does
not cause performance problems for other unrelated queries.

FossilOrigin-Name: d9625a9eafe88859c3255849ee750cc6322d5a13ddad45107f9b2085c20eff50
2023-09-15 14:06:44 +00:00
drh 581b229362 Do not try to convert a double into an unsigned 64-bit integer, as that does
not work on all platforms.  A double can only be converted into a signed
64-bit integer.

FossilOrigin-Name: ce339046416e9ddeffe77f71a8bfb8c5b302160c3207f919aebd62902618ed59
2023-09-15 10:24:29 +00:00
dan c9823e3960 Tests for java versions of Fts5ExtensionApi.xSetAuxdata() and xGetAuxdata().
FossilOrigin-Name: 858fa236f8b673eb03cf2242e8202cd33ae1f13a43b59bdbb2eecdb21d42ef55
2023-09-14 20:56:42 +00:00
stephan a0f878959f JNI: move XTokenizeCallback interface out of the Fts5 object - that level of indirection is a holdover from when those classes were in the core package.
FossilOrigin-Name: 385ec08570451c0be212677d1b681c13d225dc690965effa2293e64cb3d56023
2023-09-14 20:02:49 +00:00
dan 93717c2306 Add a few more tests for Fts5ExtensionApi methods.
FossilOrigin-Name: 4ce2a7e8ed0f2859efc1038fd0812bc2b667be1479f45c081deccc2cf188c807
2023-09-14 20:01:18 +00:00
dan 9ba2d13ed3 Add a test for the java version of Fts5ExtensionApi.xRowid().
FossilOrigin-Name: 227facf879d2ade348cdb51f5b50ba2f95b8621acc0cf7a5fed4a310b4c76baa
2023-09-14 17:42:34 +00:00
drh e744f66008 According to [forum:/forumpost/9f6db917e1|forum post 9f6db917e1], older
MSVC compilers are unable to convert a double directly into an unsigned
long long int, but must first go through a signed long long int.  Work
around this by restricting the range of doubles that are converted into
long long integers so that only the lower 63 bits are used.

FossilOrigin-Name: 0ab05871ba36e2a0eddfbc700c1d39891c54c453727a4effcdac3327efec163b
2023-09-14 16:45:46 +00:00
drh e68decd54c Document the use of OPFLAG_ISNOOP as the P2 argument on OP_Delete. No
functional code changes.

FossilOrigin-Name: d88f41b2cf2b721170d2428a50b717976091389d9a07d0ffa15c3323b0eaad37
2023-09-14 16:02:56 +00:00
dan 82cdf4987d Avoid a segfault in test program showdb.c that could occur when analyzing a corrupt db.
FossilOrigin-Name: 6bf3e90687d48243544cc07baa01cb1b25ee0b77c59437f96c8da8e5d8b8095e
2023-09-14 14:05:51 +00:00
drh 60e53c57cf Improve the sqlite3IntFloatCompare() routine so that it is testable.
FossilOrigin-Name: bb221381fa38db5be1222d89c567a9bca0e6c441e26a8c797bd326b263556afb
2023-09-14 13:10:17 +00:00
drh dc03af67a4 Do not set the global bUseLongDouble flag based purely on sizeof(long double)
being more than 8.  That is a necessary but not sufficient condition to use
the long double routines.  Instead, do a run-time test to verify that long
double really does provide extra precision.

FossilOrigin-Name: aa999d490b743f455f2fb6c41009098330380f0bc2e9b1c28fbc82141a9138ce
2023-09-14 10:46:50 +00:00
drh 647e3ed2ba Reduce the number of calls to sqlite3_mprintf() made by RTree.
FossilOrigin-Name: f158b7d4917e0951fbb86a6f438abcb618d8602566fa54bf04c05a37f3a73513
2023-09-14 01:46:57 +00:00
stephan 595c3ecc94 JNI: enable the fts5-customization bits by default but elide them from the docs until it's clearer where they're headed. Unrelated minor doc tweaks.
FossilOrigin-Name: 21adb1aef24398fba918a5e1479ce2b6fd7e86d87eb9f2bea86a0eb43e8809c2
2023-09-13 21:32:20 +00:00
drh 23164c984e Improved comments on the hasHighPrecisionDouble() routine. No changes to
the underlying code.

FossilOrigin-Name: 810c635ce063d873e969bf83339c654f6008e84ce8a61f0ffc61806e98d13dde
2023-09-13 20:35:04 +00:00
drh 5b5d4492f2 Determine at start time whether or not the underlying hardware supports
high-precision long double computations.

FossilOrigin-Name: 9a854b919667e0e679a259542b2ee444ee416dbd73ecd9458f6ced35d9d3f264
2023-09-13 20:06:46 +00:00
drh 7de8ae22f7 Omit the Reinsert algorithm from RTree. This causes most benchmarks to run
faster, at the expense of having a slightly less dense and hence larger index
(example: 33 entries/node versus 34 entries/node).

FossilOrigin-Name: b3049a1d3dbdd63c471499c2f6b417655defe9ad90228e7cc722f5be877aae01
2023-09-13 17:30:12 +00:00
stephan 1eca330a08 Doc corrections for the previous checkin.
FossilOrigin-Name: 9ea0a9f39d03707d251af5af08d1ba94958704ba74019d7e8823ccd21936023c
2023-09-13 17:16:25 +00:00
stephan 181063d477 Add JNI sqlite3_prepare_multi(), based on feedback.
FossilOrigin-Name: fa1c1534724b03debc83ae35c2fadab83faf4b4e62b91981fed103888de41396
2023-09-13 17:11:32 +00:00
drh f9967cb302 Performance optimization to cellContains() in RTREE.
FossilOrigin-Name: 43cde22bf3f36687df231eddf642581d1d4f4102ad8568d31f5b2ff0302ca800
2023-09-13 14:07:07 +00:00
drh cdae00b3e8 In the ChooseLeaf algorithm of RTREE, do an initial pass through the cells
of a node looking for solutions that involve no cell enlargement.  Only look
for the minimum cell enlargement if the enlargement is non-zero for all cells.
This results in a performance improvement by reducing the number of calls
to cellUnion().

FossilOrigin-Name: 59f0e239d19393190911ea3bd89b5a24be0d85d64ccf51906efc9537ad5d7298
2023-09-13 13:12:08 +00:00
drh 386e359044 Reduce the number of calls to cellArea() in the ChooseLeaf() algorithm of
rtree.

FossilOrigin-Name: c633fe6be0edc07a0aa1ac819d58b6c629b7c95b47b63516ae1b0a937f6a2b89
2023-09-13 12:06:02 +00:00
drh f4ae37ca44 Remove unused variable from the ChooseLeaf algorithm in r-tree.
FossilOrigin-Name: fb2e4a8067edc29ba64973820f265b21dfee0d32f4d675d32100d5eef93c2830
2023-09-13 11:36:44 +00:00
dan fef1c11f92 Replace an if() condition in fts5 that is always true with an assert().
FossilOrigin-Name: 2170312c8d7f076cbb8319227de3fac981432dae186bc1928cd217e41119b580
2023-09-13 11:24:58 +00:00
larrybr 111815609c CLI .import to accept EOF in lieu of record terminator on last field of CSV (with multiple field records), per RFC 4180. [forum:5b21c25bdfa|forum post 5b21c25bdfa]
FossilOrigin-Name: 6f7842f577a28df1f809cd4bae9e8eafa26f2b54a25a1362ebbdebf5026be57c
2023-09-12 23:21:39 +00:00
dan 12439f9c16 Fix a use-after-free error in fts5 that could occur when querying the "rank" column immediately after another connection changes its definition. [forum:a2dd636330|forum post a2dd636330].
FossilOrigin-Name: cb54c2da52d3175844127ead41cfed18793e91ea52e02af2d57c1d17b4312437
2023-09-12 18:36:46 +00:00
stephan 0148013416 Clarify some metrics output in Tester1.java.
FossilOrigin-Name: 4ae7199a82f0b6e0f5575e39d8bd454e527ab949f0c59b773a44a078c65c5676
2023-09-12 17:49:25 +00:00
stephan 18e143d1d5 Re-add the fts5 decls to sqlite3-jni.h, removed in a rebuild without fts5 enabled. Need a better solution for this problem.
FossilOrigin-Name: 6f269790a09350f49103bb0925e2a78a1f9b7a8aad84408eabbb8d64ef79d551
2023-09-12 17:06:51 +00:00
stephan 09d72e4a6e Rename JNI sqlite3_errmsg() to sqlite3_errmsg16() for overall constency with the text()/text16() family of functions.
FossilOrigin-Name: c4a8d8a5711bd2abeaf1c31046784d1bccff4b98e96522250866d1ee8ec6a9fe
2023-09-12 17:03:40 +00:00
stephan 1110f8ffcd JNI readme fix.
FossilOrigin-Name: 7dde000efcf170c33701947d1aab71bb45b2b0ae8646ee41bb2b2adbee56d1e6
2023-09-12 16:33:57 +00:00
drh 50925d23ab 64-bit builds on Windows default to using UTF-8 output (as if the -utf8
command-line option had been specified.)  32-bit builds continue to use
Windows code pages.  There is also a new -no-utf8 command-line option to
force the use of legacy code pages.

FossilOrigin-Name: d8c7560e8822f336e8c4b300eb1f2e55203fdf9e756d2e8af719563235f79ebc
2023-09-12 14:36:07 +00:00
drh 09e96ea731 Show the "(utf8)" tag on the CLI banner in Windows if in UTF8 mode.
FossilOrigin-Name: 822699a4d9756ec68f17d6e06da7f994730bcf5c7d69ca15e9131721c4ff3b19
2023-09-12 14:09:45 +00:00
drh ceeb9eeaac Add the -no-utf8 option to the Windows CLI to cause UTF-8 mode to be omitted.
FossilOrigin-Name: 4cb799c690986b8bd38d07461998824fa53418f7fe31f59d0bf38cae328d9b89
2023-09-11 20:09:05 +00:00
drh 17ca53da26 Make the -utf8 option the default behavior in the CLI on Windows for 64-bit
builds.

FossilOrigin-Name: 8b3c306def81546c5987f0f245d2bf3960218254c2833e9c27cd242e9118db62
2023-09-11 20:02:38 +00:00
drh 833ed7ee64 Refuse to build amalgamation-tarball and snapshot-tarball if the TEA version
number disagrees with ./VERSION.

FossilOrigin-Name: 3308fdda4b81c110ba4a66d0b325e7653c2f8155e7864aeb78991ed1da061836
2023-09-11 15:27:27 +00:00
stephan 5c3a25b450 Merge in JNI's significant divergence since it adopted sqlite3_client_data().
FossilOrigin-Name: e8e07dbbeaa7799eb0b90726e18e994aaf8c8d316ab4bdb06da732bc51cf0919
2023-09-11 15:20:15 +00:00
drh a9d788f08f Add support for the sqlite3_get_clientdata() and sqlite3_set_clientdata()
interfaces, to better support JNI.

FossilOrigin-Name: 9806c0dd2802d68b67c25c4f3347ed532f9a98b051e775d34e9182dd2f099891
2023-09-11 14:55:05 +00:00
stephan 7fa8d65539 Roll back [84e38341aeab] because a direct pointer comparison is ill-advised when the client-data string is dynamically allocated.
FossilOrigin-Name: 7b884832b71c23e62ba3c0d53f7c89199734c351f909d84ac19ac18c7ddccbd8
2023-09-10 10:56:28 +00:00
dan ef2e43304e Remove out-of-date comment regarding use of Parse.pConstExpr.
FossilOrigin-Name: 71548f72ad578db3d6ee409956386ace23ab2258d37cdc29ef2fb126e48f1ee3
2023-09-09 17:53:55 +00:00
stephan 8f825a7a1b In the JNI build, emit a reminder to not check in the javac-generated sqlite3-jni.h when FTS5 is disabled because changes in that feature flag result in unnecessary and large diffs in checked-in generated code.
FossilOrigin-Name: caceb52fb58f924e7cbbebac05e7fd7822b7f6f56a3f834392d2be801bf04633
2023-09-09 11:20:35 +00:00
stephan 8ea10ea520 Disable FTS5 in the JNI build by default until it can be thoroughly tested. This is mainly so that the public-facing javadoc does not include it.
FossilOrigin-Name: 6f25f02b70f561864264a71a85cb5efa05550743371fd655804033ea08d00f33
2023-09-09 07:52:05 +00:00
dan 1b2d170a8f Add extra tests for [4b60a1c3].
FossilOrigin-Name: 6357491bd1b97f7abddf1a83860f3763cd870033bddc74bd9994a6440fe8c762
2023-09-08 19:18:02 +00:00
dan 410acf6944 Fix a problem with fts5 secure-delete mode causing fts5 to corrupt its records.
FossilOrigin-Name: 4b60a1c358c966f161601e749ccb0a7c5f4126e20350f4d6c83b6f35841fbe0a
2023-09-08 18:34:42 +00:00
dan 9d086cc562 Fix a sanitizer runtime error in wal2 code (that (1 << 31) cannot be represented as an int).
FossilOrigin-Name: 79a920c036c91bbae737e4f4c5310052d26407fe7d23bcf95a2aa79ea3773a0f
2023-09-08 16:49:45 +00:00
dan 2f3d5b1aaf Fix an apparently harmless buffer-overrun that could occur on this branch when committing multi-file transactions in wal2 mode.
FossilOrigin-Name: ffe043ba061b9ed59e770fb3f0811e1900ecb42b7c54dea0c47c9610e578f875
2023-09-08 16:07:05 +00:00
dan 1fd73f2693 Fix an apparently harmless buffer-overrun that could occur on this branch when committing multi-file transactions in wal2 mode.
FossilOrigin-Name: dde9fcbde16bf71c4ab708e2db769b9cefacd0d7bce8223a46bf1264ae7ddecd
2023-09-08 16:05:09 +00:00
dan fb77b3ed3f Add -DSQLITE_OMIT_LOOKASIDE=1 to the "Sanitize" configuration in testrunner_data.tcl.
FossilOrigin-Name: a917ebf17b27951d1c6906e9068e816ef29861701f2e8f4f1a13acce5737f680
2023-09-07 17:54:12 +00:00
drh 8525e2d931 Enhance the ./configure script and its associated Makefile.in so that the
--with-linenoise=DIR argument cause the linenoise command-line editing
library located in directory DIR to be linked with the sqlite3 CLI.

FossilOrigin-Name: 6c7822b554c01f01e3e02a385b3370313022b16ba8d932c2ba5d3c3be61361fe
2023-09-07 16:53:30 +00:00
dan 09f1652f36 Fix a buffer overread in the sessions extension that could occur when processing a corrupt changeset.
FossilOrigin-Name: 0e4e7a05c4204b47a324d67e18e76d2a98e26b2723d19d5c655ec9fd2e41f4b7
2023-09-07 13:53:09 +00:00
drh 0a4c31d87b Fix a harmless compiler warning in the sqldiff.c utility.
FossilOrigin-Name: e6390a656713b855258277b65066d2a701cedd05981f1672c9acc2d6dc37d032
2023-09-07 13:48:42 +00:00
drh 6e2b63893c Add the "tool-zip" makefile target with the intention of using it to build
the "sqlite-tool-*.zip" deliverables.
deliverables.

FossilOrigin-Name: 06e4af492d669bb45c00b1b876cd82ec8813ac9ed580eb1dddd18147f7fbe7f6
2023-09-07 13:43:46 +00:00
drh dd69129f2a Fix a problem with the new xIntegrity method for virtual tables, and also fix
a bad assert() in FTS3 that was found by the new xIntegrity method.

FossilOrigin-Name: 52bbf44f2d9addc2b5f68b0fe33542470852310ce3a283e2c7ff4c52831d0ed1
2023-09-07 02:13:01 +00:00
drh 93b461ff00 Add documentation to sqlite3_get/set_clientdata() to make it clear that these
are security-sensitive interfaces that should not be exposed to potential
attackers.

FossilOrigin-Name: 2837061d66672a0a73c3fd2ff35ce0e1a66ee6d5365be25d4cc0ccba170c3665
2023-09-06 17:07:53 +00:00
drh d4bb0896d4 Add the xIntegrity method to the sqlite3_module object, thus enabling
PRAGMA integrity_check to operate on virtual tables.  Make use of this
new method in the FTS3/4, FTS5, and RTREE virtual tables.

FossilOrigin-Name: 2f08e7d233b6ddc28b61f0d17ae9b815a91c3c6e080862ce009dc0ad73c64cf5
2023-09-06 16:51:13 +00:00
drh 9e5d21092d Fix unreachable branches.
FossilOrigin-Name: f00fc16394427f25249dd58c25a9115fef1ae389e232d574b4e91db2e7f8c04d
2023-09-06 16:22:48 +00:00
drh d5ab4dd9e4 Change the xIntegrity virtual table method signature so that it returns
an integer error code and writes the error message into a parameter.

FossilOrigin-Name: f1d4024a8ca06cf954aaf1f612684d1a5d28492bde757695db3f22c50c649709
2023-09-06 14:00:01 +00:00
drh 961c2a9f36 Add the xIntegrity method to the sqlite3_module object. Implement this
method in RTREE, FTS3/4, and FTS5 so that "PRAGMA integrity_check" also
verifies the correctness of shadow tables associated with those virtual
tables.

FossilOrigin-Name: 17bede8cdefd968210dd8a5a2617acbe12ba2c99fdd5e88c5def8665e7bec2d7
2023-09-06 12:52:00 +00:00
stephan 7c1588b3ad Move Java-side FTS5 bits into the fts5 subpackage to (A) make it easy to optionally bundle it (or not) and (B) set a precedent for puting extension APIs in their own package.
FossilOrigin-Name: 4c773e1f331cd5854d4c6e8825197933c0bb6827b7437e56ac0ad650d1ddf55e
2023-09-06 07:39:25 +00:00
stephan f31b9fcf86 Java FTS5 API renaming, for consistency.
FossilOrigin-Name: 5e4cc9f386364858c6457c40f13f01e1826a59086aa7f0fa10244ea846f5acfd
2023-09-06 06:54:25 +00:00
drh 5a05a68315 Fix a few SQLITE_MISUSE returns so that they call sqlite3MisuseError().
FossilOrigin-Name: 93f74490faf8cc07e107afdab6737c6e5141ae1f01a05142bfcede2dd1b2ba4e
2023-09-05 15:03:23 +00:00
dan dd24b1c8a1 Add tests for, and source code comments to, fts5. No changes to code.
FossilOrigin-Name: b12afff4efe4de84388d102060e8c312abd2f9eca8270b3c0f01ac6d1f2e329a
2023-09-04 16:48:31 +00:00
drh 25a6e6ec8b Do not make assumptions about the byteorder of PowerPC processors.
FossilOrigin-Name: 4a2498fed4c5436fbcd4179db85e2741fdab37d42b0eebf12f41ec4573ce2c61
2023-09-04 12:50:17 +00:00
stephan 0c072b698b In JNI routines which set an OutputPointer.value, always do so, even on error, for consistency.
FossilOrigin-Name: d0df62f0af19a69f7ceb3ad1dceda0d8cc2ca5016a6e2a3a1e36356424c3ca49
2023-09-04 06:15:37 +00:00
stephan 3aeb58c90c Remove several unnececessary functions.
FossilOrigin-Name: 84433c72b0b08e7906ef87758caffe98f95df812945a7aea675bbc20791e33c0
2023-09-04 05:58:13 +00:00
stephan 4eaf5883b3 Move an internal utility class out of the public Java API.
FossilOrigin-Name: 8e1ae3197a62796565b7260b230057bdb3f136edd00a83f8a63532d44d386243
2023-09-04 05:34:30 +00:00
stephan 411d9e7b2e Expose sqlite3_bind_parameter_name() to JNI. Extend the definition of null for the @NotNull annotation, to consider closed/finalized Java-side handles wrapping C-side resources as null for its purposes.
FossilOrigin-Name: fc34e2bfa4a2255f019a2514367e58596837289de49f7fab87279c8b40c7f119
2023-09-04 04:23:31 +00:00
stephan 45cf8ad02a In mksqlite3c.tcl, pass EXTRA_SRC files through verbatim instead of applying the post-processing needed for sqlite's own source files.
FossilOrigin-Name: 44f3d7d054055fab4cca0ba753e75e237ca159ae868ffd1d13ed45a874a772f2
2023-09-03 15:01:11 +00:00
stephan 14412e9526 Add the missing sqlite3_blob_read() to [9a9945c405cf]. Elaborate on the definition of the @NotNull annotation.
FossilOrigin-Name: 85952b4a52a09e9da35754c1a1ec70b1bf277296011a2d37ac536b33437223d7
2023-09-03 14:41:13 +00:00
stephan 805e8c96b9 Expose the sqlite3_blob_...() APIs to JNI.
FossilOrigin-Name: 9a9945c405cfe9b6bf399655f498457da66b06c8b92014510ab9fb83fbb1c7d4
2023-09-03 14:13:29 +00:00
stephan 7dfacc9f96 Expose sqlite3_system_errno() to JNI.
FossilOrigin-Name: bac54da29c19ef5e0a76599b5d10b377ecd7b391a651af99526529df16a83c7d
2023-09-03 12:46:09 +00:00
stephan 286f30f85c Expose sqlite3_stmt_status() to JNI.
FossilOrigin-Name: d266acc23ecb7e76c8c68c6e89a76e6f3054f33f0262e60e06b258db5a5e2ccd
2023-09-03 12:32:09 +00:00
stephan 1943356268 Expose sqlite3_randomness() to JNI.
FossilOrigin-Name: de9692242132b8f2c92ef4acb08dd3063327b18666cbb17c4f1153dee9146eaf
2023-09-03 12:17:34 +00:00
stephan 83a8b6dd82 Expose sqlite3_get/set_auxdata() to JNI.
FossilOrigin-Name: 0de3271717e0298070097d7ea0ecb996d2e95cf65384e494515a554d0a1dffed
2023-09-03 11:58:33 +00:00
stephan e30436b605 Expose sqlite3_get_autocommit() to JNI.
FossilOrigin-Name: 67391c6588c71af411767d1e4b63c5041a8a9e669102da318e3482ccfc9d9bb9
2023-09-03 11:16:48 +00:00
stephan ce9902f8c2 Remove some unnecessary code.
FossilOrigin-Name: c7f9bbbbf0252c9749450e3ec994bf5ac945bd62428657ddff469291a0fd4b0d
2023-09-03 11:08:31 +00:00
stephan 54021c3bd0 Add a couple more sqlite3_backup tests.
FossilOrigin-Name: 5c2cf89d2c3ae5fc31b5e311bc2f2c1fef0037bdb0dff48c0a2a6bb6610e0215
2023-09-03 10:36:21 +00:00
stephan 24a689cf01 Export the sqlite3_backup_...() APIs to JNI.
FossilOrigin-Name: 865a3df5d550f6f4edbf95854750706e4e747d8f1825ae73d300b8e31934959b
2023-09-03 10:23:03 +00:00
stephan 2966b85df3 Export the sqlite3_keyword_...() family of functions to JNI.
FossilOrigin-Name: bd554db39159d8a538ce913d55285e3c417db8823c717a3e34bfa0678de42be7
2023-09-03 09:28:45 +00:00
mistachkin 2065a9b730 Also add support for EXTRA_SRC in the Makefile for MSVC.
FossilOrigin-Name: bfc18ef4323cd73fc63cb93f18bdd9c3add41bea83c1cbc1c20de5c1e4296a60
2023-09-02 21:55:10 +00:00
drh d3c5471a9a Performance optimization to the OP_MakeRecord opcode.
FossilOrigin-Name: 2aef9af3dd493a82ba090ccb8ab1b8974349392ec3f1c35db7a3dbbec41b5c51
2023-09-02 19:35:15 +00:00
stephan de4d1c357f Add sqlite3_config(SQLITE_CONFIG_LOG) support to Java and correct clearing of SQLITE_CONFIG_SQLLOG. Correct missing names in propagated UDF exception messages.
FossilOrigin-Name: 652e2e82100ac3d8238d15ef6d43de4162fd02d695db4936ad5706fe4f2d7dc3
2023-09-02 14:36:41 +00:00
stephan 7d31de600b Expose sqlite3_complete() to JNI. Slightly widen the definition of "canonical" for the purposes of the @Canonical annotation. Add a few new tests.
FossilOrigin-Name: a59fe0cd9552128ca4186f4e66f0d188f2a57c7647f43828e42cb95801f350ce
2023-09-02 12:53:25 +00:00
stephan 216433636d Duplicate code consolidation.
FossilOrigin-Name: a1872ae959cca11d5d8bc224aaeb0b53e772d999df9924b2e9ea84ceedb99148
2023-09-02 11:26:36 +00:00
stephan 70b58fc119 Remove a superfluous level of indirection in the JNI internals.
FossilOrigin-Name: 8dca6f7660c15eacbda20da1c66c9ef1de36864f78750658226b1a7baf22b726
2023-09-02 10:18:10 +00:00
stephan 63f4a9c4db Further JNI cleanups.
FossilOrigin-Name: 30e38173c3ece0c9f8e7a9710f46cb5e8e8ef101c04531318a7adb070242f5dd
2023-09-02 08:51:14 +00:00
stephan 21d4f0cc7b Optimize sqlite3_get/set_clientdata() for the presumably common case of a static string pointer.
FossilOrigin-Name: 84e38341aeab4fd51690e8536d0101d00e3fb11cc50ebcff05ed219c98328872
2023-09-02 08:04:56 +00:00
stephan 87ac2b043a Add EXTRA_SRC build var. If set, it is treated like a list of files, each of which gets appended verbatim to sqlite3.c. The intent is to enable easily extending the amalgamation. Requested in/around [forum:3fcc655f0ac0efe8|forum post 3fcc655f0ac0efe8].
FossilOrigin-Name: 934b84339860fa84a6faa3507bdd3cc58199e2636a394fc93176c80da0ba6f1c
2023-09-02 05:37:58 +00:00
stephan 49a40ef6d2 Add a timer to the JS SQLTester app to warn if it appears that loading the module has failed, which is commonly caused by the test scripts not being compiled.
FossilOrigin-Name: f9c1f9cad1ad22f689a4afa13d59bf9969ddaed6624cbc16cb1bf5d1fd0c8a5b
2023-09-01 17:51:09 +00:00
stephan a5f425cc6e Correct a Tester1.java test which inadvertently created a spurious db file in the current dir. Unrelated minor cleanups.
FossilOrigin-Name: 016bbdadbf9f95cef185d2606c23e3b3c724586fcca470e11e6fbdbaefd75c79
2023-09-01 16:28:46 +00:00
stephan 4ea63e5609 Make the JNI-internal metrics opt-in rather than opt-out so client builds won't have them by default. Unrelated doc tweaks.
FossilOrigin-Name: aa72d25a22c80a7a376127266b8923f0661147c8bc483baa327230364851a255
2023-09-01 14:20:39 +00:00
stephan d086065644 Make JNI interfaces of sqlite3_column/value_text() match the C ones better. Internal JNI cleanups and simplifications.
FossilOrigin-Name: eb24f97253d9c8e7c728c2ac67a0824b10ca67362ac4f8abb94d5d3c54c58c5b
2023-09-01 13:27:59 +00:00
stephan dba0ceea60 Second half of [1c532e80].
FossilOrigin-Name: 8ca1d815a57be1fade59fb8ea5705c27b10294e4959c8a9c624f1623df6a5f63
2023-09-01 11:10:09 +00:00
stephan f551aa4e80 In the OPFS VFS' importDb() methods, overwrite the header bytes 18 and 19 with 1 instead of 0. Both seem to work, but 1 is correct.
FossilOrigin-Name: 1c532e807bf8466b67d9600ab9630a6736de77259e9a71ac435641715ee0e04b
2023-09-01 11:05:22 +00:00
stephan 7862e19468 Remove some newly-dead code.
FossilOrigin-Name: dc0fa76b395c5e352116dc33cc1b455e5e9c9f01c285af52ccba4d37e4453b87
2023-09-01 10:37:15 +00:00
stephan 03ca59afb9 Expose sqlite3_limit() to JNI and remove the all SQLITE_ macros from the Java interface which are not directly exposed to client-level C code (SQLITE_MAX_... and SQLITE_THREADSAFE_...).
FossilOrigin-Name: 5fe5b0a585dbfa06cfca1fea40aea5ea3ccfe8295f41306ad2d5b851ace27203
2023-09-01 10:32:31 +00:00
stephan 55171a75ef JNI build improvements. Rename a Java class. Get the jar target working again.
FossilOrigin-Name: d086b7844cace5c997261c97565aeef62aaeeef727ccc7e83f17c54d6217b779
2023-09-01 09:43:56 +00:00
stephan a4b47b034c Expose sqlite3_stmt_explain(), sqlite3_stmt_isexplain(), and sqlite3_stmt_readonly() to JNI.
FossilOrigin-Name: 208b786afe16eafaf0ce791f319a5e05f733a7b71ce1c542e1b83481b013ec38
2023-09-01 06:50:17 +00:00
stephan 0d3f0a9c11 Export sqlite3_txn_state() to JNI.
FossilOrigin-Name: d732f71d0a292dbb493f79f7c6ecd8a4effbfbc91453b1c54bdd9becf2d75bdb
2023-08-31 19:35:59 +00:00
stephan 63e7ed3b44 Minor JNI cleanups.
FossilOrigin-Name: 2e2bb841ef8b53266ec48d8c3408d6396f29f17922e858eac1b4f12f4adeb05e
2023-08-31 18:34:43 +00:00
dan 8628c10e27 Do not disable unused columns in a UNION ALL sub-query if any component of the sub-query is DISTINCT.
FossilOrigin-Name: c84d5602ac9bfb4f12c3cf62033af616e51913c26877d1a0761363a625295f53
2023-08-31 18:00:10 +00:00
stephan 062bafb851 Add convenience overloads of JNI sqlite3_table_column_metadata() to simplify usage.
FossilOrigin-Name: faf4e6d398f444d970be8b1364c984c67ccbc5f8a27f3aa93f05b8a8de7bec9b
2023-08-31 15:24:46 +00:00
stephan 0c2ba994d2 Export sqlite3_(db_)free_memory() and sqlite3_table_column_metadata() to JNI. Further internals renaming for consistency and legibility.
FossilOrigin-Name: 7c86aa3400ed591d38c1828f366f4b5de97954c2b301919d3f06d9c2d3d7d1f2
2023-08-31 14:57:01 +00:00
drh 3823208d5b Use mutexes to make sqlite3_set_clientdata() and sqlite3_get_clientdata()
threadsafe.

FossilOrigin-Name: 443ea20ddb0f3bf5d77ef59cd4678f0e32d7da328002bb44d6fc080a53a37e29
2023-08-30 18:51:26 +00:00
stephan 7fc4223411 Replace JNI's per-db-instance state tracking with the new sqlite3_set/get_clientdata().
FossilOrigin-Name: e7c11d34ee2eebdca4d9db1496bbb4152e4c62745c083ad5e0337733e8d1254e
2023-08-30 18:45:42 +00:00
drh a7ec1f9a1b Fix a bug in json_array_length() introduced in version 3.43.0 by
check-in [df099ad713011b67].  If the JSON input comes from json_remove(),
the removed array entries are still counted as part of the array length.

FossilOrigin-Name: 69a635958a4a14187e88dd297df8978a4805b1b0c7bff3ec29d5632c0f68d889
2023-08-30 18:19:40 +00:00
drh d25d938877 Remove another unreachable branch in the sqlite3_set_clientdata() logic.
FossilOrigin-Name: 57e31e6b10b2aa68b6039914d191819bae0aa60999a99d5ef23ca18128b72e0e
2023-08-30 17:41:55 +00:00
drh a5af4a6614 Remove an unreachable branch, and improve documentation of
sqlite3_set_clientdata().

FossilOrigin-Name: 565c68adee64b6c838f2fcb162485b6cfe9a976adb79de4370be98fcd4af2baf
2023-08-30 17:14:12 +00:00
drh 7bb5a6db40 Fix the AVG() and TOTAL() functions (after the SUM() fix of [77d3dcd283595c52])
so that they work with infinitites.  Fixes a bug introduced by check-in.
[c63e26e705f5e967].  Bug reported by
[forum:/forumpost/8960fb40cc|forum post 8960fb40cc].

FossilOrigin-Name: 6df6f17ccb404c648076ccff4200d0eb5437f0e3e82424bf3da5ea682b107bb4
2023-08-30 16:03:27 +00:00
stephan 85a05d895a Further tweaking in both SQLTesters.
FossilOrigin-Name: 46b0eb98e7a56cc188836c46f4289de6281f80e6bc6eb33e0aa9133a0f86f00c
2023-08-30 15:51:42 +00:00
drh adf54de51e Fix to the sqlite3_get_clientdata() interface.
FossilOrigin-Name: 937cd1762b36d48d9e7bafe42e99e524c9b5528d39f64521bce2f81d9c887db8
2023-08-30 15:42:41 +00:00
drh 10deb35995 New experimental API for attaching client data to a database connection.
FossilOrigin-Name: d542837fdb42ebe810fc99225860d2cc7e6dd829a635bde820a09beff6bcb481
2023-08-30 15:20:15 +00:00
stephan 8dd07389ac When a JS SQLTester script throws, report the exception details back to the UI regardless of whether it's fatal.
FossilOrigin-Name: 273d3b05f630d399d42914e95c416b107b4746bbef129cfba9d00fd921666261
2023-08-30 14:20:02 +00:00
stephan 24c32c2e39 Add a JS implementation of Java's SQLTester.
FossilOrigin-Name: b530792a514d95c4e8f93cf2170d9fc4de367055fa1704fc171551c946024fa9
2023-08-30 13:07:35 +00:00
stephan e621556724 Add a UI, of sorts, to the JS SQLTester.
FossilOrigin-Name: 249e82b9917ea47c56ee1cbd3345a977d335fd3fc0d67a1ef157813ef4571c7c
2023-08-30 11:54:43 +00:00
stephan ac5e1f82ce Add a mechanism with which to import external SQLTester scripts into the JS testing tool.
FossilOrigin-Name: bb08ba020ce1d86ca6aa92f43d5ae915f67d08fa73120e1f603d150e76166624
2023-08-30 00:22:54 +00:00
stephan 4f1387e9ab Add directives support to JS SQLTester comparable to the Java impl. This brings the two to feature parity.
FossilOrigin-Name: 7cef4a8300826adbdcb3b205e134a4272b12b4aa7dbee97731ac12282a4a9f06
2023-08-29 21:30:37 +00:00
stephan 267c44771f More fleshing out of JS SQLTester.
FossilOrigin-Name: 8c503dfb9fa15389613a819fcc1792e23d3c05f99a9f450f82eac5125298726f
2023-08-29 20:44:40 +00:00
drh 5267068d3d (no comment)
FossilOrigin-Name: 9e927eed2d9e379dd79ef82aa1e41102463c5654b8f4e343bb88bb0daf55c689
2023-08-29 20:19:42 +00:00
stephan aa15047796 JS SQLTestRunner can now run the Java impl's core-most sanity tests, missing only support for directives.
FossilOrigin-Name: 5e798369375ce1b0c9cdf831f835d931fbd562ff7b4db09a06d1bdca2ac1b975
2023-08-29 20:01:01 +00:00
drh a910f3f1c5 New conversion letters added to strftime(): %e, %k, %I, %l, %R, %P, %p, %T, %u.
FossilOrigin-Name: 058722b2d0b995195a8ce3effe9722ae1c18cb1e7a520b481030da0bd579fe41
2023-08-29 18:28:08 +00:00
stephan 69a55ca17d Get the JS SQLTester command handlers in place sans those which have to run SQL.
FossilOrigin-Name: d21b1217964a53f33b7ba3958b34aa8560dff8ede33e66f54aa0afbab7099ec3
2023-08-29 15:39:57 +00:00
drh 9d6acd960f Change the version number to 3.44.0 to begin the next development
cycle.  (Should have done this a few check-ins ago.)

FossilOrigin-Name: 253fe4a45decdcc4180c5aeccaf4cd0d1325a0f6df0913b0e6721ea3e26b4fe2
2023-08-29 15:38:54 +00:00
drh e1e67abc5c Add support for the CONCAT() and CONCAT_WS() SQL functions, modeled after
the PostgreSQL behavior.

FossilOrigin-Name: 0b434ca7aa19eff4ad134a8c6f88f6a7ccab88864faa55e93579a2462d8ac3bc
2023-08-29 15:24:41 +00:00
stephan 0fc20a32c0 Get the basic parsing pieces and command dispatching in place in the JS SQLTester.
FossilOrigin-Name: 8fcc2a553c1e26734902bbdee0c38183ee22b7b5c75f07405529bb79db34145a
2023-08-29 13:28:36 +00:00
stephan 524ddc940d Init bits of a port of Java's SQLTester to JS. Far from complete.
FossilOrigin-Name: 60eec5ceda80c64870713df8e9aeabeef933c007f2010792225a07d5ef36baef
2023-08-29 11:22:45 +00:00
dan b44b802378 Change a variable from "int" to "i64" to make it easier to prove that it cannot overflow.
FossilOrigin-Name: 00a8b3a263f3537588063ce42fad6e21fa343dad850b086d0929ed1617eb44fc
2023-08-29 10:50:11 +00:00
stephan d10ed826eb Remove some dead code. Add a skeleton SQLTester script for fts5.
FossilOrigin-Name: 6c83e31fa96f65b61377c0c801cc32b3c8ca27a0c8442f860364bec258c003cb
2023-08-29 00:10:31 +00:00
stephan 4e379078e7 Factor out a superfluous struct member.
FossilOrigin-Name: 76d3911c370b3dc02d119045003f87ad20a6efd9f7a44d4feb59b7c801ac8981
2023-08-28 23:18:19 +00:00
stephan 924c4545d3 Further minor internal JNI simplifications.
FossilOrigin-Name: 1808d12ee0d1f1e5ee49d48c699ca10c4f822989ac9b4ac08f2b861513ee5997
2023-08-28 22:52:04 +00:00
stephan fa23b4fc61 Improve threadability of the JNI collation-related bindings and add infrastructure for similar cases.
FossilOrigin-Name: f02dad66b965b9e3c504001e9603af8f74977f151bede9db369f88e86a4aeb00
2023-08-28 21:27:32 +00:00
stephan a08f737503 Correct JNI mapping of collations to be 1-db-to-many-collations.
FossilOrigin-Name: b927b0f5a68684b0a9799396d153bf1e2306351e8039c2bacb3d5b2056a0634f
2023-08-28 20:21:56 +00:00
dan 35c7f538e9 Updates to testrunner.tcl so that it runs fuzztest using multiple jobs.
FossilOrigin-Name: ceeabe9f8b31a30c65147fd270b92d43c7842247548cee9de165113991f6c2cf
2023-08-28 20:14:19 +00:00
dan ecdc5b3e6f Fix the "sdevtest" testrunner.tcl command so that it correctly enables asan on windows.
FossilOrigin-Name: 6463192369ff22eb15d4e34590ef55cd8a6f2501227835664816277806bf961b
2023-08-28 20:02:59 +00:00
dan 9b0e9f9089 Fixes for testrunner.tcl on windows.
FossilOrigin-Name: 26ae090889640bfcf02a6dd5aa58044edd4d0708a11d8e005149f1a3c8495bba
2023-08-28 18:15:14 +00:00
dan d7def73c97 Another fix for valgrind permutation. Remove the "copydir" table column - just add a [cp] or [copy] command to the test script where this functionality is required.
FossilOrigin-Name: 448b9a575e66960a0885864f88b99b6ea15655c77a42eef956c09c8dc70e539a
2023-08-28 16:28:00 +00:00
stephan ece2d7f2be Remove a pair of what are arguably unnecessary mutex locks (and often hit). More JNI-internal cleanups.
FossilOrigin-Name: ecf07a0144dc6402b1e0924b1775d99dc465b27aa86a2718cac60a9b4c974312
2023-08-28 16:22:31 +00:00
stephan 8d64c12a10 Update tool/mkctimec.tcl to account for [fe7365254d343e].
FossilOrigin-Name: 349aac7e8d513bc420e8948b84cf715e454443439fdcd5aff0f2b13815a9cbb3
2023-08-28 16:05:19 +00:00
drh 57b1800773 Add support for the -DSQLITE_EXTRA_AUTOEXT=name compile-time option.
[forum:/forumpost/00829394c74a670f|forum thread 00829394c74a670f].

FossilOrigin-Name: fe7365254d343ed03e11a4e9cad7f0e5d5182b9220c6fde6d30e434ebdaba2af
2023-08-28 15:58:00 +00:00
stephan 25e7d8af75 Duplicate code consolidation.
FossilOrigin-Name: 71f239747c7934310dedf9fc0cbf84fbeeed53808234067147335c12396849a1
2023-08-28 14:59:27 +00:00
stephan bdb149d063 Move a metrics counter so that it can be used to indirectly witness the race condition fix from [f5274e00f17d58e0] by recording varying final values for that metric across consecutive multi-threaded test runs.
FossilOrigin-Name: 54d2209d24547dbb05e7c12daa27211593c34de8005e8adba0989b8d219f5f3b
2023-08-28 13:18:46 +00:00
stephan 4e9b48cfc4 Resolve a JNI-side race condition. Removed a now-extraneous struct member. Internal API renaming for consistency.
FossilOrigin-Name: f5274e00f17d58e075f90ae5c1d4b38933da315e51592171fa35bcbd67b40b2a
2023-08-28 13:06:26 +00:00
drh bd953dfcad Fix an issue with infinity handling by the SUM() function that goes back
to the extended-precision SUM() enhancement of
[check-in c63e26e705f5e967].  Problem reported by
[forum:/forumpost/1c06ddcacc86032a|forum post 1c06ddcacc86032a].

FossilOrigin-Name: 77d3dcd283595c52f24c07fc59ba60c9133b71c440cf3f799cf48c907c6fae3e
2023-08-28 12:20:18 +00:00
stephan 0d066bc8a6 Resolve a condition which could cause an is-interrupted db to call its progress callback.
FossilOrigin-Name: a0d0f1aafc6086726131dff5e6628f2771c20db3122a53bdbb82945ab5d326d1
2023-08-28 12:06:38 +00:00
dan f98612b1c5 Fix the valgrind permutation on this branch.
FossilOrigin-Name: ccbf06760e8377f9209ba3fd4895a25fe1a5ea688c384bf357e765b6a2b4c2f7
2023-08-28 11:22:33 +00:00
stephan f0e9e59d8d Further JNI cleanups and javadoc additions.
FossilOrigin-Name: c393b1c9f5639c259406e587b3e08efdc1962203001823747681379ffc54fc10
2023-08-28 11:10:13 +00:00
stephan 2597ec6385 Lots of javadoc-related tweaks.
FossilOrigin-Name: cfe06f90e2c0231efded98767ef3cc646d3a7daa34d77b62b7c04b5aae9448fd
2023-08-28 07:28:36 +00:00
stephan 23dfa67c1e Correct a package renaming bug in the previous check-in and start applying the @Canonical annotation to functions to distinguish canonical API functions and Java-specific APIs.
FossilOrigin-Name: 8a016006805b08b72bfc4093c795d8cd8d7fe72f8ae234c175a8b7be3a841cbf
2023-08-28 05:48:34 +00:00
stephan b041f8ffdd Move the JNI annotations into the annotation subpackage.
FossilOrigin-Name: e4dedf90a92a069daef967dfe975469bf8ec7883c44c95e73345d4eded48e996
2023-08-28 05:39:05 +00:00
stephan 56d3b21317 Resolve a makefile bug which causes sqlite3-worker1-bundler-friendly.mjs to be built incorrectly. Reported in [forum:a874e435cf4690c1|forum post a874e435cf4690c1].
FossilOrigin-Name: 9d68c7ef103b9b51ed35fb91e62351eb6c62ad63ab03ac456ff9787cc83f10ab
2023-08-28 04:59:16 +00:00
stephan 5020ddc243 Reimplement JNI's sqlite3_value_text() as a native instead of a Java-side proxy. Unrelated minor simplifications.
FossilOrigin-Name: 401dd6ee283a4e7da2e50175a1b58e6a83afb0ea2d3b6cf80c7c621e17bcce98
2023-08-27 15:15:46 +00:00
stephan 0639c58836 Generic cleanups in Tester1.java.
FossilOrigin-Name: fbc1b6310b01f4f79439bb6ce59b2aaabd126da7201f36d4fc5d7ef95118a494
2023-08-27 14:47:45 +00:00
stephan 5575d6421b Remove JNI sqlite3_column_java_object(), as the protection rules of sqlite3_values makes it impossible to implement safely. Add JNI sqlite3_bind_java_object().
FossilOrigin-Name: 29bd4a23a4afd96b2cc06d2b91a4f30c0bbf2347af0b0d18f8d4cf8aafa63160
2023-08-27 13:43:45 +00:00
stephan 88bd53dfd0 Correct the signature mismatch between JNI sqlite3_column/value_text16() and add related tests.
FossilOrigin-Name: 77f6e70f17c0cb6d031f983c458c9ec2e88d92b4716397533a029af39da2d128
2023-08-27 11:28:57 +00:00
stephan 95f5d85d4a Make JNI sqlite3_trace_v2() thread-safe. Re-add a piece removed in [bae4d022aad9b] to work around a JVM crash which is unpredictably triggered by its substitute. Fix the THREADMODE=0 JNI build. Further internal API simplifications.
FossilOrigin-Name: 3f9f7a9cb08b0687ad206605a5109306762df9ae8bdeab2d8d60bf9373c9ad32
2023-08-27 10:40:00 +00:00
stephan 32a79760b5 Factor out an unnecessary struct member. JNI sqlite3_shutdown() now frees up the various object-recycling bins. Doc touchups.
FossilOrigin-Name: bae4d022aad9bbeb78cb027ecad799af87afe331e697add44ec22297c873141d
2023-08-27 09:12:50 +00:00
stephan 60aca33a8b Factor out a superfluous JNI class. Doc and code style cleanups.
FossilOrigin-Name: 0f37f27148dfa93ecc42381ad3455a9059285d1af2df027429044942dc4d861b
2023-08-27 08:10:59 +00:00
stephan 0f4bf3435a JNI code reorgs and simplify the failing-alloc interface a bit.
FossilOrigin-Name: deed5797de65a25896e991a441f0d05eb92662536296485920fb081e84ad5d32
2023-08-27 07:26:33 +00:00
stephan c7e7c88873 Apply the JNI OOM checks to memory returned by JDK APIs, as distinct from our APIs.
FossilOrigin-Name: 1ff78582bfd934e0c76464b5f23ed9bf09a3491b145e0ca34acb6e59c4f53995
2023-08-26 22:34:26 +00:00
stephan 0deaea218d Eliminate a superfluous JNI-internal middle-man class.
FossilOrigin-Name: daede0f801f59d6501a863c4688e4635b34171e98b56b8ab4432c779113f1997
2023-08-26 21:13:57 +00:00
dan 1f1169ad09 Updates to testrunner.tcl so that it runs "make fuzztest" using multiple jobs.
FossilOrigin-Name: 7596ea7074e0ac73312586ed3d28cdacf97f54b8af73f804cbc8066c94d4b4ef
2023-08-26 21:04:54 +00:00
stephan 506a8b52f3 Whether or not OOM is always fatal in JNI is now a compile-time option.
FossilOrigin-Name: 320a34c080d8bc1feae1578697923dfa7c4144b78de36f704c24cc4a4ce9d535
2023-08-26 21:02:50 +00:00
stephan 6428cd18d6 Correct a string length misuse in JNI sqlite3_result_error() in an OOM case. Unrelated minor JNI cleanups.
FossilOrigin-Name: 4252f56f3d8574b7b43306440726daf3b5f5500d5d9105784b2f82753e7c71dd
2023-08-26 19:34:49 +00:00
stephan 6a1ebbdb90 Remove a bunch of commented-out debug output.
FossilOrigin-Name: b49488481e2952294960bb0ee971f6eca126c19d68ef92152894aa28393e6865
2023-08-26 18:15:33 +00:00
stephan 220b6145f5 Re-order some out-of-order JNI APIs. Correct JNI sqlite3_open_v2() behavior in a particular OOM condition.
FossilOrigin-Name: 49d3be002ce5e594027f47a3ba448f0c21ec68b416b8df997497753f53e3ca52
2023-08-26 18:01:36 +00:00
stephan 2a6835fe0c Two significant typo fixes for [1a30919bfbb686].
FossilOrigin-Name: 6a6ee6dd54ec1a1dc7fb7d2f331951525d394a8362aef911f62878ac3961898f
2023-08-26 17:36:15 +00:00
stephan 3cb610c010 Bind sqlite3_db_handle() to JNI.
FossilOrigin-Name: 1a30919bfbb686464e8020e684120d22dd2375b910a17446b75452fda96a7449
2023-08-26 17:30:14 +00:00
stephan 2a0dc4895b Correct a potential duplicate xDestroy() being triggered for Java-side collations.
FossilOrigin-Name: 25331172f78544f7f23ad7821cbf065589f6d02706184d4c83fe3988452dac5d
2023-08-26 16:55:27 +00:00
stephan fec793dd8a Recycle per-UDF JNI state.
FossilOrigin-Name: cf406528eb86d8d0d55a468b2c4ec32a11a4947f45c4bbabdde8742ae199ce1f
2023-08-26 16:29:48 +00:00
stephan a7da592bd7 Code consolidation cleanups.
FossilOrigin-Name: d6b5ecd28740c2c5d21797fce9fe137c8a83f702f22901720cc6e8b1b42af001
2023-08-26 14:55:44 +00:00
stephan acfbe5284e Add -sqllog flag to the JNI 'tests' target because it hits APIs which are otherwise not tested.
FossilOrigin-Name: cc3153ed341f59262485c3541a8879c4e86520c8a10f4ce819344a88099e7d0e
2023-08-26 12:19:51 +00:00
stephan 3ff458d61e JNI internal cleanups and docs.
FossilOrigin-Name: b7b26bfb4f86e0b8aaabab258ccb0713737ffd4ecd3156d6a83a4f9a1d25edf6
2023-08-26 11:57:34 +00:00
stephan d9cf47e377 Remove the Java BusyHandler.xDestroy() method - it should not have had one. Eliminate the last of the potentially-significant MUTF-8 cases.
FossilOrigin-Name: c852f1ebbde273c3d28fe5aff0bf73cfc06b41dd371a94d7520536dc7a1dbcc1
2023-08-26 10:51:19 +00:00
stephan 6f8f587d2f Improve threading support for all types of JNI-side callback hooks, making them safe to invoke if another thread is busy replacing them.
FossilOrigin-Name: f2af7bbf493fe28d92fc9c77425f8bb9d48c02af9a5eabceb0365c705651e114
2023-08-26 10:20:38 +00:00
stephan 25033b6b3b Move the 3 Java SQLFunction subclasses from inner classes to package scope.
FossilOrigin-Name: 21fd47a68db9df1828f4cc4131d326a193b5751d56a40ae77ed0a78dc0621af1
2023-08-25 16:43:51 +00:00
stephan 9fed4aab77 Missed a straggler when renaming Java classes in [8ca528006533ac1]. Document threading limitations.
FossilOrigin-Name: 5786b95f5d09b4462aff0fdeac37992a2b64c47b004d18960f51e4e6a5796106
2023-08-25 16:26:16 +00:00
stephan a5f09eb29d Hopefully the last major API rename in the JNI bits: rename all lower_camel_case classes which do not have direct counterparts in the C API to UpperCamelCase, as that's more idiomatic. Java types which reflect C-style types are unaffected.
FossilOrigin-Name: 8ca528006533ac152e9b322e421cf5d339414c77332d9836a9ff429b793d0d2d
2023-08-25 14:17:36 +00:00
stephan ad818380cc More Java API renaming for consistency.
FossilOrigin-Name: 4e457778de189d8393e88f399dc521c8a5ad4ddd80a5fd69295bcb104fac5cbf
2023-08-25 13:23:45 +00:00
stephan a899186c86 Re-enable SQLITE_THREADSAFE=0 in the JNI build and add multitest target which builds and runs the tests in all 3 threading modes.
FossilOrigin-Name: 6b1c816f6aae9face07ed7ead969257cf915c21ff9cdad9a21b7cf0e7496e09e
2023-08-25 12:29:59 +00:00
stephan 0199669fa1 Code restructuring. Force SQLITE_THREADSAFE in JNI builds for the time being, as threadsafe==0 leads to as-yet-mysterious JNI-level reference errors.
FossilOrigin-Name: 5a099caa2c21bec647f0a521e7f5d0d1cc2f96d388d3d6c53d5ec80947f33e8d
2023-08-25 11:32:56 +00:00
dan 35f9b1719b Remove incorrect (but harmless) dependency on source code file "sessionfuzz-data1.db" from main.mk and Makefile.in.
FossilOrigin-Name: 17d56c0207f63614b34ef3594d06602ab7a6e85604f3589b30aa79316f1744ee
2023-08-25 11:06:26 +00:00
stephan 5ecb51c260 Only build in the JNI-side metrics tracking in SQLITE_DEBUG builds.
FossilOrigin-Name: 646e7fc3b5ba81c207f013c9a06781986138379f20e787320a811ba3ed5489dc
2023-08-25 04:27:17 +00:00
stephan aec1361a2b Javadoc and internal API cleanups.
FossilOrigin-Name: 10a43cba9bb2c23b5f31b31fcac81be34acec1940cca47d0b8e5a5ccbf1aa8eb
2023-08-25 04:02:33 +00:00
stephan 5276552083 Replace all of the JNI XyzHook/Handler classes with snake_cased ones which follow unified naming conventions to make them easier to apply.
FossilOrigin-Name: 76e62a381249b3b4262b22bdffe7fc2816c820115c9df266956ab8817b127aca
2023-08-25 02:57:30 +00:00
stephan 44b4b8260f javadoc additions.
FossilOrigin-Name: bedf33d403677d243a1505ce549166850ab55671700b143278db5feb84883ab3
2023-08-25 00:27:28 +00:00
stephan 3401736694 Re-frame the incongruous SQLite3Jni.uncacheThread() API as sqlite3_java_uncache_thread().
FossilOrigin-Name: 7232b033954fae40df3db43e489e0e5a703c03308f500a1ae36fd9d707632d7f
2023-08-24 22:28:44 +00:00
stephan 70dcc2822d Update some outdated JNI docs and account for a function renamed earlier this evening.
FossilOrigin-Name: 3f684ef5018116f4be46a07779451c8983ac87a5db182477f71ee7bf28287a04
2023-08-24 21:45:30 +00:00
stephan 0f0bf3ff9e Do not pre-allocate sqlite3_aggregate_context() for Java UDFs, as it unduly complicates UDF initialization.
FossilOrigin-Name: e8308f0c6ec2d8999c8a2502fb130cb3501ba326f23f71f2cd8d452debae79b5
2023-08-24 21:31:56 +00:00
dan 36018803d6 Add doc/testrunner.md, for documenting the testrunner.tcl script.
FossilOrigin-Name: 9c69a28401c7273823f2c2b291fd417febeb278afb9ce085a4b944505ca13d23
2023-08-24 19:08:50 +00:00
stephan eef599dbd4 More code legibility and style improvements in the JNI pieces. Start work on a javadoc build.
FossilOrigin-Name: 62b404d62fd62f4d220838b59c9f38a71afa2d4a8c3af0a5c9495fa7020972cf
2023-08-24 18:43:25 +00:00
stephan e336d92688 Doc, code style, and legibility cleanups.
FossilOrigin-Name: cf185bcd25629d882a030b8b87048179a120ab1f84aa1d68b279c499dbdf0dba
2023-08-24 17:25:05 +00:00
stephan cf8a93f804 Doc style fixes.
FossilOrigin-Name: 862f9828e078ae138c3533658c45e4c45155975794e752b9b3a71a693842f37a
2023-08-24 15:42:08 +00:00
stephan e6c299a5c8 Add a new non-goal to the JNI README.md.
FossilOrigin-Name: 53000cdae9b265752f8d8616679da621a571a6590c6eaeca23cf87a0e13aa4c4
2023-08-24 15:11:28 +00:00
stephan 8e52281659 Merge the jni-threading branch into trunk, eliminating the JNI API's prior threading limitations.
FossilOrigin-Name: 3739c8aa7080d8e1044ca51ab7b699b50da4d29620b35acfcea2745059b65bac
2023-08-24 14:49:29 +00:00
stephan 49ca4d20fa Merge into trunk improvements made to the wasm APIs which were too late for 3.43.
FossilOrigin-Name: ac9da5c79a4f56d25202d50974e16e2a463d77c99b1907aee2605d5a3e50a565
2023-08-24 14:43:30 +00:00
stephan e7a4685837 Correct JNI layer's misuse of an sqlite3-internal error-reporting API (no mutex held). Style cleanups. Eliminate lookups of per-thread state by approximately 85% across the test suite.
FossilOrigin-Name: 1f46ba8d3bc61af771c1e33d09ad25f0da4fc4f915f7a9f6223ebfd99526d81d
2023-08-24 14:31:36 +00:00
stephan 6043947eee Ensure that db mutex is held when using sqlite3ErrorWithMessage() to avoid a potential assertion in debug builds.
FossilOrigin-Name: b8f6a50a4bf9478324f0272d79f2fe6992a49b753e79e39a268c6afb261bb01e
2023-08-24 14:17:30 +00:00
dan 6859ca450f Fix a test case problem in wal2recover2.test.
FossilOrigin-Name: 81361769e17b3d1589c5d0f9991a139963c6633d26bc72f189a69e0391d8f52f
2023-08-24 13:42:10 +00:00
drh 4924030bf5 Version 3.43.0 for the reuse-schema branch.
FossilOrigin-Name: e7b1ee1a999ced9aaae66d3bf51d93c20909541a83ecba79a19e5573fa7d649e
2023-08-24 13:26:51 +00:00
drh 0205717b93 Version 3.43.0 for the wal2 branch.
FossilOrigin-Name: b5970fc4599d4fe81312e049420724b14e3d2c1de04769844fb1ab8d953170cd
2023-08-24 13:19:33 +00:00
drh 995f96f8e1 Version 3.43.0
FossilOrigin-Name: 0f80b798b3f4b81a7bb4233c58294edd0f1156f36b6ecf5ab8e83631d468778c
2023-08-24 12:36:59 +00:00
dan e174dba739 Change a variable name in fts5FlushOneHash() to avoid shadowing another.
FossilOrigin-Name: 3f5bec9a684f6b3d88383617831862fc5595a51ad99ec12430ac6650b704ccbb
2023-08-24 12:01:30 +00:00
stephan bfdc7ab5a7 Add more JNI docs, tests, and a handful of Java-side overloads.
FossilOrigin-Name: d19a431facbde6a6b960664674753ee85d2c051a76109ce7db0b079c65fbdea0
2023-08-24 11:57:51 +00:00
drh af885d9fc4 Fix typo in the Lemon documentation.
FossilOrigin-Name: 5daeb318c0472e8c004109caacd8bc977145a2c1bfa6f5bdcf87ab89ea51c30b
2023-08-23 18:42:04 +00:00
drh a4709f27d4 More changes to documents for GitHub-style markdown.
FossilOrigin-Name: f20f84ef717cea11f92d6a1ab3b4e0fec5752b7ec98ad5b59e06745e7e25b6ee
2023-08-23 18:29:41 +00:00
drh 2bab8774ba Fix the Markdown in the compile-for-windows.md document so that it works
on GitHub.  Improved header text in README.md.

FossilOrigin-Name: a25360c020275064711d0aa3eb3cec3d764b5266b86f9728243e501d082306ec
2023-08-23 18:26:00 +00:00
stephan 8cafdfa916 JNI cleanups regarding building with certain features disabled.
FossilOrigin-Name: a9e6d5158b8a4a6b8554a5f8f0a35785ee450d42ea877275dc27085e89716c18
2023-08-23 17:52:51 +00:00
stephan 1c3bf8a3e1 Remove unnecessary jclass-type struct members.
FossilOrigin-Name: d67255f7251cc5d1d27d77d4c84ff216e2da71202db989718189a6b4beff1cd0
2023-08-23 17:15:48 +00:00
stephan 495046ef88 Add a note to the JNI README explaining why the Java API has callback names like xFunc() and xPreUpdate().
FossilOrigin-Name: 415447a310f6a7d06b4aa9ef51f110cf8e2ef9545c69cb5983c367c50fe641d2
2023-08-23 13:36:27 +00:00
stephan bea9ed0f1f Bind sqlite3_preupdate_hook() and friends to JNI.
FossilOrigin-Name: d0c425b5c1d3aac5ead18a501a3760b4506d68d373cb3be484247042cf2fa8d4
2023-08-23 13:17:37 +00:00
drh 44a430b33c Minor adjustments to the wording about pull requests in the README.md file.
FossilOrigin-Name: 6b2d68afd8ecdfe8eb04e3f6c13bea4ba42f7c2f16e6fc4263ffefa29c55c527
2023-08-23 12:53:35 +00:00
stephan 4e97ab4296 Bind a subset of sqlite3_config() to JNI: threading modes and sqllog.
FossilOrigin-Name: fce8ecaf7f2e69a168978e6993e58c452c45f76c39da33f2869c9d947c16cab1
2023-08-23 10:36:12 +00:00
drh a2a4c3f14a The pageOnDirtyList() assertion is too slow even for debugging builds, for
some corner cases.  It makes the query appear to hang.  So make it an
EXPENSIVE_ASSERT instead.

FossilOrigin-Name: bb9dcdaf3244c4fc6872850a82b80c469203911e2f6e3a3211e508be39b59fa3
2023-08-23 10:20:39 +00:00
stephan 4c8ef3894e Numerous minor cleanups and code style conformance improvements.
FossilOrigin-Name: 6c92d884920e4ace54913fc60ceef6e43a4351f45a4cb3c4a0ed3d29d544a31b
2023-08-23 09:05:16 +00:00
stephan 336bc8a281 Improve C-side exception handling from Java-side UDF callbacks.
FossilOrigin-Name: aebbc24afb339ed07b7cd767fbc0d25f3e9c3d9bb5ef3d1c10b04b605c7261bc
2023-08-23 00:17:28 +00:00
stephan d53565b4f8 Minor Tester1.java cleanups.
FossilOrigin-Name: 70d936953ba55cfed32efe7e3a9d4b71da9a7ffc8818b6540471e0bf311bc688
2023-08-22 23:00:44 +00:00
stephan 3600976bf1 Fix Tester1 so that exceptions triggered via threads are not silently ignored. Disable auto-extension tests in multi-thread mode because concurrent threads rightfully interfere with that.
FossilOrigin-Name: 56b2a077ace6e6ad5834e1a597b710f212a5b7d5db5b9a27a41f2aa0f6952c55
2023-08-22 22:13:08 +00:00
stephan d1c7216b2f More work on the JNI multi-threaded test runner.
FossilOrigin-Name: 9a74ad716bded1e14333bf7c72392916f800d58a96240eabe4988ca5fc9e8752
2023-08-22 20:10:28 +00:00
stephan 87bb103038 Disassociate JNI db handles from the thread that created them, as it's no longer relevant.
FossilOrigin-Name: 8b78b737e66a399b04e555a8197f63a73198a4105cb2f37ffd5b0e6014302caf
2023-08-22 18:36:30 +00:00
stephan c675add616 Correct JNI binding of sqlite3_shutdown() to clean up all cached JNIEnv objects.
FossilOrigin-Name: 02e868690f97ca728b0f2dd018aa79a9d13c85dd85b164caa895d319ae8f3ff5
2023-08-22 17:51:57 +00:00
stephan a7e3a1c09b JNI internal cleanups and correct two leaked db handles in test code.
FossilOrigin-Name: f927a30b5bba35991f472084ebaf02779e84c343a4e84f0efb3df7679ff212f8
2023-08-22 17:36:59 +00:00
drh 675342a5a8 Fix a harmless memory leak in the sqldiff utility.
FossilOrigin-Name: 724bc15701f3f647c741b614d4ac4fd98e1cdfe49a85f48c64fb1df5b83811b0
2023-08-22 16:44:33 +00:00
stephan 9828aa223a Move the JNI per-thread cache of NativePointerHolder refs into global space. This allows better-targeted mutex locks and incidentally eliminates the lagginess and post-run hangs in Tester1's multi-thread mode (presumably caused by deadlocks).
FossilOrigin-Name: e209f56a9745695aadc04418c7bebe62b79e38e5aee26c3248a30f73bfa460c2
2023-08-22 15:30:35 +00:00
dan 44e24e9640 Add makefile target "sdevtest" - like "mdevtest" except asan and usan enabled for the non-debug test.
FossilOrigin-Name: a449d650cd3dcd1baaeb3a3de2aaaac45594397e04f95fe637b0fe4ddb273404
2023-08-22 15:19:50 +00:00
dan 310700228b Update version number in autoconf/tea/configure.ac.
FossilOrigin-Name: 055ae724ff34bd41a1d97180b90fce081861856ac67572a633d31f518c20aa1e
2023-08-22 13:41:18 +00:00
stephan 0a102087d8 Move most of the per-JNIEnv global Java class refs into the global state, saving a bit of per-thread overhead.
FossilOrigin-Name: 7342bf578790e1a87c128a7c1c7745fe2e7c442890370feb160d406597d4d8ec
2023-08-22 11:34:34 +00:00
dan 216524f858 Update autoconf/Makefile.msc (by running mkmsvcmin.tcl) to support the "ASAN=1" option.
FossilOrigin-Name: 17028ba7a9bb1c5e26cb8c0d051d709df95d9f779fd3b1ec42adc924246f8f19
2023-08-22 11:20:38 +00:00
stephan 484f9bed4e Minor JNI cleanups.
FossilOrigin-Name: b88910aaaaaaa0936974379bb3eb8a5a3a634395b14e67cc9030f8a520f471f1
2023-08-21 23:45:19 +00:00
dan a7866eefb5 Merge latest changes from the wal2 branch into this one.
FossilOrigin-Name: 4d8df0c426b8ce3db6cfb71e23f752026ef886b9bb833dc4be9717db9955b1db
2023-08-21 18:31:53 +00:00
dan aabb710293 Fix problems related to structured-exception-handling on this branch.
FossilOrigin-Name: d3d77e35ea39df9e22032a7e1af5b7f38d53a4d43bc46afe36e342cdcfd17528
2023-08-21 18:25:41 +00:00
dan cc74eafee5 Merge latest trunk changes into this branch.
FossilOrigin-Name: 135bf72c6b5f436b11463dd40b3a73bcc610bf9c2dcbe6646dbedd0d9325f6a9
2023-08-21 18:02:57 +00:00
dan da7662a4b0 Fix an undefined integer overflow in fts5.
FossilOrigin-Name: f5c50349b1e4da233b554f371be49297b8b40ca5fe0188899ed906f682bf70d6
2023-08-21 16:14:58 +00:00
drh ba481c3756 ifdef adjustments so that SQLITE_OMIT_WAL works with SQLITE_USE_SEH.
FossilOrigin-Name: 0462a2612d1fc1d077acf60ef415f358aa2c44174c4ef28f4bbfdbcbf03d3b7c
2023-08-21 15:29:21 +00:00
drh 3404b452a4 Fix another UBSAN inaccuracy in fuzzcheck.
FossilOrigin-Name: a6fb9de8f87261194e6034376f62b1cac30b764f106dd3adcf728e020f8a4fd5
2023-08-21 14:40:19 +00:00
drh 114ad2f86f Address minor inaccuracies in fuzzcheck and RTREE found by UBSAN.
FossilOrigin-Name: a64e6c2fd8ef1d2d7f14aa9265629853298bbc480c2683c79dfd24847a61b68b
2023-08-21 14:28:47 +00:00
stephan 1089277105 JNI cleanups.
FossilOrigin-Name: 0a84131008a2e7886dac64a3545dea634811f6eac2b90885ec9c61ed1e6544c3
2023-08-19 14:49:08 +00:00
stephan 6f92f35449 Merge trunk into jni-threading branch.
FossilOrigin-Name: 8254479c6ff1ea3cc9e56de1698db8405c03da90b9bf4c401182e47e0842baf8
2023-08-19 12:34:23 +00:00
stephan bfa486d5fc JNI doc additions.
FossilOrigin-Name: 0c7ac34f30e1f7e35a2ac4e5e55e5f24857b24afa81a7abecba60f1c9c68b9ff
2023-08-19 12:32:00 +00:00
stephan 187da43379 JNI test code cleanups.
FossilOrigin-Name: e202b6e69da8cced114d027cf2e91a04dfdd50b601b3274214783f7d750c558c
2023-08-19 11:52:36 +00:00
stephan 8d9179bd07 Minor JNI cleanups.
FossilOrigin-Name: 1cecb9e0383aa78c491f9ba88c831a88b4b2d40ceef1b87be494b6ddc0789e41
2023-08-19 11:26:52 +00:00
stephan 3cf6c0f276 Add multi-thread run mode to JNI Tester1. It works but hangs on exit sometimes for Java reasons as yet not understood.
FossilOrigin-Name: bdbaf7a4534f40e550b646979e67e7b7731566bb5a2631ed376ac85a9bec40a7
2023-08-19 10:43:05 +00:00
stephan 46d677e713 Replace JNI::NewStringUTF() for the remaining cases where output may be incompatible with MUTF-8. It is now only used when we know the output to be plain ASCII.
FossilOrigin-Name: 2d955eef25ab116c487ebc34c6f2d2836d310af239ef1993f5aeee5a3f68d590
2023-08-19 08:22:34 +00:00
stephan 78ad745ab9 shell.c.in help text typo fix. No functional changes.
FossilOrigin-Name: c319033276c3565d0f1f2cae1c91791940d322fe79696bc26d74fddeb2664373
2023-08-19 08:12:10 +00:00
drh 4d8eb16f07 Rename the decimal_sci() function to decimal_exp().
[forum:/forumpost/fa027bb0ab|Forum post fa027bb0ab].

FossilOrigin-Name: c48f1be8cc505a7e2902c79e26c1d9a121ff5c55785ac812d2e09232b2414695
2023-08-18 15:39:38 +00:00
dan d5a9d522fb Avoid running test case sort2.2.3 under address-sanitizer. It runs out of memory on some 32-bit platforms.
FossilOrigin-Name: 925be7539947078547ee129b3d328ab0c59b19c6e2e5a391840f2d58f6d59acd
2023-08-18 15:17:52 +00:00
stephan 656f6c0c80 Improve exception handling in OpfsDb.importDb().
FossilOrigin-Name: a4eedd63b58f5c273b671f687cb68a32259963dbb053a710770383c735fb7f26
2023-08-18 14:41:21 +00:00
stephan ccbfe97cd5 Extend the importDb() method of both OPFS VFSes to (A) support reading in an async streaming fashion via a callback and (B) automatically disable WAL mode in the imported db.
FossilOrigin-Name: 9b1398c96a4fd0b59e65faa8d5c98de4129f0f0357732f12cb2f5c53a08acdc2
2023-08-18 14:16:26 +00:00
drh f0eabde9a4 Enhance the CLI and the fuzzcheck utility programs to report whether they
are compiled as 32-bit or 64-bit for things like the --version option.

FossilOrigin-Name: ec7eeb055bfb0e5a4467d8a45fa53d84bb8ae80ca0474b687e2783e971648008
2023-08-18 12:15:44 +00:00
drh 2cffc5be2a Enhance the notes on compiling for Windows11 to explain how to do
32-bit builds on a 64-bit system.

FossilOrigin-Name: dd749e4042bdffb20fe71dd2dba91eab5edc32e4b2773c2fadba43cb54027a17
2023-08-18 11:36:33 +00:00
stephan 383df02b16 Remove some obsolete JNI-internal docs.
FossilOrigin-Name: 00a2a3736a6dcde81d920815520040f3c47f965165e7128ca1f4062e6ec7c17c
2023-08-17 22:04:07 +00:00
drh 2e0ce58d2c Do not allow bound parameters in the ON CONFLICT clause of an UPSERT.
dbsqlfuzz 9983e2c77634a8ccf33b5c91fa9982599de5f9e9

FossilOrigin-Name: 3c06709335eb4b98e3a684e3ebbae69eeb6a21b452bce29159c82bb632d6a042
2023-08-17 17:48:20 +00:00
dan e15b35d5a6 Fix an error in documentation comments in fts5.h.
FossilOrigin-Name: 24d96bd65d8ccdd50a6f4fe217d196728a2f8658d128f9352c287f64e034104a
2023-08-17 16:59:35 +00:00
dan fab6882fd6 Add extra test to fts5synonym2.test.
FossilOrigin-Name: 39ea11be3b35c5ecde9bc0fe66af673b8f440dc523422887d05a29feef036b01
2023-08-17 16:49:06 +00:00
dan 699803e114 Add "Windows-Sanitize" configuration to "testrunner.tcl release" tests on windows.
FossilOrigin-Name: e68b46dc6aff48c81a2a5e7662d0f694615b64aa50bcfee9a0b8b25fa4e5f33d
2023-08-17 16:06:17 +00:00
dan 58c44132d1 Have all test names in json101.test start with "json101".
FossilOrigin-Name: af56d7b6b9c50f2541a51fcb7d94b89aafba4f343a9103b21a15aad3f283c63e
2023-08-17 15:14:43 +00:00
drh 81ef0f8e89 Enable address sanitizer in Makefile.msc by adding the ASAN=1 argument on
the nmake command line.

FossilOrigin-Name: 3eaadaff85111103c9441a9d6cd31291b95db9f3a6ec7003e0ff4ce5db7490da
2023-08-17 15:11:59 +00:00
drh 36a9f5c24c Fix a possible UAF in SEH if an exception occurs at an inopportune moment
during WAL processing.

FossilOrigin-Name: 91b91037e348fa10cf6a9d8a0ffbfdebb4a40e0e336b65b112ed1c828e75cda7
2023-08-17 14:19:44 +00:00
stephan 6c5f96fc4d Tighten up the JNI auto-ext handling.
FossilOrigin-Name: c09c8d05a20d916a9d9304eeea723ef7666a862a9e53f5feeeb1b03f9153d4b2
2023-08-17 13:13:22 +00:00
stephan 1dcb246988 Minor internal JNI cleanups and fixes.
FossilOrigin-Name: 0e9437de026cbfb333b90bb3400f1c015f85d49d73a25ad1000623216b88bfa0
2023-08-17 12:44:52 +00:00
stephan 0fa2545e7f Remove the FIXME markers related to threading. Code style cleanups.
FossilOrigin-Name: 154ab26dc6ba2d1fd976e8fe6dc1b1a06c734f7e9a276a3edc5c2f30b0d6d36a
2023-08-17 10:49:06 +00:00
stephan 95484726f6 Add SQLITE_EXTRA_AUTOEXT, similar to SQLITE_EXTRA_INIT but adds a builtin auto-extension provided by the client. Suggestion from [forum:00829394c74a670f| forum post 00829394c74a670f].
FossilOrigin-Name: 423e77277a61d7febf4c3fc737981fa22a82b5c774a8ada5375a01a0611535b2
2023-08-17 09:49:53 +00:00
drh 936f7deee4 Refinements to instructions on compiling for Windows.
FossilOrigin-Name: d543c36c35f71c5f0a7ebf6f496feca40d16566d0c5b2c2ba205ff43437ffcd1
2023-08-16 17:23:42 +00:00
drh 7335f75daf Improved instructions on how to build SQLite on Windows 11 and similar.
FossilOrigin-Name: 0f6b2b33736ee07f17f3a4e5f077bb4d0e2481c8f81251b8ce6b78510f372237
2023-08-16 17:12:26 +00:00
drh 0d83c71b34 Make sqlite3_stmt_explain() accessible to loadable extensions.
FossilOrigin-Name: a237a31ff4a7cd3dc744af1e1ebb9466e42eebafa31c1b5c0027cbaee9babc9b
2023-08-16 15:27:49 +00:00
drh 7309b50fbc Mix the current process ID into the randomness used for generating
temporary filenames on Windows.

FossilOrigin-Name: 775a36ee093df4b5f7529a43eeaee9d5a9a943ad5ed8ae03bc74e459e87ba438
2023-08-16 15:10:07 +00:00
dan af8980bdce Update testrunner.tcl to use environment variable %NUMBER_OF_PROCESSES% when running under tclsh on windows. Also modify the internal database schema used by testrunner.tcl to be compatible with old versions of SQLite.
FossilOrigin-Name: 6542ed3b9e028c44aca504eadca843ee9b2ba08f5f650523238dd1253f7e221b
2023-08-16 14:18:53 +00:00
dan fe18355267 Fix test script problem preventing veryquick.test from running.
FossilOrigin-Name: e73886574042108eb31641d0820c273c10b83fbf08ac6cb52d3e4c27830e2c23
2023-08-16 13:55:38 +00:00
stephan ff9ff54801 Cherrypick [00ac653562a6] to remove stray JNI debug output. No functional changes.
FossilOrigin-Name: e12d70f38c0ed5c0abf9a1e30ce4db59a4426e1e8cb1f8f07834fc1bedfcc65a
2023-08-15 21:44:22 +00:00
stephan c131675401 Cherrypick [653ed92dc391] (went to wrong branch). Doc change only, no code changes.
FossilOrigin-Name: 79a14457f0067814e7c33de709e2f9e19ab880d3eebbcb6f9fcce52a4720ecac
2023-08-15 20:57:42 +00:00
dan f05630fb1c Update testrunner.tcl to make it easier to add things like mdevtest.
FossilOrigin-Name: a531b71c4fd64fff6008876af825107ebc403f80dcb95273a05936ff6587b7aa
2023-08-15 18:52:25 +00:00
drh baa574ef32 Improve the portability of the src-verify.c tool on Windows.
FossilOrigin-Name: 4068d94acd042a0a9887fba8ba38fa94c6f41a6d3c7a61d5514a57f613b3d278
2023-08-15 18:08:14 +00:00
dan 8c5611a169 Fix problems in fts5 test code.
FossilOrigin-Name: c16ed3b1cc7b4c21bfa96062bf9028ae071c89e63df3b444b59f1753594e4e94
2023-08-15 14:53:49 +00:00
stephan abfe646c12 Add note about the current threading limitation to ext/jni/README.md. No code changes.
FossilOrigin-Name: 653ed92dc39185cdedfab3ea518bc7ec2d2826120e5fa4cbdee3343301396184
2023-08-15 13:01:20 +00:00
drh b93760e1e7 Use a dodgy substitute if the INFINITY macro is not available. See
[forum:/forumpost/8e66e19bb9|forumpost 8e66e19bb9].

FossilOrigin-Name: 4ae72c896761df5cd37c98ca3eeee7d4c6af3e4ccf74edd88ff902ef38f0765b
2023-08-15 12:27:27 +00:00
drh 00eee07911 Fix the PRAGMA case_sensitive_like command so that the LIKE function continues
to be innocuous after the PRAGMA.
[forum:/forumpost/925dc9f67804c540|Forum post 925dc9f67804c540].

FossilOrigin-Name: 84c268c34cba7207a90dad2a8e972ce90c85304e91f4933c76963822ad1ae48b
2023-08-15 11:58:22 +00:00
dan f0c1ea9e86 Fix an off-by-one error causing a buffer overread in test2.c.
FossilOrigin-Name: e1edf95ecc8f6840fd62abe83121fddeab7413c316b2e20b60fd3b2c8fdd4dca
2023-08-15 10:57:08 +00:00
stephan 6b51e35a9b Minor reshaping of Tester1 moving towards making a multi-threaded run mode.
FossilOrigin-Name: f104c14c26c123ee78c09fc1bc59efb8668dc624da05c1d8dbeaf3c9dd02a393
2023-08-15 09:26:47 +00:00
stephan 39c3f5f363 Remove a piece of stray debug output. No functional changes.
FossilOrigin-Name: 00ac653562a66aad3112ea322d08be68e05e6bf7413c814dd3f81bf850fcf43b
2023-08-15 09:16:41 +00:00
drh db9a708ee7 Fix the autoconf makefile for the CLI such that it disables double-quoted
strings by default, as all other builds of the CLI do.
[forum:/forumpost/9794b6aaa5|forum post 9794b6aaa5].

FossilOrigin-Name: 85fd4f0ef41a883448f16d55b7f8bd5c7b1d9c89a3d7d9bcae9c2744ed843099
2023-08-14 19:10:48 +00:00
dan 9541fbc283 Fix a problem in sqlite3_snapshot_recover() introduced by the [8a6b0c24937e855b] merge.
FossilOrigin-Name: c2577eb0a4a627c65d675f1446514843486092be9cb271b682793a0107e496cd
2023-08-14 18:21:56 +00:00
stephan 9019e2e667 Bring handling of the Java auto-ext handler more in line with the core in terms of locking and mutability during traversal. This removes the explicit synchronous requirement from the Java open() and auto-ext bindings.
FossilOrigin-Name: 42994b952e092ae4fa319395208622e887387ca3ff8ac57961c824a6c272bf0e
2023-08-14 17:12:55 +00:00
stephan 24b4cdd8d3 Remove some obsolete, commented-out makefile code. No functional or build changes.
FossilOrigin-Name: ea574cd5de6dbc7db2f4cffe0b6c83c795c4ba634ae1984e5c95bb3529daa5bb
2023-08-14 14:21:28 +00:00
drh 1f72b53461 Change the name of the ".binary" command in the CLI to ".crnl". The sense of
the setting is inverted.  The original ".binary" still works for backwards
compatibility but is now undocumented.  ".crnl" is also undocumented on machines
where it is a no-op (all machines other than Windows).
[forum:/forumpost/8bd0b0fbdbc12477|forum thread 8bd0b0fbdbc12477]

FossilOrigin-Name: 544de2da09cd7b9bbb6cb4c52bb22325ace3391fca00c6a43847bfc158032b66
2023-08-14 13:33:19 +00:00
stephan 7f2dea75ad More work on the JNI-specific mutexes. Rework the NativePointerHolder cache lookup to be slightly simpler and O(1) instead of O(N).
FossilOrigin-Name: c84ded0e59aea4861d72b53b4b40cf580747c0f6ca58c334a996f1a825276cb5
2023-08-14 13:27:40 +00:00
drh 592ae2823c Fix harmless compiler warnings reported by MSVC.
FossilOrigin-Name: 391e21bb6e0b2f4632972b6617a3a18192f88deb29eee5bc060846468e624b21
2023-08-14 12:20:44 +00:00
drh 07ef60350d Enable SEH be default in Windows builds.
FossilOrigin-Name: faae792fd5db7afeb7e4345d96f14c8986ea9a3cd4764ddee71314bfca36ac71
2023-08-14 12:05:38 +00:00
stephan d518e94adb JNI-internal docs and removal of obsolete code.
FossilOrigin-Name: b62d93258b6a661f3a9b61468b3b641c14faf2d2196f78aca95fe14de43c9444
2023-08-14 08:28:46 +00:00
stephan 6a1ed4c811 Internal API renaming for clarity's sake.
FossilOrigin-Name: 911e4fc5aaf9478214095a65f74af3ebca883922c36cf7a8d911116c42cf9de8
2023-08-13 20:58:12 +00:00
stephan 88381e53fc Add a mutex for auto-extensions, tied in to the open() process since that's the route into auto-extensions.
FossilOrigin-Name: 8da97e0db4eeacf91aa6fd909fd7cb73b050d194dfc7739a502b55f7eca6d7b1
2023-08-13 12:40:27 +00:00
stephan 4b4a911c5f Add some docs and metrics for the new mutex internals.
FossilOrigin-Name: 33d1780b43182d2574adbc1928707af825c485c99762738e58bc6d7c6c52ac6a
2023-08-13 10:28:35 +00:00
stephan 8a6888c19e Remove SQLITE_DEBUG from JNI build.
FossilOrigin-Name: 82f004cf7408c503c5787b46c7b13ceb0a20a3ad1d4f69d56127772c40e214a1
2023-08-13 10:22:15 +00:00
stephan 71e5694cd5 An initial attempt at protecting the JNI global state via mutexes at the C level instead of relying on Java's synchronized keyword. It seems to work but increases the run time of the single-threaded batch tester by roughly 3 times.
FossilOrigin-Name: c64e6a52ac79164be37fe643a4a39bd187af198a379410def8b8419f7c2224d4
2023-08-13 09:53:27 +00:00
stephan 0c07549fd6 Bind sqlite3_interrupt() and sqlite3_is_interrupted() to JNI but with caveats regarding mutexing of the JNIEnv cache. Add a loud warning to the JNI 'dist' target that it should be built with JDK8 (a.k.a. Java 1.8) for compatibility reasons.
FossilOrigin-Name: fbf99a2423dd20e4544bdeea85f714e9368ce3b92fefe97efb39a0fb4a557abe
2023-08-12 23:47:58 +00:00
stephan 202651fe86 Merge the Java Native Interface (JNI) binding into trunk.
FossilOrigin-Name: 48b13edcec6935bf125b265b41a3e6f7b2407afff89d5b4daa2939e3c5679ca0
2023-08-12 21:39:18 +00:00
stephan 54402964b0 Update wasm's push-testing rule to exclude files which were recently removed from the build.
FossilOrigin-Name: 0a6930a7ff8f8c6ca244d1d654532f3d2a02d77ef67c6cae0c53092743d59ea6
2023-08-12 21:08:41 +00:00
stephan d8a94dbb36 Correct JNI .jar rules to only include *.java/class files, not *.* (*~ files).
FossilOrigin-Name: 1ba7754045a009d9c94b23ac76b9bb8d9c9cb24d42dcdf1203ee75ac85765d3e
2023-08-12 20:50:29 +00:00
dan 46451fa8b5 Fix the mdevtest target so that it works with msvc.
FossilOrigin-Name: 01f49448cd0cfe3af499aedfe887b7b0be4f1ab09cd5a16119ddafb32b107708
2023-08-12 19:58:47 +00:00
dan 890a9ede3b Fix testrunner.tcl so that it restarts itself using [testfixture] if its current interpreter does not support [package require sqlite3].
FossilOrigin-Name: 408be98dddf500f2d6c9049703997836d80d156d86cf297f664175a107f7331e
2023-08-12 19:25:26 +00:00
dan 7c036fabe5 Have the mdevtest target support the OPTS variable on unix.
FossilOrigin-Name: 940f58d9b944d3344b40a4bc020842d9b6703685c9c51f4010e265afd98608ff
2023-08-12 18:23:38 +00:00
dan a6f09a0193 Add "mdevtest" target to makefiles. Equivalent to running "devtest" on both debug and non-debug builds.
FossilOrigin-Name: af354fbc0be06d5792a1e712ec4e9cec4c6cb4521335aa31f8f51ca5c94499ae
2023-08-12 17:36:57 +00:00
stephan 68522e1627 More JNI docs.
FossilOrigin-Name: 290028d3ca5638f3bb18a0b243c7ba3c31c8a2b1a837c36ec29e0fc4ed6533f0
2023-08-12 15:37:53 +00:00
stephan 5b3a754360 Further simplifications in the interface of the OutputPointer family of Java classes.
FossilOrigin-Name: 962c3e0de2d64ab8a2bcf1a19f9c4224df3d15a41ac9f9b29da685be95c4ef7a
2023-08-12 15:09:09 +00:00
stephan c216df75c6 Tweaks and docs for the OutputPointer family of Java classes.
FossilOrigin-Name: 265c8fd0d4d425054f6bf7e9cb607ad2e0e46189f16c3014f7fdf9b650085497
2023-08-12 10:39:26 +00:00
stephan 613390680d Bind sqlite3_db_status() to JNI.
FossilOrigin-Name: b79477a0af94127b0638a8822de01156bef855a7e167f678809e1c978e1a0c3e
2023-08-12 10:27:08 +00:00
stephan 238bea2ae6 Bind sqlite3_status(64)() to JNI.
FossilOrigin-Name: cefb6614e65ca1764ec72702f92f801382e63aa9b221fc9c68719d497e7499fd
2023-08-12 10:06:59 +00:00
stephan 6c6badd2ef Modernize the sqlite3 shell man page a bit, per [forum:d9aa201dee553bb2|forum request].
FossilOrigin-Name: 3bc0693c4633f545f09dbee702e25354504b20836373a068447e6c61cb2ebd79
2023-08-11 22:40:06 +00:00
stephan 48dcdd3b74 Add timing info to SQLTester.
FossilOrigin-Name: b69b5facbf94e03e74d4a739ab85c5baac1c9ecbea8c330b2135d77e525b5d8a
2023-08-11 21:25:33 +00:00
stephan a6746456d9 Merge trunk into jni branch.
FossilOrigin-Name: 4f0aeeba0287e846908180eab6f7080ebe1323ebe49340771864d110e1ca5b2b
2023-08-11 21:24:08 +00:00
stephan 783f5b5aa2 Fix a makefile deps problem which caused ext/jni build to fail if sqlite3.c/h were not created beforehand.
FossilOrigin-Name: 101de670774f63757180282763730aa53e70198bd7a674c27e6044632d39d22a
2023-08-11 20:32:40 +00:00
dan f08877f094 Merge latest wal2 changes, including support for handling of structured-exceptions in MSVC builds, with this branch.
FossilOrigin-Name: 331f2f3e5db9b6139be984f1b959cd0d51563adaa68452aa2f42741c69bc6319
2023-08-11 19:36:24 +00:00
dan 45d92dd539 Merge latest trunk changes, including support for handling of structured-exceptions in MSVC builds, with this branch.
FossilOrigin-Name: 00bfae9ffafd71dfd0bbdbdc174e4ed7d7a850b385416f83be36a4f4a55d1873
2023-08-11 19:33:54 +00:00
dan 2a28cff2e7 If SQLITE_USE_SEH is defined, handle structured-exceptions thrown by MSVC builds if the *-shm file mapping is accessed after it becomes invalid for some reason.
FossilOrigin-Name: 8a6b0c24937e855b710f97b4aea973eff53e6d43e1182842731547aa4b37db2a
2023-08-11 19:31:51 +00:00
stephan 16c8c96745 Minor SQLTester test tweaks and have 'make tester' include the out-of-tree tests if they are found.
FossilOrigin-Name: 6c8538d83495ce65dbd7417263b3b06dbbb2a649e9a61a743911944599d75ffc
2023-08-11 18:59:57 +00:00
dan 4e5c5daae7 Fix a problem with handling savepoints in BEGIN CONCURRENT transactions.
FossilOrigin-Name: ca422d00d7843ee72a1ae20753ba512896d6f6d2b1218523323946bf66f3c758
2023-08-11 18:31:17 +00:00
stephan 7009c43eaf Bind sqlite3_sql() and sqlite3_expanded_sql() to JNI. Start marking C-side functions which would need explicit mutex support if we remove 'synchronized' from their Java entry points (but there are many more left to mark).
FossilOrigin-Name: c7fb32d1ef30d34449c3289c384ce33317c770927534af20d4b96fa385da40bc
2023-08-11 18:04:53 +00:00
stephan c3b7e1c4e8 Add a doc link for the new sqlite3_js_posix_create_file().
FossilOrigin-Name: 0d7aac45b8e7078cc80757e12d6f0b2584f2b0b184dacc2348ad3519978e5bf9
2023-08-11 17:45:23 +00:00
stephan b949244ea1 Add sqlite3.capi.sqlite3_js_posix_create_file() and oo1.OpfsDb.importDb() as alternatives for the newly-deprecated sqlite3_js_vfs_create_file().
FossilOrigin-Name: da6eaf8d8258f3e2c8633fd7faf4e90c3307b5c60bd8b69c626b3c82b19dbdef
2023-08-11 17:38:17 +00:00
stephan 7e13152952 Deprecate sqlite3_js_vfs_create_file() because, it was discovered today, its out-of-scope use of the sqlite3_vfs, sqlite3_file, and sqlite3_io_methods APIs triggers unresolvable assertions in the core when built with SQLITE_DEBUG.
FossilOrigin-Name: f3647a3ac8eca8c821b0b1e403da7bfb0feabd0eb5ee83709cd4956dfc56a492
2023-08-11 14:31:20 +00:00
drh 08901378d9 Merge all the latest trunk enhancements into the bedrock branch.
FossilOrigin-Name: 859932ebce830c757cf4e889f2efc54807bd4d7947cf5d64a5d9e570bf75ed80
2023-08-11 11:46:08 +00:00
drh 6fe6b73a8d Merge the latest trunk enhancements into the wal2 branch.
FossilOrigin-Name: fe1bf30ca0c529c3c68f2531e28aca5659aef5d15c2e3a6925ecd35a5098386b
2023-08-11 11:40:32 +00:00
drh 59d01de837 Provide the -DSQLITE_LEGACY_JSON_VALID compile-time option to restore the
(incorrect) legacy behavior of json_valid(NULL).

FossilOrigin-Name: 00bc9f1b573d683829bf5eb301606c38d6a60fba957d8edaf59116c02cc650bf
2023-08-11 11:30:43 +00:00
drh b8a0fc52a2 Fix bug in the test case for the previous check-in.
FossilOrigin-Name: c5daae88612607aa7a8b13b021cf586fc66e8bcd1fa9d948a860c881b7247761
2023-08-11 11:24:44 +00:00
dan 7a1cc96197 Update test scripts specific to this branch so that they work on windows.
FossilOrigin-Name: f0ca13edad058628e454cc895faac5751de77115c5fea62e98c29d9fdb6b0559
2023-08-11 11:23:35 +00:00
drh 91c0092917 Up until version 3.42.0, there was a bug in json_valid() such that it would
return False (0) for a NULL input.  That bug is fixed in 3.42.0.  This
check-in adds a compile-time option -DSQLITE_LEGACY_JSON_VALID that restores
the old buggy behavior for applications that depend on it.

FossilOrigin-Name: 15c2eadbff8e732cca45d6c3771d1fcea5aab2127e87f2a611b41ccfef4d1a0d
2023-08-11 11:12:46 +00:00
stephan a0423e0fc1 Mark _all_ JNI binding funcs as synchronized so that Java can lock them and protect our global-state access. The alternative is writing a mountain of C-side code to do the same thing.
FossilOrigin-Name: afe190a940441de9bef8835c2dc6d278f861a772c3b7c7a2d399b2eabd4872e3
2023-08-10 21:50:52 +00:00
stephan 5c9f5e4022 Work around jdk8 and jdk19 mangling the C name of sqlite3_db_config() differently. Correct the variadic arg handling of the JNI-side subset of sqlite3_db_config() options.
FossilOrigin-Name: 746894c3c043c47f8b4c231de8921df81c5d0634260d299359bea73132dc7867
2023-08-10 21:29:59 +00:00
stephan 0c08c8c208 Add a working dist zip file for the JNI bits.
FossilOrigin-Name: d6a4d212ceba662470d8957b6a8d7075d18a84bd0d3e13ce7adcab03604fc3b7
2023-08-10 20:52:14 +00:00
stephan a52b8c8845 Initial pass at 'make dist' rules for the JNI bundle, but they still need a bare-bones, posix-make-compatible makefile to include in the bundle.
FossilOrigin-Name: ff54e66a4d43f2f0f8b25ded970779e6760865a05346e09b39607bb035b02bd7
2023-08-10 18:57:37 +00:00
drh 89e1caf294 New testcase() macro in the tokenizer, to better document its behavior.
FossilOrigin-Name: b2fdac0b151864eb2aa79f0b0ee60f9c6d9f3eb8c7626605eac17a02a8cf59bc
2023-08-10 18:50:00 +00:00
stephan 2ee01b0c9a Merge trunk into jni branch for the newly-relocated version-info tool.
FossilOrigin-Name: cc8e8cba67c0dcfb9b416041a19456cf5248d909f3efb6fee707a5950be4f374
2023-08-10 17:39:26 +00:00
stephan bb51af68f4 Move ext/wasm/version-info.c to tool/ for re-use in build other dist bundles.
FossilOrigin-Name: 4b0871fd367b6d9706e892aa13f64604967f5e3ba92381960f73aeabd3d23f84
2023-08-10 17:32:37 +00:00
dan 6de6121e36 Merge latest trunk changes into this branch.
FossilOrigin-Name: 3ed89c344fcb3b7ee8b764d95144643e42e053e1116150d6eda8355fbd6669df
2023-08-10 17:07:34 +00:00
dan 1e6ee1929c Mark test script wal2big.test as "TESTRUNNER: slow".
FossilOrigin-Name: ae7f3bcb5babfa50084f5b4d3c5663489dbf306440706152642b7f0fb47fa1f5
2023-08-10 17:04:29 +00:00
dan 832406c2b4 Merge latest trunk changes into this branch.
FossilOrigin-Name: 72f7807b34eee1fcfd94f72d27535e5273ed1f7ff3d464179a774b7c7241f5f5
2023-08-10 17:03:33 +00:00
stephan a22a2d266f Add SQLTester --keep-going flag to allow it to continue to the next script after an error.
FossilOrigin-Name: 4d635f781b55ed9011bdf07ee6bed2d004b1c2ebba76aa110e26d8fe3152a733
2023-08-10 16:42:22 +00:00
stephan 92f699d384 Document that auto-extensions registered via JNI will fail if they open a db (to avoid triggering an endless loop in the auto-extensions). Discover the hard way that JDKv19 creates different mangled JNI names for some functions than JDKv8. Start reformatting much of the JNI API decls in prep for making them even longer.
FossilOrigin-Name: 99c0941f1c006622932a9cca12661f354f363a6c8a2b5675ea66149e0a9eb927
2023-08-10 13:10:25 +00:00
stephan 1528014703 Resolve two assertions in the auto-extension JNI which were triggered via new SQLTester infrastructure. Move SQLTester's db-init SQL injection into an auto-extension.
FossilOrigin-Name: 2952906c30bc2b7987f2c39837d56bd121f5817dc094e6ccdb6d4eea5e9b8d17
2023-08-10 12:36:40 +00:00
stephan 61186d1243 Make test completion status more visible at a glance on modern terminals.
FossilOrigin-Name: a4e96c306c4c270f417243e7923d7e6c4f860528dd67990dfd8d9768a6c4873f
2023-08-10 11:15:20 +00:00
stephan e51fae4134 Disable REQUIRED_PROPERTIES handling in SQLTester, per /chat. Scripts with that directive are now skipped.
FossilOrigin-Name: ddc534cb25b59faf18a860a51f2dd41a1a73963aeb541b9553301fe784608393
2023-08-10 11:04:46 +00:00
stephan 0a8cc1f634 Add a visual indicator (emoji) to TestScript verbose messages, dependent on the message's level of verbosity.
FossilOrigin-Name: dc323d3894f2d53470cd8be261632267fa3d2af73500acfa1e9adbfa53b771fd
2023-08-10 10:58:55 +00:00
stephan 34fac74362 Add the current --testcase name to SQLTester --verbose output.
FossilOrigin-Name: f87367402b25adf30f35ab75aa5efc495230d4a83f2fc10b99734c3f3f593840
2023-08-10 10:44:53 +00:00
stephan ec8298fbd1 More SQLTester docs. Add --verbosity command to help zoom in on script areas while debugging. Spice up test-start/end output with some emoji.
FossilOrigin-Name: 8dd08021496f504c23945ecc2bbe1e4a13109fdd03457ca6269b4cb1cc4cd04c
2023-08-10 10:34:50 +00:00
stephan 63fb588d30 Give DbException the option of closing the db to simplify error handling in one case.
FossilOrigin-Name: 908c9a44505422a3a15bef3a174d8b931863bc9c74485311a0e62cfec30087bd
2023-08-10 05:25:13 +00:00
stephan 6a61a19f9a Initial version of REQUIRED_PROPERTIES support for SQLTester, with TEMPSTORE_(FILE/MEM) and RECURSIVE_TRIGGERS options.
FossilOrigin-Name: 48d16c9d2fe5f54b09004b4f09759c4e2ad247ae84130feb557951e32f48976a
2023-08-10 05:14:22 +00:00
stephan f703dfa381 Generic cleanups and fixes in SQLTester.
FossilOrigin-Name: fc5d3cc30d2b96da42ea10dfb39f1631ff93b8384514fffd641b343df51da2a6
2023-08-10 04:24:12 +00:00
stephan eda67031de Change the SQLite3Jni API annotations to use SOURCE retention (used only at compile-time).
FossilOrigin-Name: 3c3fea6bf284721ac376e2ab5a757cf30245dd39264aaf98a8d6cd5575484275
2023-08-10 02:09:12 +00:00
stephan 0c6df29cba Move all of the SQLTester code into a single file, since it's only got 1 public class. Remove 'public' from many methods which don't need it. Add more documentation to it.
FossilOrigin-Name: 2815d676951abdab674c374fd903486ea5796f8ee4cb338d41f19693419f8471
2023-08-10 01:44:48 +00:00
stephan 2a91065145 Defer static JNI-side init of SQLTester until main() is called so that its auto-extensions do not leak over to clients of the main library.
FossilOrigin-Name: e461fdd53bd3212bee24ec5f5d5c234011ab30f3f67e115de9f85fdb760e3848
2023-08-10 01:19:40 +00:00
stephan 58c2ca4483 Merge trunk into jni branch.
FossilOrigin-Name: 52fa6f78414c41073431c166550806bb8a835bd38cfc1236c9363784c78b81b9
2023-08-10 01:05:28 +00:00
stephan 50a17a1b3a Treat all args to --glob as a single glob.
FossilOrigin-Name: 4737bd345732e34cabbf90821e007d3027e68d6583c2e05e1e8bf5920373a6dc
2023-08-10 01:03:19 +00:00
stephan 3a0e45d0af Remove --print's ability to read ahead.
FossilOrigin-Name: 15cfee972bcbd857d18ea626c9a9da64b2a445516946f3dd83c75a4f6b8a92fa
2023-08-10 00:58:48 +00:00
stephan 84c1a7b49f Replace the SQLTester infrastructure with a line-oriented, non-regex-heavy parser. Add --column-names command.
FossilOrigin-Name: 88863908ee2059c2d18a095cbd91f41674c7b0d0a8864ec21715a5317054df4d
2023-08-10 00:34:38 +00:00
stephan e35a703b76 Port the SQLTester 'v1' commands to the 'v2' evaluation bits. Still TODO is swapping out v1 with these separate impls.
FossilOrigin-Name: 0cf57e5b0f90779e450e9db1ca009610df5e6f4487337d49017636bde3bb02d6
2023-08-09 23:47:14 +00:00
stephan 35af4c5ff1 Correct REQUIRED_PROPERTIES handling to not fail if there are no properties.
FossilOrigin-Name: 7a19bef4f572a90fb7896b9360f9c72b052955ca9b0549be870b2b245c1f1b2b
2023-08-09 22:30:10 +00:00
stephan bef0369ff4 More for the SQLTester rework. Can read input and dispatch commands, but only --print is currently implemented.
FossilOrigin-Name: 4fa2ad33edbcef393dd98dbf90586ad8f32ec0beab02f197c8038a44be86c314
2023-08-09 22:18:22 +00:00
stephan e41c09ac8c Initial sketches of a line-by-line parser for SQLTester to overcome its compatibility shortcomings. Far from complete.
FossilOrigin-Name: 43534cd042499c1bef44ca5c4a8305a710d99e70e8b0adce6df50c6a1f0402b9
2023-08-09 19:51:39 +00:00
stephan e0fb44090b Adapt TestScript to skip REQUIRED_PROPERTIES, per [7a07863e082664da], and improve a couple of adject reasons-for-skipping messages.
FossilOrigin-Name: f937097e9b22a6c78c242cbf00c71bdc57f04b1b9a15ae24058bc2813c99688c
2023-08-09 18:25:50 +00:00
drh 7f66a3eeb2 Add an extra constraint to the test-script-interpreter.md spec.
FossilOrigin-Name: 7a07863e082664da2efcf4ecd36785d2583abbda12526cdb643cf1aa0568292e
2023-08-09 17:47:34 +00:00
stephan 16e620e07f SQLTester --result command: do not double-{}-wrap error messages and do not throw on db error.
FossilOrigin-Name: f7be20f5a62f8970f957e5c3a1d1c6536995df5c078dbac34a44f09682e43945
2023-08-09 17:23:52 +00:00
stephan 5d2a618340 Move the test command body's trim() back into the TestScript class (to simplify Command-level code) but do it after double-verbose has had the chance to emit it as-is (so that debug output is not mangled by the trim).
FossilOrigin-Name: f15ecb68f7ca129478336b462508d2c40ea052b4040facefdbc67b13e6aea99d
2023-08-09 17:04:58 +00:00
stephan 583fc5b963 Tweak the SQLTester --verbose and double-verbose output a bit for legibility.
FossilOrigin-Name: 46b79afaafda40cb1f920cc96600adf11e8c688184c9559a08eb86776ccf3663
2023-08-09 16:56:42 +00:00
stephan 589857b965 Strip --oom commands from SQLTester input since (A) we can't currently do anything with them and (B) they can appear as body content of --testcase commands and the current parser cannot deal with that. If --verbose is provided once, emit the name and args of each command as it's run. If --verbose is used twice or more, also emit the command's body text, if any.
FossilOrigin-Name: 0770e8467d4bb9490d9ed6e8a20766ffee7049cc3667db6d036c13fccbb6f3ab
2023-08-09 16:29:07 +00:00
stephan bd87174bae Update the definition of "special characters" for the SQLTester and correct the code to match it.
FossilOrigin-Name: 217f6e0c9c09c576b09ea59fce085a53d1a133927046102b4d00fd58109efc93
2023-08-09 16:03:12 +00:00
stephan 65c7226e31 Extend TestScript to be able to report why it should be skipped. Expand the test-skipping rules to account for the current spec doc. Add the {} empty-string case to the spec doc.
FossilOrigin-Name: 4fcc8cb0cc2bbc0da71bdb99dacfdec54814af4c0e4c37619bad6a8e5fa62937
2023-08-09 15:46:55 +00:00
stephan 5b400552be Add a test for the --null command.
FossilOrigin-Name: 83ac815debcc75dac1fbbdc17736f5e33fb675fdab0bf649367592a0d18074e4
2023-08-09 14:47:01 +00:00
stephan aa6b35cc80 Extend SQLTester glob support with '#'.
FossilOrigin-Name: 756ef83f45b69d9f78965ef1171d36477a32f938fe179e59b95f32f07849c0e5
2023-08-09 14:43:54 +00:00
stephan 4340f27e17 Implement the SQLTester's result escaping rules.
FossilOrigin-Name: 61bb950873a1ec45a71b15a0ab5128a50417c4ecdd7d5bd9add0c18afcbadf34
2023-08-09 14:24:22 +00:00
stephan 5158a9a9a7 Add --json and --json-block SQLTester commands.
FossilOrigin-Name: 478129d901824e675d86494044f73c313532e9f80e7ee6f425474df8237a82f5
2023-08-09 13:51:50 +00:00
stephan ff6b15fbb2 Add SQLTester --tableresult command.
FossilOrigin-Name: 8c5b6d893df4a4e82c6d8e07507fc160b11412ede4bb903ff4e3f5ffa59a9cb9
2023-08-09 13:16:10 +00:00
stephan aec9aa9289 Clean up the SQLTester output a bit by using the module name, instead of filename, where appropriate.
FossilOrigin-Name: 5323e4fd254274cc527af7536c622b786394599c68eca2da6c7fc641727dbdb2
2023-08-09 12:05:17 +00:00
stephan 283e87146c Document SQLTester's --print command and add some argument validation to it.
FossilOrigin-Name: ab9c945bb0b4210b3f47e6341f150f8a7cc45f9e4e4c2247e91d2528ed4772a6
2023-08-09 11:10:48 +00:00
stephan e6c29da777 Rework how SQLTester's Command objects are dispatched and how TestScript stores its command entries.
FossilOrigin-Name: f929f1f7f70181813f74562614f3f2aa29e65590560e3fce1677b8b176e3c6de
2023-08-09 11:05:43 +00:00
stephan 23a35c2145 Some minor tweaks in SQLTester in prep for larger refactoring.
FossilOrigin-Name: 1d93f93ac9708839e62d2f1b489adc5d47ff290c2d5aef4dd56be4e1e46c81b2
2023-08-09 09:56:37 +00:00
stephan 41f9449016 Remove the current-statement tracking from the JNI internals because it will break down in the face of client-side mixed-mode native/java code, e.g. in cases like SQLTester. This makes tracing of sqlite3_stmt a micron slower but also reliably correct.
FossilOrigin-Name: 4c0ec89dca00a9199d1e36768c034aa5eff03b13b5e015cf580f160dc4f141ad
2023-08-08 22:10:27 +00:00
stephan 78fc4ae4d9 Adapt JNI build to be buildable with or without SQLTester.
FossilOrigin-Name: adae7d78692af73e770a9cc0a4264ab32ecc18a5c0deb64f3c1e790d959bab43
2023-08-08 21:22:56 +00:00
stephan c783fd36dc Add SQLTester dup() and dup_count() UDFs. Correct arg handling of the --run command.
FossilOrigin-Name: 0dba3073f44685a51a5db7ff8886295fe04dfd43f69cbf53ad3d5afce741076b
2023-08-08 21:05:39 +00:00
stephan 4d3aa08f67 Correct --result arg count check and add infrastructure to let us add custom C-side behavior to SQLTester via an auto extension.
FossilOrigin-Name: bb8321702eea52fa9d42987a4b053b32d8eba15580a39d7831cd8d6f1ceb62bf
2023-08-08 20:41:29 +00:00
stephan c15dce9f92 If SQLTester now treats a no-args --result as comparing against an empty string.
FossilOrigin-Name: 49005ca5cc191c52279bc7fdb45d95eeb6f8e344f78ce9dbd97aac814bc21202
2023-08-08 20:15:42 +00:00
stephan af825cd50b Add glob/notglob commands to SQLTester and complete the interrupted-midway impls of the strglob() and strlike() JNI bindings.
FossilOrigin-Name: 4ba98ec0bf24c31cce498031cb3727e09f928f54ec13c76fec50e439e0f2ba15
2023-08-08 20:02:10 +00:00
stephan 746bb422ea When SQLTester hits an unknown command, emit a warning and skip the rest of that script instead of aborting the whole run, per /chat discussion. Reduce verbosity a bit.
FossilOrigin-Name: 3e78d22d04e6ac2606bfc5ce250a4c3b39a2062e14011ca0a8a0a85491efbfde
2023-08-08 19:20:12 +00:00
drh 80c438613a Minor cleanups in sqlite3Int.h. By reordering some fields in the Parse
object, it packs more tightly and uses less memory and less CPU to
initialize.

FossilOrigin-Name: aa6de539c09faa320b68c63659e602107145c4263fa680d5b40fe4d7d7ac4534
2023-08-08 17:36:03 +00:00
drh 73cdbeb632 Fix an issue in the amalgamation generator in which it was not correctly
expanding the SQLite version in the header comment.

FossilOrigin-Name: 293f6191e9b328cb8a8d3fff0f7bd5a6f5390b5ff090497a170c791b6ea22917
2023-08-08 16:53:12 +00:00
stephan bff14ecfcc Correct the spacing output of multi-select SQL blocks for SQLTester --result.
FossilOrigin-Name: 8d98645a9e524b30f7faa1cffd8f09e7aab3c25ac7b08dd6884141dfe9cdb0d3
2023-08-08 14:58:00 +00:00
drh 358c531e9b Updates to the test script interpreter spec: Rather than failing immediately
upon encountering an incompatibility, simply abandon the rest of that particular
input file.

FossilOrigin-Name: d2c99b96f4b61561c3fa34947ca7bfd2cd214b1913aff7ba64b7b897a574fea3
2023-08-08 14:49:16 +00:00
stephan 671757f4d4 A number of baby steps for SQLTester.java. It can now handle basic --result cases.
FossilOrigin-Name: 0404f688f6a22b6bbe009de1bee3341ca00e19e2cc32081265cf151876dc032f
2023-08-08 14:40:47 +00:00
drh aa29140637 Add a few words of clarification on how the SQLite Test Script Interpreter
should be initialized to start each test script.

FossilOrigin-Name: 3aa2b5a5cadb214dc64a3db412b7dfdd805abd8681b61da857b886cba3b937b5
2023-08-08 14:25:47 +00:00
stephan 5f4e94722b Correct JNI sqlite3_prepare() to emit a null stmt handle when the C counterpart succeeds but results in a NULL pointer.
FossilOrigin-Name: 94628f88b5cc82832f0ca2b00fd5346bfe99323097c6e659c5ac818c4e31d3e9
2023-08-08 13:05:12 +00:00
stephan 405dffd591 Implement the new/open/close SQLTester commands.
FossilOrigin-Name: dc823bf00f78e7cd626329220c42c46da12d565e3273a08eda5fb512c1d807c6
2023-08-08 11:46:46 +00:00
stephan ec9b37b33e Add a JNI-layer sqlite3.toString() for debugging.
FossilOrigin-Name: 456691649aa2a7672d5d110acdde92426a9d34552863db3e0c86b73d9c5d9aac
2023-08-08 11:46:26 +00:00
stephan c693bd59f7 Bind sqlite3_strlike/strglob() to JNI.
FossilOrigin-Name: eb5440f71be32812f6310756b8e30958002e8e8e41a7eb16f081058ff733b47c
2023-08-08 10:58:16 +00:00
stephan aeae7904a5 SQLTester: add print command and improve argument error reporting infrastructure.
FossilOrigin-Name: 1b6e84f6aa5c7626a308b5e8efe5c3d83ec8e7eaa803f047576b7c65333c2d44
2023-08-08 09:45:33 +00:00
stephan 58eebdaa25 Add missing license header. Minor cleanups in SQLTester.
FossilOrigin-Name: 5be50fd5887e5378f7d66405bff3a44117a826a17e5f1a18c5129d1109ecdae2
2023-08-08 00:59:40 +00:00
stephan ec8edf3295 Rework SQLTester dispatching and add stub impls for several commmands.
FossilOrigin-Name: 9e61af75ac83e74487a6ae681ee3ff891d8cf1f1d23bf895e9e3963ddf6eaf28
2023-08-08 00:37:31 +00:00
stephan e9ed375876 Add command dispatcher to SQLTester.
FossilOrigin-Name: e0a06931e91459ea43fed2954568bfafa7ad6b794fcff66e0d3bf0ed181db386
2023-08-07 23:59:08 +00:00
stephan b7f75b7bb2 SQLTester now ignores tests which contain constructs specified in the spec doc.
FossilOrigin-Name: ecaeee652aa2cc6893ded9231d7e9b2783465516016740b307b74e4e81598ae3
2023-08-07 23:04:17 +00:00
stephan fdeaee5f57 SQLTester can now split a test script into a series of individual commands.
FossilOrigin-Name: d3d1accc8b4ba0cd396ee3a58d9710a54b8e1d1b171d67595d4ef1fc7faea8cb
2023-08-07 22:32:22 +00:00
stephan 70679d135d SQLTester can now read a script and strip it of all noise content.
FossilOrigin-Name: 59bd392817ac69ffdf60ab7a2094b0d616bf593da060b6acf1b4ce9837847fcb
2023-08-07 22:02:43 +00:00
stephan 3897a882f7 Initial skeleton for adding an SQL-driven test script interpreter for the JNI bindings.
FossilOrigin-Name: 2aa8f0edecd3fc30eec28987cdbf1003ace154ddc1447b6f8715ecf38d3b06fb
2023-08-07 21:04:13 +00:00
dan 786a9dcf5d Enhance sqlite3_vtab_nochange() so that it works with "UPDATE ... FROM ..." statements. Use this to allow some updates on fts5 contentless-delete tables.
FossilOrigin-Name: 27ff86e4d8d251dbbcc9f0682d3d7b040518cbeee891cfe253661d1fdbec4e4f
2023-08-07 17:09:25 +00:00
dan be3ab76bfb Changes so that sqlite3_vtab_nochange() works with "UPDATE ... FROM...". Use this to allow UPDATE on a contentless fts5 table if new values are supplied for all indexed columns.
FossilOrigin-Name: 16cd2161e312cf97129011fc829079db8f762b822b2f4fabf7ff6742c071302f
2023-08-07 16:15:56 +00:00
stephan c7f7b45a15 When converting a Java exception to a db error message, use Throwable.toString() instead of getMessage() so that the exception type's name is included. More internal API renaming for consistency.
FossilOrigin-Name: 2d44720d06d9e50cb037e92981d2473a3ad0b7560f2f5923d428f59de6fd6aaa
2023-08-07 11:18:44 +00:00
stephan be7aef1f85 Lots of JNI internal API renaming, for consistency, and moving-around of utility functions. Make it safe for more callback types to throw.
FossilOrigin-Name: 9a494394b9eb28cf88dc5e7075a4b8c682c8e14fdd6837b595bec8011d7e9e72
2023-08-07 10:59:27 +00:00
stephan becf29a36c JNI API renaming to better match the C API.
FossilOrigin-Name: 6e0bd03d0ba9ee8422853241ba1c4e963d158d1f042855c0cb0026701907896e
2023-08-07 10:01:59 +00:00
stephan 7c26811c2f Rename fts5_api pContext parameters to pUserData, per /chat discussion. This is a cosmetic change made to reduce confusion between those parameters and the two other context-type parameters in that API.
FossilOrigin-Name: 2ca064d8eb37252e16b0fec9924e9ba9289d96a737346431c6ba9cb1c161e5de
2023-08-07 09:44:00 +00:00
stephan 5d48fb1784 Make sqlite3_stmt() Java ctor private - it's only constructed from JNI code.
FossilOrigin-Name: ce82c42f151e38b23945e6f5dd99cb6a77b3c6440508f41abc35e9f6c29cd440
2023-08-07 01:06:27 +00:00
stephan 005baf67d5 Minor internal cleanups and additional test metrics.
FossilOrigin-Name: fa0a6b6e8e6c711585bca30357e465f7a2f08a1c7159ecf23031af1e5158b89d
2023-08-07 00:29:38 +00:00
stephan 8af781fc4d Bind the auto-extension APIs to JNI.
FossilOrigin-Name: 746a5fa079ad80b3c59411202ee601e0b5c50e79e5994d5e464fa06d3c276324
2023-08-07 00:06:31 +00:00
stephan 60745265e1 Rework the sqlite3_open(_v2)() order of operations so that pending auto-extension support can get ahold of the open-time Java state despite the Java/C (sqlite3*) binding not having yet been established.
FossilOrigin-Name: 34da294ab558880e81eebd7d261bc590551d5a7d2855e844695cef6394647ea7
2023-08-06 22:09:09 +00:00
stephan d85f9bf6d5 Completely rework how the JNI sqlite3_open(_v2) and sqlite3_prepare(_vN)() bindings deal with output pointers to give the JNI side full control over the origin of db and stmt handles (necessary for solving chicken/egg situations in auto-extensions and prepare-time trace). Lots of adjacent internal API renaming.
FossilOrigin-Name: 644999caff9db79562d45520d94aaa24ee88c65e397b6fb9c20a4f0e7f84e1a5
2023-08-06 21:29:13 +00:00
stephan af90dcf324 Start adding JNI-side support for auto extensions, but hit a brick wall which requires slightly awkward semantics changes in the JNI bindings for sqlite3_open(_v2)() to resolve, so stash this #if'd out for the time being.
FossilOrigin-Name: 77a32d238e80fe1d237768d88780043a7bd2b3543e6672536254782cbea0039c
2023-08-06 20:01:30 +00:00
stephan 1a2563a950 Doc cleanups.
FossilOrigin-Name: 5f56b007704f2aad4cbc6f0ccd1e1f1c974865971f99451352714ee7e077c284
2023-08-06 15:01:38 +00:00
stephan 5558624b43 Bind sqlite3_config() to JNI for call variants taking (int,int*) and (int,const char *) variadic arguments.
FossilOrigin-Name: 6119289da85ac0c83e2a7236d24bbfff22334d6cf1d852756dc658ad6a75dfec
2023-08-06 14:19:09 +00:00
stephan 9c4c16c80d JNI-internal cleanups and API renaming. Add a C-side java-string-to-utf8 conversion.
FossilOrigin-Name: 672d85795d04131135b1dc6a02d64eceb8b5084217c17766afeca4af23c07ec4
2023-08-06 13:02:43 +00:00
stephan 6a9364a518 Remove doc outdated warning about sqlite3_trace_v2() JNI binding being incompatible with MUTF-8. Use new to-string capability to simplify Fts5ExtensionApi::xColumnText() JNI binding.
FossilOrigin-Name: ebcfc2379be12f76a96f3605b734f406b3354d4c985062cdbfca0cf7e3f31379
2023-08-06 11:05:17 +00:00
stephan 57645b67fa Add a way to convert from standard UTF-8 to a Java string (JNI lacks this capability).
FossilOrigin-Name: 07dd082c9e371829a18aeb574f842891e545e1fc125760238ede7e7e2b6a4262
2023-08-06 10:49:47 +00:00
stephan 1bce6b468e Bind sqlite3_db_filename() and (closely related) (A) add many more docs about the UTF-8/MUTF-8 discrepancy (B) start adding internals to enable us to perform the standard-UTF-8-to-Java conversion from C.
FossilOrigin-Name: 586720fa714ac74491cd85d0c6645242e55e5989ad312ef6e15e0b0acc6906ff
2023-08-06 10:14:53 +00:00
stephan 153288dc89 Bind sqlite3_set_authorizer() to JNI.
FossilOrigin-Name: e0fa03135942cd2fe732a74510d380ba78ab230c452168e638f32b4aee04b3f7
2023-08-05 22:41:36 +00:00
stephan 28e95830ad Refactor the per-JNIEnv cache from a fixed-size static array to a linked list of dynamically-allocated entries. Uncache all per-db state (which is necessarily JNIEnv-specific) when the corresponding JNIEnv is uncached.
FossilOrigin-Name: 9dd8b78419e19e88bc3fbff9bf200390b146b2461af2bb6b93d8467036619e33
2023-08-05 21:35:58 +00:00
stephan 67214a9594 Add SQLite3Jni.uncacheJniEnv(), a way for Java threads to clear their thread-specific cached state from the JNI bindings when they're about to terminate (or are otherwise done using the library).
FossilOrigin-Name: 7468f8761bece58f7ced3d112bbe2fb454432d9c54c9b96cedb5a15bc2926d0f
2023-08-05 20:19:45 +00:00
stephan 27cb7d6a1d In the JNI docs, note that the sizeof SQLITE_TRANSIENT and SQLITE_STATIC should ideally be the same as the platform's pointer size.
FossilOrigin-Name: 7d4ac44ec419ed0474bdb9d237b97660cf0d8faba8fe686f6a914d7bc04dfa3b
2023-08-05 19:20:15 +00:00
stephan 47c5db7f77 Tiny doc typo fixes from [forum:31eb707250|forum post 31eb707250]. No code changes.
FossilOrigin-Name: 8ede50082e7bcf0226a3b42a590f188b5a139bbf081c9eed7ff8f6f2a6a274a0
2023-08-05 19:15:19 +00:00
stephan fdeb2c8121 JNI internal refactoring and cleanups.
FossilOrigin-Name: 7397973a2f191d75b149cf73a6d7ee1798820c1cd37d83af14e565067ede1b04
2023-08-05 12:48:33 +00:00
stephan 695d0b698a Bind Fts5ExtensionApi::xUserData() to JNI and extend xCreateFunction() to accept that argument. In test code, use assert() instead of exceptions if assert() is enabled so that test failures (exceptions) thrown via callbacks do not get suppressed (which they otherwise necessarily are to avoid crashing the host app).
FossilOrigin-Name: e43837377696e468cd31cf71585fe235ffe67a9f4d3b036c5f9d0cb7141d0f57
2023-08-05 11:16:54 +00:00
stephan 4f07be39b7 Add missing Fts5PhraseIter.java to the build.
FossilOrigin-Name: 96281ad0d5b2f020622c4f85f8694886e6a29fb43e1fbeb2a346ed2e94f109fb
2023-08-05 04:30:31 +00:00
stephan ec71e555c4 Bind fts5_api::xCreateFunction() to JNI and demonstrate it with a test.
FossilOrigin-Name: c653bf16cbdccae05ab14059b140191afd5c17740fb78d756d8822986e54b17c
2023-08-05 04:23:27 +00:00
stephan 977b6919f2 Start adding fts5_api to JNI.
FossilOrigin-Name: 14d18fe983c83412d72fd2005a45a2b8c48d347b7bbf8ef9630ae460cff85c32
2023-08-05 01:28:30 +00:00
stephan 0c179dd5b4 Bind the remaining Fts5ExtensionApi methods to JNI, noting that all are completely untested because the higher-level bits needed to do so are still missing.
FossilOrigin-Name: 23383c1dfd240ce47f504dd5c3402c9a31f166fbde5bb72d91309a5655074b33
2023-08-05 00:40:28 +00:00
stephan 195611d8e6 In the opfs-sahpool VFS's importDb() and exportFile() methods, throw if the actually-wrote/read amounts differ from the expected-to-write/read amounts, per feedback in [forum:a4122e986f|forum post a4122e986f].
FossilOrigin-Name: a617ebf4e5d1af1b5b15e9782ad111399caaa3ea7b99bb0c8691c8b4283b6d6e
2023-08-04 16:01:55 +00:00
stephan cc8202b646 More work on the JNI binding of fts5 customization (still a long ways to go).
FossilOrigin-Name: 1a246fd21657f5bb13eeacc4059894ab787ea9a3c45bd9bdd3030a66643d2fef
2023-08-04 15:38:59 +00:00
drh d0c8358982 Merge the json_remove() bug fix from trunk into the bedrock branch.
FossilOrigin-Name: a8872aa588dd1367997924ea65267af6bd85e076e00f40fe150e3f33ef6d0dda
2023-08-04 13:29:22 +00:00
stephan c7795cfd47 Eliminate code duplication in the two JNI tester classes.
FossilOrigin-Name: 63e7bbe3d5fcfb531f9d7fa88398c1191570e69b5d11adcb9c5e64b8345b4e6c
2023-08-04 13:27:45 +00:00
drh 7331bc0002 Merge the json_remove() bug fix from trunk into the wal2 branch.
FossilOrigin-Name: e6c066cae4ed5d698820c0c1ea8840773146e9d1a665551d8d51bb9df121cdb7
2023-08-04 13:24:40 +00:00
drh b87278f414 Fix a bug in the jsonLookup() routine used to search JSON. Bug was
introduced by the recent (unreleased) JSON caching enhancements
check-in [a4c1af616e672a0d] merged into trunk at [df099ad713011b67].

FossilOrigin-Name: ba7a66db13b28da0b41f03be825a593608474a5ebda89e12d58d89db709b8f65
2023-08-04 13:19:11 +00:00
stephan 5e208f712f FTS-related JNI refactoring. Move FTS-specific tests into their own class and dynamically load it, if possible, from the main test app.
FossilOrigin-Name: b7a8428fcd969e7a29a23c2dae61883f69501094f2de0f79bbee3c02c672cbf5
2023-08-04 13:03:31 +00:00
stephan b15223bce6 More work towards binding FTS5 customization to JNI. Add Fts*.java files missing from previous checkin.
FossilOrigin-Name: 91263178f463ca4623dd0203696eff6bcfd68abde5d2471be3f5a3edd791c52a
2023-08-04 12:44:06 +00:00
dan b383744835 Change comment on Fts5ExtensionApi.iVersion from "always set to 3" to "always set to 2".
FossilOrigin-Name: b7de399c277394a6a3bbf253d4177aa82753271e5087b98b2c2268f629a7dfdd
2023-08-04 11:10:10 +00:00
stephan c0952c11a7 Start including fts5 customization bits into JNI, but it's far from functional.
FossilOrigin-Name: abaf5edd0430e3301a11bd0acb9ce4b81b310237e1799701411db56ef7605e01
2023-08-04 11:08:25 +00:00
stephan e133a0ec05 Improve internal error handling in the JNI create_function() impl.
FossilOrigin-Name: 2c88390faa108a60c8fb1eb7aad05d90f3daf4cfef14ca73987597aaf7be83c9
2023-08-04 09:53:13 +00:00
stephan a24769454e Resolve the timing/ordering issue of a JS-to-WASM-converted xDestroy() function being uninstalled from WASM right before the underlying native call tries to call it. This has been a long-unnoticed bug which appears only when removing such functions or replacing them.
FossilOrigin-Name: 031c9a76b6ad1572e7a88f4d2d62f206b0d37bd1170e2c8a24248c5ec628f2f5
2023-08-04 08:45:25 +00:00
stephan 444424dab8 Part 2 of the fix from [a0f808363318c00fd1db78b].
FossilOrigin-Name: 5c3104228d75c8fafbe24e46623777f3a8647f1b50267af61d46f7fad0e0434e
2023-08-04 08:41:55 +00:00
drh afdcd07fae Merge the latest trunk enhancements into the bedrock branch.
FossilOrigin-Name: 877ffece0531054f24410c9e4c0b89599a9506689cc97233d33c18b9a15e6308
2023-08-03 23:26:39 +00:00
drh 6f5c4810bc Merge the latest trunk enhancements into the wal2 branch.
FossilOrigin-Name: 6fa71e11197111ab52d1983b35190899a5881dcb9ed822488c92bc839c72cb55
2023-08-03 23:22:28 +00:00
drh 1050f1d2a1 Fix harmless scan-build warnings.
FossilOrigin-Name: 65a8716d8a1b7c5cffe9cdd25e4cbfa6528fcb146ff0fe67b0ce771b0537473d
2023-08-03 18:14:09 +00:00
stephan e98fa205f8 Fix a discrepancy between sqlite3_values_to_js() and its API docs/intended behavior.
FossilOrigin-Name: 7b85c68fe252c0aa69724c82a95a4432e33ebdfb879b4b1b2ee2f9b98e9b792a
2023-08-03 16:22:59 +00:00
drh 86d77fd95d Fix build problems associated with SQLITE_OMIT_WSD and SQLITE_OMIT_TRIGGER.
FossilOrigin-Name: de031a5ec863b11f0216d1a114740c8cf4572adb034a5d35ff1a5c07d81df800
2023-08-03 13:30:00 +00:00
drh 038ac625af Fix a test case whose value changes under the no_optimization permutation
due to the recent change to add the ability for SQLITE_TESTCTRL_OPTIMIZATION
to disable one-pass.

FossilOrigin-Name: 976e39c6322f9e51054546e2d9bc135b683d141054c645c9098002cdc310c7c7
2023-08-03 13:07:58 +00:00
drh 9b1334b7d3 Unix builds now assume the presence of nanosleep() in the standard library.
The -DHAVE_NANOSLEEP=0 compile-time option can be used to build on systems
(if any still exist) where this is not the case.

FossilOrigin-Name: 779d5dc8797ea246d0397f7e94b1be716b0baa735e8d9f5a6fc4cffd887a7420
2023-08-03 12:41:30 +00:00
stephan ed86e0a0a7 More JNI-internal docs. Correct handling of a Java exception if a trace callback throws.
FossilOrigin-Name: 306b269a01037bc5c98276276fdb17b37027d1ee0d603183f42a65966245bdff
2023-08-02 17:29:18 +00:00
stephan decc1425a3 Internal doc additions in the JNI bits.
FossilOrigin-Name: b56643644e065b03fe2c3545e02ed3b2f6fd9f77c39aa84a02a00d695e9a12dc
2023-08-02 17:20:52 +00:00
drh c874d60353 Performance optimization for JSON rendering logic.
FossilOrigin-Name: ea0b9aecbaca9a8e784fd2bcb50f78cbdcf4c5cfb45a7700bb222e4cc104c644
2023-08-02 16:06:02 +00:00
drh d3229ce23e Remove an unreachable branch in the ascii-to-floating-point conversion that
was added by [e989a37ff9d5b52e].

FossilOrigin-Name: c4347e4400e96f932ac12f8f22484a2ebce2a578d1b2181977954c432f117bfd
2023-08-02 13:45:16 +00:00
drh b1dcebfea3 Stricter enforcement of the idea that a MATERIALIZED common table expression
is an optimization fence.

FossilOrigin-Name: 354425f8d97437bd156265a6914f98cce91b42abc9e773453ef4d817e308dc9a
2023-08-02 13:38:02 +00:00
drh 26cf4af5d7 Remove a condition from query flattening that is now taken care of by
the caller.  Factor out the reverse_unordered_selects processing from the
main loop of sqlite3WhereBegin() for performance.

FossilOrigin-Name: f068f105fb158634321bf6401f0774c81059932d213a18b627ae98bcffc10912
2023-08-02 13:29:01 +00:00
drh aa250db66a Simplify the select.c logic that enforces the optimization fence around
a MATERIALIZED common table expression.  Do not allow the
reverse_unordered_selects setting to affect a fenced CTE that itself has
an ORDER BY clause.

FossilOrigin-Name: 165b9bda243777ae7fb6f51ee7ad58771c581b82ef9a70268eb9764e9e6eba6f
2023-08-02 12:39:56 +00:00
drh 5e82c06cf8 Never flatten a CTE that is labeled MATERIALIZED. This is really something
of a bug fix because the documentation says that a MATERIALIZED CTE is an
optimization barrier.

FossilOrigin-Name: b7ef9796f548ce43e06673f86bbec68157aae5e4ee8451e1d87ee5f70af1bb27
2023-08-02 11:06:27 +00:00
drh 4178849736 When a query is inside an AS MATERIALIZED CTE, do not attempt the
omit-ORDER-BY optimization.  If the developer specifies MATERIALIZED,
that means he wants the sort to actually happen.

FossilOrigin-Name: aa769ee7477171f26a091cadff98aa4f48da7752ade20b405a95c4525456f62e
2023-08-02 00:09:15 +00:00
drh cbaef88980 Avoid ASAN warnings when converting over-sized long double values into double.
FossilOrigin-Name: e989a37ff9d5b52e0090d59be077ad2260c8df5d4c2c2d8088b1160de64dffd4
2023-08-01 19:10:30 +00:00
stephan 2877c43e6c Filter the wasmfs-specific JS module result type check out of non-wasmfs builds.
FossilOrigin-Name: ef7c121049f77200ecc4da260a90776264af0e84122d0133b4986959aedc653a
2023-08-01 16:41:12 +00:00
stephan 887f6e5082 Fix part 2 of 2 for /fiddle: a make deps problem which caused fiddle to load but fail to run when sqlite3.c was missing.
FossilOrigin-Name: 6011bc26764760af750de68f1553f45382766dd2195633ae267bdb4f9ac75d0f
2023-08-01 16:38:08 +00:00
stephan a08d363f67 Fix part 1 of 2 for /fiddle not running: build flags which were renamed during refactoring.
FossilOrigin-Name: 214ffeeff77997c3352d30aaa6e51e2dea9706f0d08f73f4eacfad1864817fc6
2023-08-01 16:17:00 +00:00
dan c1a07d3bab Fix an assert() in fts5_index.c that might fail when dealing with corrupt records.
FossilOrigin-Name: aa55c83f35c2ab134e0842201e46e021079283f9c65595c86664060b3aa8d715
2023-08-01 11:03:06 +00:00
stephan ed77e13f05 More docs for the Java side of the JNI bindings.
FossilOrigin-Name: d8e9bcee96b90d56701f7907a8bd48853211caf757e1aa8decc7ed25eece6770
2023-08-01 10:19:05 +00:00
stephan e209d144c3 Internal exception-handling cleanups in the JNI bindings.
FossilOrigin-Name: 057b1d4f6ffff73c120566895f61ea3fd8118968464d67ec9262096a7aa03f39
2023-08-01 09:44:26 +00:00
stephan 0bc72d2b91 Minor JNI-related doc tweaks and code re-ordering.
FossilOrigin-Name: b663b27e425966f34fb62482a18048f0e2934380e5c411ae3627f1fe6a765c04
2023-08-01 08:48:13 +00:00
drh b9894f88b8 Remove redundant comparison operator.
FossilOrigin-Name: 018984329892c243b21adb10aa8efc7f49313ecf80477e6b61dc6365f62c489d
2023-08-01 01:07:45 +00:00
drh d192b4441d Restore part of the UPDATE one-pass optimization that was removed by
check-in [2c56b984a0bd3be5]: only disable one-pass if the WHERE clause
contains a subquery.  Allow subqueries in the SET expressions.
Fix for performance problem reported by
[forum:/forumpost/8ab195fd44e75ed0|forum  post 8ab195fd44e75ed0].

FossilOrigin-Name: 42916af9fc0f379a608a08db894400bd735a28e26ab1ffd604d1fddfbdb3ec0c
2023-08-01 00:34:03 +00:00
drh 00d635dda4 Restrict the new column cache to table-btree, which is the common case anyhow.
That way, writes to indexes do not need to clear the column cache.

FossilOrigin-Name: 659284ab0e22a4746c1337b3489e7b166d497fb7e5301e24dc115d2b0c4e097d
2023-07-31 22:03:24 +00:00
drh 98ab970bac The original column-cache implementation from check-in [ab1edcc7fedcf279]
(merged to trunk at [771fe35074b50b8d]) is unsound.  This check-in fixes
the issue.  Had to give back a little performance, the optimization is still
a overall win.

FossilOrigin-Name: ec95e970fb737adf0fab3cb4363040b036949e5eb966fc2d030a20f95e2bde60
2023-07-31 20:02:11 +00:00
drh 1f097a2f29 Add bit to sqlite3_test_control(SQLITE_TESTCTRL_OPTIMIZATIONS) that will
disable the one-pass optimizating in DELETE and UPDATE.

FossilOrigin-Name: 22f6dd5cd32ed9e7b932fdba98aa8652fde999df79f4995d3d8ac1acf430cf53
2023-07-31 17:39:36 +00:00
drh c8c6763782 Improved debug tracking of the Parse.nQueryLoop value.
FossilOrigin-Name: 4a8a17498ecaccca5e5aab65d4596cbca9799547f50683237d08c1fa3556f1cf
2023-07-31 17:20:12 +00:00
stephan db6f0bef91 Experimentally change the JNI sqlite3_trace_v2() callback type to have more convenient access to the current Java-side sqlite3_stmt at the cost of some uncomfortably fiddly current-statement tracking in the JNI layer. Subject to change.
FossilOrigin-Name: 459db332af6ea358b42bac096b9d26f1045b9ec32fad8463bca06807b2396b2c
2023-07-31 13:52:46 +00:00
stephan e3e9bdcd0b Add some JNI-internal metrics, accessible via passing -v when running Tester1.java. Document an OpenJDK bug which leads to incorrect -Xlint:jni warnings.
FossilOrigin-Name: a5d68a6b64abe3c2dfc3a32157f70fd8a4ad89feef2510b3bbb2d86b325d51ae
2023-07-31 12:10:32 +00:00
stephan 8f714e21a0 Correct a mis-cast in the JNI wrapper which just happened to accidentally work. Update JNI binding of sqlite3_context_db_handle() to return the bound-at-open() db instance instead of a new/temp proxy object.
FossilOrigin-Name: 9faca5d9ed4a749421e08bd1da8b7672c0fd31366124fdb613c46e19dece0fc1
2023-07-31 10:55:30 +00:00
stephan ede8900912 Add another test for JNI-bound scalar UDFs.
FossilOrigin-Name: 8b322d92e247be606f83977767dc361ee4f7bc819122a630bdaa1110177db9b8
2023-07-31 10:42:05 +00:00
stephan e2e696d11f Update some internal docs for the past two checkins. Add a way to dump out some debug info about the current JNI environment.
FossilOrigin-Name: ac9b8bb1e64450d980e2986084996549ae5c59e68c9f0c4c69539c239b64468b
2023-07-31 10:22:34 +00:00
stephan 452108b4fc Refactor the busy-handler-specific JNI hook type to use the generic hook type.
FossilOrigin-Name: d9efdc6dd20a34bfdaad5d4bf8e67cce7e35238299eb91e4459d59fda11978a6
2023-07-31 10:08:36 +00:00
stephan 46c46e400a Refactor the collation-specific JNI hook type to use the generic hook type.
FossilOrigin-Name: 02c1d3b6501fedf3d6e6d1ca60699df268522182c5ba3b49ae8f4691499ef0fc
2023-07-31 09:45:49 +00:00
stephan 41a7ac637a Internal JNI doc tweaks.
FossilOrigin-Name: f4aa2c82882cb6be1fd52977de19fd03c2e38abb857b520f951b32d610972ab6
2023-07-31 09:28:15 +00:00
stephan 9c1c6da930 Consolidate triplicated xDestroy()-calling code. Remove some unnecessary casts.
FossilOrigin-Name: 24c0763d0e025187c74002ffee11fd48d3cd7b40e01469d28484bb67f701884b
2023-07-31 07:15:25 +00:00
stephan 09947d0aad Internal JNI API renaming.
FossilOrigin-Name: fcfc070673cef2f657f4737f096678439ed7c011fb2e5391e0721f82f5d8af51
2023-07-30 18:41:25 +00:00
stephan 91710673e7 Internal JNI refactoring towards consolidating support for callbacks with and without finalizers.
FossilOrigin-Name: 120983a570d6de055cef9d916096de3410897ea9f46d23ea6eff1f9b549e423a
2023-07-30 17:24:01 +00:00
stephan 72bb29283f Mark the Java wrapper classes of the C-native types as 'final' (cannot be subclassed).
FossilOrigin-Name: 4fd3d93623d67c25fb8a490e0d4ea56d531d858067011ab1b28cce694098feff
2023-07-30 13:47:51 +00:00
stephan 13b059025f Minor tweaks and optimizations in the JNI bindings.
FossilOrigin-Name: 41fb5898f1a78d9fd85a020e28a6048a7359b54e35632e9072917cbdbcd8b07d
2023-07-30 13:30:52 +00:00
stephan 502a5c2e26 Bind sqlite3_collation_needed() to JNI. Related adjacent cleanups and fixes.
FossilOrigin-Name: 16ff167691733350907d2d995c774a885214acd0fe8ec491c16b786f00fe85d4
2023-07-30 11:36:41 +00:00
stephan 45fe10d02b Internal JNI refacoring to support the pending sqlite3_collation_needed() callback. Correct a bug in the linked-list handling of PerDbStateJni which triggered an assert().
FossilOrigin-Name: 7ac6614e69b03304d09745619ed83f12c7eb775aaf4a636a79289b01642ddd14
2023-07-30 10:47:38 +00:00
stephan 9c113744d1 Incremental checkin to minimize the diff while narrowing in on an assertion caused by refactoring.
FossilOrigin-Name: 2d7a91b1396d87852f1153ab7af7385514a9537cb64ba3bbd0faba2d28704214
2023-07-30 09:45:54 +00:00
stephan 70576de5e0 Consolidate some warnings for java callbacks which must not throw but do.
FossilOrigin-Name: 5e592ed2dfc89225fff3a1c76509adc799a238282413984e0c4b32af18525d18
2023-07-30 08:12:15 +00:00
stephan fecad503a9 Bind sqlite3_update_hook() to JNI.
FossilOrigin-Name: a5bbaa9017839f8d8b92bfb44472d4c60fa3037bfae7846dc8350262c1332cde
2023-07-30 07:44:03 +00:00
stephan b162c68d62 Bind sqlite3_rollback_hook() to JNI.
FossilOrigin-Name: 5f8ee44098d74ac2b98e4dd43ad80d3b919528358b3f992b425af7fa6262dcee
2023-07-30 06:44:21 +00:00
stephan 00c0c5883a Add commit hook failure to JNI tests.
FossilOrigin-Name: be4459b495cc2555e4d8ca24dd56c3da1036d25af7cc077bbc3d93c2a3e8c40b
2023-07-30 06:00:53 +00:00
stephan f52de0fdaf Bind sqlite3_commit_hook() to JNI.
FossilOrigin-Name: c687297fcae082cbd7f9258c43f3841fd34904d8b62b0adf3cd61fcddeee483d
2023-07-30 05:50:34 +00:00
stephan ca379859c9 Bind sqlite3_progress_handler() to JNI. Add some metrics to Tester1.java.
FossilOrigin-Name: 437ecfe8abf8d294d429d191d811da6148e0b2ebb74cf66998480bfc8ef58bdf
2023-07-30 04:31:56 +00:00
stephan f0c701c410 Add tests for SAHPoolUtil.exportFile() and importDb().
FossilOrigin-Name: 4aa88b8fb1c99137ed7531e5c82a241738a2b0554f7af82c0ed0ce8d16a0879a
2023-07-30 03:54:10 +00:00
drh b47e1c7866 Do not try to add MEM_Term to an MEM_Static string either.
FossilOrigin-Name: 8920dcef1f75f578e3f274ab2b980a83d6ea6b761d99fc48410b26886170597b
2023-07-29 22:58:47 +00:00
drh e809d84618 Do not try to apply MEM_Term to a MEM_Ephem strings.
FossilOrigin-Name: 2a378a66783ce4d749f7e1e7d08d8404a4aa78d47aeeefd6794b6753188438f6
2023-07-29 21:22:26 +00:00
dan 808cf29b55 Fix a buffer overread in the fts5_structure virtual table (test code).
FossilOrigin-Name: b837aff79cd159061b46af59eaf96a1a1920eeece27e9e27931cf3387068d96a
2023-07-29 20:13:19 +00:00
drh fb546c0bf4 Simplification to the sqlite3_stmt_explain() implementation.
FossilOrigin-Name: dd5eb4c7f71c658aff8c2f11ff163c5e79f0ebfd49fd13ca8e28381fe63cc82f
2023-07-29 17:05:35 +00:00
stephan 5d800cdb6e SAHPoolUtil.importDb() now accepts either a byte array or ArrayBuffer.
FossilOrigin-Name: 3848f04e542e2f0f8975e82632af598aac3b60312bd244b0472f7ddf1dda77df
2023-07-29 15:53:53 +00:00
drh 0c6b869263 Enhancements to the new sqlite3_stmt_explain() interface implementation and
documentation.

FossilOrigin-Name: dc98ac6b6de27fc096a715e650067154a545709cf1eecd6c2722ab30a91c2d1d
2023-07-29 15:31:48 +00:00
larrybr 104e225074 Fix doc error noted in [forum:/forumpost/11fc7308d5e607f2|the forum] and a misspelling.
FossilOrigin-Name: 6b79cc7dd07d41da6034386b39c0dff32f80a977db42a9602330ffd15706ab9b
2023-07-28 21:38:50 +00:00
drh 3e053c50b4 Do not allow the same JSON parse to be input to both arguments to the
json_patch() routine.

FossilOrigin-Name: 8e781c6742792b3899ca60efc49c7eaee1df9f6b03d27c254b0e148b82946e19
2023-07-28 20:59:25 +00:00
drh 9a897cb0de Merge the latest trunk enhancements into the wal-shm-exceptions branch.
FossilOrigin-Name: c44fb4d06591ef4872f015aa9e81ce8b93a38bd9143cf20db38cbe5c77a0d71f
2023-07-28 19:07:25 +00:00
drh f715af11ba Merge the latest trunk enhancements into the bedrock branch.
FossilOrigin-Name: 1d6ded3234bbfa3226ef718eec8d0ce06ae2844628866cb6197383c9bf06267c
2023-07-28 18:52:09 +00:00
drh b9b790e8cc Merge the latest trunk enhancements into the wal2 branch.
FossilOrigin-Name: c1fa1cb5e0c420614481636f3ce2dd9ad2142274bb7b37d39e5c26228317f992
2023-07-28 18:47:49 +00:00
stephan 33c8e9d233 Internal JNI binding docs.
FossilOrigin-Name: 991c66197e4dc7297fce3b20a1b4846873bcd4ce8add36aac71bd2e0e73c207b
2023-07-28 18:44:11 +00:00
drh 58b5921ca4 Add the sqlite3_stmt_explain() API.
FossilOrigin-Name: 0443c0ef85f8f6f3efdee68bf59126c9459f08a58458756f8cbeeaf043d43a86
2023-07-28 18:37:13 +00:00
stephan 888e14f551 Fix an incorrect file path in ext/jni/README.md
FossilOrigin-Name: bcefa2df563260933c7ab5df90872580f71010c11419f6b1de7b1e2747237ff8
2023-07-28 18:02:02 +00:00
drh 58bf53d573 Minor tweaks to the JSON performance measurement documentation.
FossilOrigin-Name: 0bed957e46aa3bf6a70292ae100de0459486c1469dd03de61207a708cc59a594
2023-07-28 16:12:39 +00:00
drh f69084aaf7 Fix minor typos in the JNI README.md file.
FossilOrigin-Name: e21cf2e8f13465dbff33f9c21580752c1d8c077d6a253f56f04dab0d47eb99b8
2023-07-28 15:58:09 +00:00
drh 8a056d486f Enhancements to the JSON performance testing scripts and instructions to
cover the recent optimizations involving updates to large JSON strings
that indexed.

FossilOrigin-Name: f9213289d09adfb0461d9644e80c7e889f4bae51563ae2e575a2c95336052bcb
2023-07-28 14:20:31 +00:00
drh 859dd4fc2f JSON performance improvements: (1) Add a cache for large string values
on OP_Column to avoid having to extract the string multiple times.
(2) Internal JSON caching improvements.

FossilOrigin-Name: 771fe35074b50b8d4a6583e61c53871b2445f7a58c82f3fc3bf6776e562e63af
2023-07-28 13:52:08 +00:00
stephan ab12b750de Correct a recently-introduced falsehood in ext/wasm/README.md regarding ssh port forwarding.
FossilOrigin-Name: 2a3f3a9cf28849c99d83c256f813405a2f47f84532e1192d67bb7aa655dd98a3
2023-07-28 11:52:58 +00:00
drh b1926192fa Mark an unreachable branch as ALWAYS().
FossilOrigin-Name: 8fd06d1151c521515d9f7181575a04e451b522335135a27248052b067d4005ae
2023-07-28 11:30:07 +00:00
stephan c7f602497b Minor java cleanups recommended by a modern IDE.
FossilOrigin-Name: f92a9bbabebc33a771835e259b04a57d07add1eacf3d9aec5fddf929ce8a3022
2023-07-28 10:37:17 +00:00
stephan 069029011e Add Java tests which ensure that xFinal() is called with no aggregate context when the result set is empty.
FossilOrigin-Name: 7ac8c66a6b62d6bb345e8b9957a26de463019ae7d30ff442f19482c3a6827fc7
2023-07-28 10:01:01 +00:00
stephan f326577619 Tweaks to the Java-side UDF docs.
FossilOrigin-Name: 48e0079d8db675a8761f27348bc6795aa6b6e49a3f76bd4adece83309dcf2aff
2023-07-28 09:54:39 +00:00
stephan 09c2640fe3 More Java docs about making use of the aggregate context. Change the JNI mapping to set the sqlite3_context::aggregateContext member directly, instead of via a superflous setter, because that way is faster.
FossilOrigin-Name: 7af0cb998f7161296d5e5e50a42e9db26ec13c145c61194a999a1a0104818d45
2023-07-28 09:25:05 +00:00
stephan 75d3b1b5a2 More docs and cleanups related to the aggregate UDF state. Correct the OOM check to behave properly if xFinal() is called without a matching xStep(), xValue(), or xInverse().
FossilOrigin-Name: ff53f1ccdc1780f2d9bd5f59804a76dbdf4f6b70696d3a7dbdbd96d1f8f6fa5c
2023-07-28 01:51:14 +00:00
stephan 7d207bf483 Add an OOM check to the previous check-in. Minor internal API renaming.
FossilOrigin-Name: 6b56e4d62b4945e52978d00aa8e2984faa731c92a7e002e81524fcfcf8ba0cce
2023-07-28 01:19:44 +00:00
stephan 48a8352a39 Add support making use of sqlite3_aggregate_context() (in a roundabout way) from Java to accumulate state within aggregate and window UDFs.
FossilOrigin-Name: 640574984741c7a9472d7f8be7bce87e736d7947ce673ae4a25008d74238ad90
2023-07-28 01:12:47 +00:00
drh 93269a9e41 Remove some unnecessary and incorrect code that was mistakenly added in the
previous check-in.

FossilOrigin-Name: d3f458dcc379dd7648262f52579ec55ba931852065ed278367a0629228d0ed45
2023-07-28 00:54:38 +00:00
drh caf7e26c4f Switch to using jsonParseCached() for json_patch().
FossilOrigin-Name: 2ed9c59e6a51037e63485d0a92dae25443116beddfca1ac73d2f5d5be38d4ad3
2023-07-27 23:51:36 +00:00
stephan 8ba5d79c35 Reformulate jni tests to not require the -ea jvm flag to enable assert().
FossilOrigin-Name: dc356667a8f4fa31a3fef1ae35873d834d27fd6a9f0818d6fb85e4751fde9fe5
2023-07-27 22:53:02 +00:00
stephan fd022a23a5 Add locking test for BusyHandler and clear up new -Xlint:jni warnings introduced by the busy-handler binding.
FossilOrigin-Name: 7dcde2bfce54b18f391776fa1cb93c0ff6153634bedcab0007b374c06c4d4079
2023-07-27 22:42:25 +00:00
stephan 1a95091b49 Bind sqlite3_busy_handler(). Correct mapping of pointers for, and cleanup of, JNI-level per-db state.
FossilOrigin-Name: 524747796a30a5c1c6c7567b49ffb1e35e2626c73e09c335c0ab74d4ddb5f005
2023-07-27 22:05:39 +00:00
stephan fd250c2cb5 Replace some www: interwiki references in the JNI readme with their full URLs to make the doc more portable.
FossilOrigin-Name: 63ce0c9bdde210cf2f8b6099ae5c73caac18e6debc13c2f77090b77f3de72beb
2023-07-27 20:32:16 +00:00
drh 6bc4baf313 If the input JSON to a json function that uses cache comes from an RCStr
value, then use that RCStr value in the parse rather than making a copy.

FossilOrigin-Name: 509ae9c1470dd79d320e84371e1e6662fb85fa0571df5ed8c4d946d10cdfe821
2023-07-27 20:28:29 +00:00
stephan eead0ffef8 Merge trunk into jni branch for compiler warning fixes.
FossilOrigin-Name: 0514fd340ae15a95760d50c747d6fb9eae5109cb5045eeabc2bc199be0a5ae35
2023-07-27 20:12:48 +00:00
dan ea6bccaa44 Fix a couple of compiler warnings in fts5_index.c.
FossilOrigin-Name: bf71faa2a1d29ea762c4d2485522d6f4f8a5a7166981a92d3ba9c96ccbbe1213
2023-07-27 20:08:44 +00:00
stephan dff3443939 Initial check-in of JNI (Java Native Interface) bindings for the core C API.
FossilOrigin-Name: b5374b9ef58fa0be80aefccde0721f5599fb820464b13940b6361b9aa09a59d5
2023-07-27 20:02:49 +00:00
drh 594f5e24c4 The OP_Column opcode caches large column values coming from overflow pages.
FossilOrigin-Name: ab1edcc7fedcf27922d5db4bc1bc673b1495ca9c66eb6debdda7b7776c068888
2023-07-27 19:39:53 +00:00
dan 9cf8961557 Add the contentless_delete=1 option to fts5. For creating contentless tables that support DELETE and REPLACE statements.
FossilOrigin-Name: d66b182d2bc6ce0772e69401b7affe1adbc1b128c4631cb3c17f98dde72af00a
2023-07-27 19:13:35 +00:00
drh 95daf355a7 Merge trunk enhancements into the json-opt branch.
FossilOrigin-Name: 5739a16ad270a5aadcbb46b28c34fa6ba975422788dcbccb1a8e0d1e6ed75144
2023-07-27 18:19:46 +00:00
stephan 9fcca3d27c Dynamically determine whether the wasm.xWrap() argc check can be applied, depending on how the wasm environment exposes its exports.
FossilOrigin-Name: fd59226b34fffb1479fb2d7bd7c0aff982aa4a1a73e6c0d81de6eaf9c075998c
2023-07-27 17:50:10 +00:00
stephan 4cfd54256d Accommodate a breaking change in emcc 3.1.44.
FossilOrigin-Name: 2c5dd34199f5bcf729be814b8b46d9997821fe3a39ab12779c93df1bb2fd108d
2023-07-27 01:38:19 +00:00
drh 93853a4846 Fix a performance regression in JSON associated with generating small
snippets of JSON from a larger JSON string.

FossilOrigin-Name: 837f2907e10b026f6db1ca2d44b4bf60a6f069bf534bf369ad9b5c513cb0c6e4
2023-07-27 00:21:59 +00:00
drh e94e132994 Reduce the number of memory allocations when parsing JSON.
FossilOrigin-Name: 9edd67162113df57dae21d4683f9495611e2cf4717c6d12f5b7b8e44156d5fe3
2023-07-26 23:22:32 +00:00
drh f7a164f345 Minor changes to make coverage testing easier.
FossilOrigin-Name: ec8b43382e5402e15d9f2dda3cf21ac8be8c1589ddbe6c9433c33eef0036f764
2023-07-26 21:53:09 +00:00
drh 3f3dc2f5d8 Change a switch() case to default for coverage.
FossilOrigin-Name: 04f497074b9210326030f36107a43d6490a2a59c8a574e2c5429cd9bde681bf7
2023-07-26 19:22:43 +00:00
drh beb3fb6904 Make sure jsonReplaceNode() always leaves the JsonParse in a consistent state even if an error is encountered.
FossilOrigin-Name: 01d52232dd6fbd253e77419a17df3df83d49434792d288ef96e14739a89cef3b
2023-07-26 19:11:47 +00:00
dan 5e22dfb6a5 Add a comment describing the contents of the ExceptionInformation[] array for win32 EXCEPTION_IN_PAGE_ERROR exceptions. No changes to code.
FossilOrigin-Name: 8e20354242d5f34a90b6c00d9034535ef6f6086ee4971230c7fd69985e2777d2
2023-07-26 18:34:34 +00:00
dan 5028dd89ab Add the ".scanstats vm" command to the shell tool. For profiling VM code in SQLITE_ENABLE_STMT_SCANSTATUS builds.
FossilOrigin-Name: 0cbec3990d4101142bfb831f8e6527b73baabebbd30fa7f59275b217dbce6a8d
2023-07-26 17:51:05 +00:00
dan a1016b4275 Improve the output when ".scanstats vm" is enabled.
FossilOrigin-Name: 7df08fd35e9d4bc471aa9fbc4c81d2ebcfd2be6c4c38143342b3d9d727c9df22
2023-07-26 16:41:23 +00:00
drh 9095063e14 Improved caching and reuse of JSON parses.
FossilOrigin-Name: df099ad713011b67b09cb1e5f0fa2e6b45f9cee0ce9d3c118c5dbca3563d20a5
2023-07-26 14:18:06 +00:00
drh 7286c59896 Change the debugging "json_parse(X)" function so that it shows a more
complete description of the JsonParse object on standard output and returns
the mimified JSON.  Former behavior was to return the text of a decode
of the aNode array.

FossilOrigin-Name: 1bf85d4e388714a88f8940dcdec353c3e0267456697eff6963d34637912aecc9
2023-07-26 13:17:43 +00:00
drh 2b57197066 More comment improvements in json.c. Do not run jsonLookup() following
an OOM error.

FossilOrigin-Name: cd5fda8c2e354da7458b7c1a82ff18c5946f8dab16095bb0293bec57f6804f17
2023-07-26 11:53:14 +00:00
drh 7725370ff4 Improved comments and other cleanup for the changes on this branch.
FossilOrigin-Name: bac953a80d1a541e7a12aef00c86c002133859237143ad670b39ea19799a8900
2023-07-26 11:43:39 +00:00
stephan 8c7ee455ff Remove the batch SQL runner from the JS build's 'all' target, as it's long-since unused and adds noticable build time. Add makefile comments about JSPI.
FossilOrigin-Name: b8f708e35d4fa027d12089ac7c5589c36da5f68b98cf9792bb11276bf233a860
2023-07-26 11:41:41 +00:00
stephan aef4fbf203 Add SAHPoolUtil.getFileNames() method, and tests for it, per [forum:a3da1e34d8|forum feedback]. Add a test to demonstrate that two SAH pools can coexist so long as they have different names.
FossilOrigin-Name: 72dc3f8c3255186ec412412b685b0b51ddcd08240f2353ac742fc7da8c23568e
2023-07-26 11:11:39 +00:00
drh 9a184daad3 Fix jsonForceRCStr() to also add the NULL terminator.
FossilOrigin-Name: 134b01f37f8f741d7f7b7eda81384695d1cbe4c39751d87f08832d5c9afdcef2
2023-07-26 11:00:47 +00:00
stephan 3ba6914121 Reformulate [907dfc4a7aa1] using awk instead of sed for better cross-platform portability.
FossilOrigin-Name: 82ff7cc6a4b0331677be87bc069da414a56fd531bae402d0f0808b5d2b0d45da
2023-07-26 07:57:55 +00:00
drh a757026c5e Merge the latest trunk fixes into the json-opt branch.
FossilOrigin-Name: ef4e1664d159b98854d9aa580b0bb942f1726f32a190e2ea659c171131ddce9a
2023-07-26 01:15:34 +00:00
drh 02e683e467 Add the concept of an RCStr string - a "Reference Counted String". Use
these to keep track of big JSON strings that might need to be used in
multiple places at the same time with different lifetimes.  Add two
different JSON string values to each parse - one with and one without
changes.  The net result is a large reduction in the number of parses
that occur when doing an UPDATE of a indexed JSON field using
json_replace() or similar.

FossilOrigin-Name: a4c1af616e672a0d4d04f2652e645617758231598fb7161b956883512303ae87
2023-07-26 01:12:08 +00:00
drh 44f53b9647 Extend the enhancement to json_set() and json_replace(). Clean up cruft.
FossilOrigin-Name: 2dbb22c75e86f2e3ced38ac14b4943570d5c2f86cd5e37e875bf0c863be28836
2023-07-26 01:05:08 +00:00
drh 0f200bc580 Add the JsonParse.zAlt field to old revised JSON text after a change.
Demonstrate that this elminates the need for reparsing after a change
by using it in the json_remove() function.  This is an incremental check-in
containing lots of cruft.

FossilOrigin-Name: f930b139d6db0ee799bc90397b225175103c4bf22923d1c9cbcd32509adc1738
2023-07-26 00:48:45 +00:00
drh 59b8e666f6 Clarify ownership of the various objects involved in parsing JSON.
FossilOrigin-Name: afe02a398a16d51bd7482b6fbe2fbd15d9ac4fd9cdbc9d2bf81f38b3391fc567
2023-07-25 20:26:47 +00:00
drh 440e696d5e Incremental improvements to JSON parsing - trying to fold in the RCStr object.
FossilOrigin-Name: 4cb15d934a85ebc290fe6dd8cd3bd47b159561ca75d72bbffef30b9ea4623b09
2023-07-25 18:28:03 +00:00
dan 0c39b13e46 Fix a harmless "set-but-not-used" compiler warning in sqlite3session.c.
FossilOrigin-Name: 54b3c43fdfdaca6b129a5f0ee93c34eb001663775d33c087066650f5e164d1c1
2023-07-25 17:54:25 +00:00
dan cbf4a0fa8e Merge trunk changes into this branch.
FossilOrigin-Name: 719973d7f5a47b110e9919fcb96d21feab1e41356dbb3ec674c1116c17bbb778
2023-07-25 17:11:29 +00:00
dan 5326953e57 Add extra test for 'deletemerge'.
FossilOrigin-Name: bc33cff4203cef12518e0f43d380a06d53d67c725fb96cfe6e934b7dc97a7efd
2023-07-25 16:48:54 +00:00
dan 3f874b58fb Change the name of the fts5 'delete-automerge' option to 'deletemerge'. And add tests for it.
FossilOrigin-Name: 1079300db2a7d1fbc86a01c215c234a3af64889c5396e6da63ff4f3c7efae4c5
2023-07-25 15:48:58 +00:00
drh 4bca124845 Fix a minor problem with error reporting in JSON.
FossilOrigin-Name: c456e4a8999066cd96246327101b3cca78294511a71a2ac07939bb702bfcb5f4
2023-07-25 15:43:01 +00:00
drh f02cc9a324 Create the new RCStr class of strings and try to use them for JSON storage.
FossilOrigin-Name: c1b8725089bb3d006ec69add28f4fcb3f4e79412c7f438b5b1067c2227e77b9c
2023-07-25 15:08:18 +00:00
dan 039d494d5f Further tests for 'delete-automerge'.
FossilOrigin-Name: ca26c7a37a7e680be633f43be28f8877bdf9917448ea51c3bedc9b2352a00601
2023-07-25 13:53:42 +00:00
stephan 322a325703 Reformulate a (sed -i) construct in ext/wasm/GNUmakefile to account for Mac's sed -i being different than GNU's.
FossilOrigin-Name: 907dfc4a7aa129cdcedeb3ba2d75e1b68a8f22c2545ee1c8cf7d705041644e5c
2023-07-25 12:26:05 +00:00
drh b715fe9d80 It is an error to try to insert a BLOB value into JSON.
FossilOrigin-Name: 28c6e964b1e2257527df02b352f02e135f7c4b764b8b41eda6461f9538f5d042
2023-07-24 23:27:05 +00:00
drh 3d2a559ecb Clean up some #defines in json.
FossilOrigin-Name: 327fff501e36f75d4901c520123c5ca45e0e0da1d9cc8fa8fa877ceb68c686d2
2023-07-24 22:45:59 +00:00
drh 27553579c0 Rework the JSON parse structure to facilitate better caching. Passes all
tests.

FossilOrigin-Name: ecdcb1ded76e9a0591bf7a2009679f49fc3aa639d3cc12406c6d29243ed8e1c5
2023-07-24 22:34:26 +00:00
dan 24730de8d1 Add the fts5 'delete-automerge' integer option. A level is eligible for auto-merging if it has a greater than or equal percentage of its entries deleted by tombstones than the 'delete-automerge' option. Default value is 10.
FossilOrigin-Name: b314be66b9ac0190b5373b3b6baec012382bc588c2d86c2edab796669a4303c3
2023-07-24 19:13:06 +00:00
drh 9125d5ab6c Incremental progress toward improved caching of parsed JSON.
FossilOrigin-Name: f2c063884685a79d5a787590447c292f51e898a98c9508159c788f505227ba85
2023-07-24 17:59:25 +00:00
stephan c76a8862a7 Resolve an ES6 module export construct which is incompatible with node.js, as reported in [forum:b9680fa9ad|forum post b9680fa9ad].
FossilOrigin-Name: 80927c3913561dddf75cf73be871d93ae06b16f83e8cc36fc360765014209615
2023-07-24 15:41:58 +00:00
drh 1671e05696 Merge the latest trunk enhancements into the bedrock branch.
FossilOrigin-Name: a6f344399472e211941dffa6586a83fc1740c60e807e85387080a88c9c54658d
2023-07-24 13:24:40 +00:00
drh ecd48447b2 Merge the latest trunk enhancements into the wal2 branch.
FossilOrigin-Name: b2e56f62ad8f114126fda6b8c1266d194bb1356b8a302956c5e9ad90d8c1d73b
2023-07-24 13:20:39 +00:00
drh 5202b7ca75 Update the latest trunk enhancements into the wal-shm-exceptions branch.
FossilOrigin-Name: 3187ee3f69fc28a259ba0e951ac10a65c07ef2c3866acbefaf9544333a930cc6
2023-07-24 12:59:53 +00:00
drh 7d70571142 Merge the latest trunk enhancements into the json-opt branch.
FossilOrigin-Name: 00bfc4918be49ac74a3e7851c88ab7ec226e6a37853f8ad4c77f758751960456
2023-07-24 12:37:23 +00:00
stephan 479cfd5af3 When writing the filename header in a opfs-sahpool file, ensure that all remaining bytes in that part of the header are zeroed out to avoid the downstream problems reported in [forum:d50ec48a293988a5|forum post d50ec48a293988a5].
FossilOrigin-Name: c1b080e39397c983c13a5e79303223827de7b4946c18a79396851ec1814782f3
2023-07-23 22:14:41 +00:00
stephan 2ecadd8869 Add the opfs-sahpool sqlite3_vfs implementation to JS, offering an alternative to the other OPFS VFS (with tradeoffs).
FossilOrigin-Name: d2e602cda44bf35e76167143262b4f91826d25780d0e095e680a31d5dedb2018
2023-07-22 19:57:42 +00:00
dan 2159292ce0 Integrate contentless delete with auto-merge.
FossilOrigin-Name: 85c1589ab1fc69d1eef4bbc1bdefa2b10af5f6b9c08e813130b93829b592f416
2023-07-22 19:47:46 +00:00
drh 5d03b1610f Do not read past the end of a text buffer looking for a zero terminator, as
that space might not be initialized.  If the buffer is owned, just set the
null terminator.  This is a better fix for the OSSFuzz-detected
use-of-initialized-value problem.

FossilOrigin-Name: 931bccb0cc290b8bf3027641e7a7fac30e3244d7dc84aa9e38b24b7e9544ca06
2023-07-22 16:37:28 +00:00
drh 61a5b6e3bc Fix a harmless use-of-initialized-value warning from OSSFuzz that results
from recent large string optmizations.

FossilOrigin-Name: 1a6b3dd1c40277a0d0f0bb562ddc4868aadd632fc2d29be1b17bb33fc22c46c8
2023-07-22 15:21:41 +00:00
drh b2ee272075 Performance enhancements for queries using very large string values.
FossilOrigin-Name: 3661b9cd98565ea23b0cafd030668f09885dcbd54eb12b7b3cd85c289aef4c3c
2023-07-21 22:28:34 +00:00
drh 21f22f9088 Mark an unreachable branch as NEVER().
FossilOrigin-Name: 414a4d660c3b505640227066b468db1268fcb1bd3fad931644a769617c6c00cd
2023-07-21 22:22:13 +00:00
dan 330e36c2c6 Fix fts5 incremental optimization so that it too can handle an index that consists of a single segment with one or more tombstone hash pages.
FossilOrigin-Name: e61c9b083f5e0b6b6ee18f9394581ad816f445dbfb72ed1fe954f4182755a576
2023-07-21 21:10:33 +00:00
dan 4e0c157d02 Ensure the fts5 'optimize' command correctly rewrites any index that consists of a single segment and one or more tombstone hash pages.
FossilOrigin-Name: f4926006b371d9a1439a25384bd50a50c2f1c03f75a7c2c3134ae72abb971c91
2023-07-21 19:33:35 +00:00
drh 89f07b1276 Fix harmless compiler warning introduced by the previous check-in.
FossilOrigin-Name: 06f829e9e57c5aa495f519ad7bb379be611e8e21ef60d36e63d055e45f7a6117
2023-07-21 19:17:26 +00:00
drh 028acd974f Avoid a potentially large strlen() in sqlite3DecOrHexToI64().
FossilOrigin-Name: 5413b02bb629b9cbb76f7e688e94ebcf24276c01436d3feb73ff57c036e1d2aa
2023-07-21 18:38:59 +00:00
drh 569700a72e Further improvements to large string handling in relation to JSON.
FossilOrigin-Name: 1e5df0aa3dae5cadbf1d07c718ae2a5212543300b68e49d35e8c96855a7f619c
2023-07-21 18:09:07 +00:00
drh eee8687a9f Fix harmless compiler warning about signed/unsigned comparison.
FossilOrigin-Name: 75cc3c89ee2dcfefa9421ce60bee77e85d2895918e8c5cfd05c434f8932a99b5
2023-07-21 15:07:56 +00:00
drh 7a2280fe65 Multiple optimizations that try to preserve or infer the zero-terminated
property of TEXT values.  Avoid unnecessary copying of text values destined
to become function parameters.  All changes help improve performance of
doing UPDATEs on large JSON values that are indexed multiple ways.

FossilOrigin-Name: d0278cdedfa04fb0b61838ab9622be8a2c462f58d5c3ebc4c5f802a727d0974e
2023-07-21 15:01:53 +00:00
drh c8f4fec2de More infrastructure changes towards improving JSON cache performance.
Incremental check-in.

FossilOrigin-Name: 1955e66cfc4614df97b8d68b0e662f309513d62dc8aeec71af5a54e66b79c707
2023-07-21 11:09:53 +00:00
stephan bfe6dd0100 Minor internal cleanups in the opfs-sahpool VFS.
FossilOrigin-Name: 74ad31e2908af8225b7aa527dbcd1877423d58163e365317a78453b31e322ea3
2023-07-21 10:51:35 +00:00
stephan 3068d6622e Internal cleanups in the async part of the JS library bootstrap phase.
FossilOrigin-Name: b6d57ab63793241a500ea527c5b3216c54b3ff1972d3adbbf42a9a53bfec0aa1
2023-07-21 09:10:42 +00:00
stephan 35325a37ce Minor internal cleanups in JS code.
FossilOrigin-Name: 21a2ca9fc46bf746874579897872e2a45cb07f278abb670dd22b122f7d6a9a6c
2023-07-20 23:25:32 +00:00
dan 3a51f8c307 Fix some divide-by-zero errors that could occur when handling corrupt tombstone hash records.
FossilOrigin-Name: 7567ca0676f0d45026f5cd4f3fbcd09119c2eaab8ec1711499609c16c452b5e4
2023-07-20 20:29:56 +00:00
dan d6f5aa824e Avoid an infinite loop that could be entered when dealing with corrupt fts5 tombstone hash pages.
FossilOrigin-Name: 69ce2ce035279f2a00c2238187cf4d2a9092c3410f5900e4613fe4e46311169e
2023-07-20 20:09:26 +00:00
drh 51d7869096 Experimental framework upon which to build a better JSON parse structure that
supports cached of modified JSON.  All of these changes are tentative and
subject to change or removal.  Incremental check-in.

FossilOrigin-Name: e384163a4763c10b5838cbf28f9c4d8ea799bbbed6e890e8aa91b2bd1458646d
2023-07-20 17:45:09 +00:00
dan 0b3791b537 Add tests and fixes for the new code on this branch.
FossilOrigin-Name: 5aac50e92e956b15367c75c20c17bc1c75e84e2752bfffe4ad0a266cb9bd3b8a
2023-07-20 16:07:04 +00:00
stephan 0b518b8d65 Filter the OPFS VFSes out of the sqlite3-node.mjs build. Add another level of subdirectory to the sahpool to later enable transparent support of client-provided files under the VFS's root dir. Rework the awkward sahpool-via-oo1 mapping.
FossilOrigin-Name: 080a4d0aba30d8f3802b49be4a113205f069b3bdea8cebf525d654055642ff62
2023-07-20 09:06:42 +00:00
drh f9bfc32b73 Performance enhancements to the JSON parser and code generator.
FossilOrigin-Name: 0a745897d66e9f36ea6e787fad54f0c413d52c226dc96fad36ccd33a0aa1930e
2023-07-19 23:02:45 +00:00
drh 93b3c1f398 Revise the new JSON parser performance test to make the test run 25 times
longer, and thus provide a more repeatable number.

FossilOrigin-Name: bee9e403ae47103938aabb9a7a7e120dfa741b464875965e58788a51fa56a8fe
2023-07-19 20:23:34 +00:00
drh 5e4d77fa88 Describe JSON parser performance testing procedure in test/json/README.md
FossilOrigin-Name: c47056e158073ee746c233dd2729cf224ea9e38821b15aa71ebf97607e602281
2023-07-19 19:23:30 +00:00
dan d1fbaa071b Fix various issues with code added to this branch.
FossilOrigin-Name: 8d09011fa2c6ae9cc88e1766f9aad4578efbf9e0e311b8c6efdffe7a3f88f923
2023-07-19 18:47:02 +00:00
stephan 55f318e53f More internal refactoring and docs for opfs-sahpool.
FossilOrigin-Name: 64ccf6177a019eab46fb3345ad1e8ba80eaf2c9da55767031f9f04ccd16afb4d
2023-07-19 17:47:02 +00:00
stephan 96cb7007a9 Update the development-over-ssh docs for the wasm build.
FossilOrigin-Name: 500109bd0a4c134b91c37f397ff1ee828e09c17f7ecd153f975ede748caee7bb
2023-07-19 17:46:28 +00:00
drh 50834d8b74 Further improvement to JSON parser performance.
FossilOrigin-Name: 144c8ccf6e5bb2527dd98742f0d67e0a16c627e7c67f754ce8ed4c4fb5b8d8b6
2023-07-19 17:24:36 +00:00
drh 8376ae7295 Further performance enhancements to JSON parsing and rendering. Total
performance gain for large JSONs so far on this branch is about 11%.

FossilOrigin-Name: adb4d6b007cbe9d7c9670f5fc196443ebe0f3a89df1f3290ba6247fcf83fe5bd
2023-07-19 15:06:29 +00:00
drh 7a816e771a Performance optimization for parsing large JSONs that contain lots of text.
FossilOrigin-Name: c9fbe0185cd5d64950724b00cd0bfb3a7939a985040465a0f35f445acb6e94a6
2023-07-19 13:50:31 +00:00
stephan eafb4418a0 Major restructuring of opfs-sahpool to improve maintainability and hopefully make it easier to reuse these pieces in the upcoming JSPI-based VFS experiment.
FossilOrigin-Name: 534481cd0c2e6f62dd0a82f25d4b78fdcc671eb70d6966693c98212a6420891c
2023-07-19 13:31:29 +00:00
stephan cd302a5e32 Appearance and legibility improvements for tester1.js.
FossilOrigin-Name: 4fd3ed1b6405f690825c9993e56b199d4a7fb497cf9131815c803150b8b96528
2023-07-19 11:33:52 +00:00
drh 47079b19df Merge the latest trunk enhancements into the sqlite3_stmt_explain branch.
FossilOrigin-Name: 06eae046c1a1155f22590b88942db72f01482f2363da9753f8adfdb322679055
2023-07-19 10:17:42 +00:00
drh 37fd50df1f Two minor improvements to sum(), one of which was inspired by
[forum:/forumpost/af5be98dbc|forum post af5be98dbc].

FossilOrigin-Name: a0d3e7571aded8d1e03908059d2d5aa5d62ec49bff099cb38f6f35df5e4b18b5
2023-07-19 09:52:10 +00:00
stephan 780bc4c557 Internal refactoring in opfs-sahpool. Move OPFS tests to the end of tester1.c-cpp.js.
FossilOrigin-Name: 6bd5a7413dd830ca41b587a2826fb599a2196fb0186646a2333500f950b3cf4d
2023-07-19 08:18:25 +00:00
drh 9246c85b61 Do not use the [/info/609fbb94b8f01d67|viewscan optimization] on a query
that has only a single loop, as the cost adjustments can cause problems
for outer queries.  Proposed fix for the performance regression reported by
[forum:/forumpost/64d36440e473516c|forum post 64d36440e473516c].

FossilOrigin-Name: 76152ad2ffe56034f2fd93d9a1ce9358e1677a7e9cd3dcd9f3a34a5c956a463e
2023-07-18 21:06:19 +00:00
drh 6a9be2cbc7 Improved ".wheretrace" for the VIEWSCAN optimization.
FossilOrigin-Name: 27057ea76b5f72c73fb6f16094736685019643c665b49fd0bb8d60a812ce2338
2023-07-18 20:41:09 +00:00
dan d05bf0fe61 Fix various problems with fts5 contentless_delete=1 tables.
FossilOrigin-Name: 0d005112b8aca9e9eca9d86d5fed9168f6a0218fd290b5489b9e7b05714610f4
2023-07-18 19:52:32 +00:00
stephan aed5abcc0f More opfs-sahpool tests.
FossilOrigin-Name: 60713fa9c4627ef17e0b8778eee37913d2b930c5a06414721a00af30e1395090
2023-07-18 19:47:19 +00:00
stephan 0649a1a05d Fix a shadowed var in opfs-sahpool and add more tests.
FossilOrigin-Name: 9c8a73aff0f291e0c18072372e0d8961d3a05910489598d0d342227d99871954
2023-07-18 19:27:11 +00:00
stephan b0dd9d427f Further refactoring of opfs-sahpool and start integrating it into tester1.c-pp.js.
FossilOrigin-Name: 91c789234963b660ae900f0738906b28a477993709e286d8125b2f4d6101601c
2023-07-18 18:52:41 +00:00
drh 3f47e7551c Merge the FTS5 memory leak fix from trunk.
FossilOrigin-Name: fb65cb73d7ea22a8b20dccfa3abdaaa809eee4fcee6fe4846bd2e598ceb49aa4
2023-07-18 17:43:47 +00:00
drh d1db37a2f3 Enhance the sqlite3_stmt_explain() interface so that avoids unnecessary
reprepare operations.

FossilOrigin-Name: 050f773addd605f6690348c98e9000a9a3663bbc26288a71973fd7b40468e8ab
2023-07-18 17:29:05 +00:00
dan a5b1f34125 Fix memory leak in fts5 introduced by [def41225].
FossilOrigin-Name: 4dcad2db743fdb9ef72871ca5a4d1384f76cb697161b0f5110e2670a83a18e8a
2023-07-18 16:29:34 +00:00
stephan d69e6e6efb Major restructuring of the opfs-sahpool bits to better support multiple instances per app (each sandboxed from each other). Eliminate the pesky promise resolution race condition when two such instances are loaded in parallel.
FossilOrigin-Name: 95e5fa498f71708caeb3394636c4853530a8b2d54406e503f32750732d6815d5
2023-07-18 16:24:51 +00:00
dan cb1d6b997e Merge trunk changes with this branch.
FossilOrigin-Name: 5f66eb4e2603278dcc9dbfe4bf506cba1aa03180cfb492a0dfc3a8be32cc994b
2023-07-18 13:55:55 +00:00
dan 263a8e8c0a Fix a buffer overrun in fts5 that could occur in 'secure-delete' mode when dealing with corrupt database records.
FossilOrigin-Name: def41225b2f10b0294ab506f28bc87873688d4030f7056839298ff18e058ece5
2023-07-18 13:55:19 +00:00
stephan dec4cea24c Refactor opfs-sahpool to support multiple instances, each with a separate VFS name and directory.
FossilOrigin-Name: d036eaf6ac60c576428db40f015733c5d5425f7d613194fd8d9d4d98659077c4
2023-07-18 12:09:16 +00:00
dan 2cccced140 Fix cases where a row is inserted into a contentless_delete=1 fts5 table and then deleted within the same transaction.
FossilOrigin-Name: d928856a226fb7f001e55ff7e8eb58a656b982f1efa811de46c382b8b7cd778c
2023-07-17 18:40:39 +00:00
dan 55e0fd4a9d Do not allow the 'delete' command to be used on contentless_delete=1 fts5 tables.
FossilOrigin-Name: cc694b83408ccb5d42204cb624145c76e95329cbe1d1fe8815c70a7a00af231a
2023-07-17 17:59:58 +00:00
dan 24433bea19 Merge trunk changes into this branch.
FossilOrigin-Name: c4fb2f2ea0afe638fd7cffd89fbdb0a91589577c6f8299c7bbc17ac121be518b
2023-07-17 11:47:42 +00:00
stephan 99d4a2db46 Another -DSQLITE_TEMP_STORE=3 to 2 change for the wasm build.
FossilOrigin-Name: 5ad8f9987c11b5db67edd69a3471ff388140d6268d5d3f5a05dec6bb6d92ac05
2023-07-17 07:43:04 +00:00
stephan ed640943c4 sahpool vfs: rename defaultCapacity to initialCapacity, per feedback.
FossilOrigin-Name: 2c9c95d087c7db2a975ffae15af5ade621411c493145b889f0c9157a55c4952a
2023-07-17 07:14:12 +00:00
stephan 2bf2ff605f Change wasm build's -DSQLITE_TEMP_STORE=3 to -DSQLITE_TEMP_STORE=2, primarily so that clients can optionally shift temp files from memory to OPFS storage.
FossilOrigin-Name: 764430e804332b8f7b2f63a3c0c8acf8f6cc92534858d89de2d310938c1c0d27
2023-07-17 07:13:44 +00:00
stephan 039081123c Remove an extraneous JS script import.
FossilOrigin-Name: 5a205b25912f3feea594a2161a4b8f3955bd29163b39cb0e621b3abdb81fc24d
2023-07-16 21:07:01 +00:00
stephan 8d67d501e3 Add sqlite3-vfs-opfs-sahpool.js to ext/wasm/api/README.md.
FossilOrigin-Name: 58d05201042804c682a417495f13ab47a535e353da6cc307f9d726a289e71ca2
2023-07-16 20:42:19 +00:00
stephan 1ff2903785 Minor doc tweaks.
FossilOrigin-Name: 0cb0d7a9e73e48406f146ebebf211311fd0a0b14b285f0af9e2b645c3d2f172a
2023-07-16 20:20:03 +00:00
stephan d703d25a35 Correct a missing assignment. Minor doc additions.
FossilOrigin-Name: 1c4957d0ef23ff14d4f7bfb33a809dd92712ee9faf77b6052f823eb55de15cf6
2023-07-16 19:29:50 +00:00
stephan a34f27daed Doc tweaks only, no code changes.
FossilOrigin-Name: 044c28dffd45f7c4484686995edd4a1b92151450743968e7d0f662b5c850aa6b
2023-07-16 19:20:45 +00:00
stephan 24873723f4 Add a way to remove the opfs-sahpool's persistent state from OPFS or unlink() an individual file. Doc cleanups.
FossilOrigin-Name: 80982daac3c098033dbc249bb7a17ef84ae218d2d789f8644e7f4af18b553d24
2023-07-16 17:51:43 +00:00
stephan da6a42a921 Move SAH pool configuration options from the library-level config to a config passed to the VFS install routine. Extend and document the PoolUtil object.
FossilOrigin-Name: d2ed99556fa1f40994c1c6bd90d1d5733bebc824b1ebfabe978fae9e18948437
2023-07-16 16:52:09 +00:00
stephan d62c464541 speedtest1.js: only install opfs-sahpool if it's provided via --vfs flag, to avoid locking errors in concurrent speedtest1 tabs with other VFSes. Add opfs-sahpool reserveMinimumCapacity().
FossilOrigin-Name: aa94c8abfbdfc4c7b36554c4b3ea90a5065e7e3f4294c64c8cbf688b4688300d
2023-07-16 14:07:59 +00:00
stephan aa1b2a80ee During "opfs" VFS init, check for URL param opfs-disable and, if set, do not install the VFS. Added per forum suggestion to provide a way to help rule out misinteraction between the "opfs" and "opfs-sahpool" VFSes.
FossilOrigin-Name: 29905b7a75b73e32125bf9116033cae7235a135b668a3b783a3d8dcb0bc80374
2023-07-16 12:02:37 +00:00
stephan 8449dacbd4 Micro-optimizations in opfs-sahpool.
FossilOrigin-Name: 52f23db948ae9694df69c00177b85cb569e9b211350a4a2dbf249e7cd8de700c
2023-07-16 11:49:18 +00:00
stephan d0ae50411f Redefine what the opfs-sahpool installation promise resolves to. Fix addCapacity(). Add utility methods to import/export files.
FossilOrigin-Name: 809c6f4de3653ad7a7751af45a7a0d6cb20c3ee3be80c69833c729242227d970
2023-07-16 10:02:41 +00:00
stephan 28d46cce0b Minor cleanups in the opfs-sahpool VFS.
FossilOrigin-Name: 279e09070918dab7b60c39179ebb7eb931ca6bd4e589b414f436740499a2f910
2023-07-15 21:08:48 +00:00
stephan 38d1db9b79 Correct opfs-sahpool VFS after the pebkac involving the previous speedtest1 runs. Make that VFS explicitly opt-in to avoid certain unfortunate locking situations.
FossilOrigin-Name: 41bf1fe31f2f3d0daa2bac25dc57262a4b90f22fed6fa97e4e92467c32ae02dc
2023-07-15 19:08:58 +00:00
dan 0dac350f3c Store large tombstone hash tables in multiple database records. Ensure the same hash tables handle rowid 0.
FossilOrigin-Name: 4410e60d0c76e057ee962124f9239c6e17fd5ccafdbb4d9b703448eabd7781e3
2023-07-15 18:57:57 +00:00
drh e393f6eff0 Add the experimental sqlite3_stmt_explain(S,E) interface.
FossilOrigin-Name: 5683743ddf0bb051f2fe5d137cc18407e000e77e9faa9010a22e3134b575638b
2023-07-15 16:48:14 +00:00
stephan eadabc6513 speedtest1 JS: only add --memdb flag by default if no --vfs is provided.
FossilOrigin-Name: 676ffe6280c1ce787b04d0cdb4a0664229c6125c601af4b18d1bfa125aac3675
2023-07-15 16:30:46 +00:00
stephan bb65feb869 Rename opfs-sahpool removeCapacity() to reduceCapacity().
FossilOrigin-Name: fff68e9f25a57045e9d636b02ffa073cf1b984b2587d4fce10f6e35c9988469c
2023-07-15 11:37:34 +00:00
stephan 88af76f62d opfs-sahpool VFS now seems to work, in that it runs fine (and blazingly fast) in speedtest1. Add sqlite3.config options for the high-level configurable parts of opfs-sahpool.
FossilOrigin-Name: 5d92d5f4d8ac4cfa37ba473e5cc861628b783bbf1ae4d138bcae8b9d6cc6e798
2023-07-15 11:23:57 +00:00
stephan 100bc4429a More work on porting the sahpool bits. Revert to random backing-store names because it works better with the capacity-reduction algorithm.
FossilOrigin-Name: b4e005fd38b06b8d2f2317b955b93807e80a6a18db5f06d7747978d3bfa11411
2023-07-15 01:02:38 +00:00
stephan bee3213145 Switch opfs-sahpool to use deterministic backing-store file names. Delay VFS registration until after the pool's files are all opened. Fail vfs init if the client's OPFS API has an async FileSystemSyncAccessHandle.close() method (indicating that it's outdated).
FossilOrigin-Name: 7c6697ededee9a64962ac6fd78934c6d6e39258b9558a03c1a6c02bf3be1759e
2023-07-14 23:02:58 +00:00
stephan 92bf6fd4f6 Integrate the opfs-sahpool VFS into the JS build and get it loading (but it's still far from functional).
FossilOrigin-Name: ef96e6b586825a2b3ed011174309cba8ce1031876c86dc59ed87ab9bbc64f57f
2023-07-14 21:48:45 +00:00
stephan 50c61825fc Initial sketches for an alternate OPFS VFS which uses a pool of pre-opened SyncAccessHandles to bypass the need for a dedicated I/O worker and the COOP/COEP HTTP response headers. Currently completely non-functional.
FossilOrigin-Name: a93de9f2a553a3a4edd1b361dd6f465a1b0b5b51f7bb8ede432067aedcfefda4
2023-07-14 21:17:29 +00:00
stephan a02d6d8237 Minor internal cleanups in the OPFS VFS.
FossilOrigin-Name: 984d491eb3fe06f714bf07d6873321f3992a072812b46508e599bfefd39dff3e
2023-07-14 21:06:00 +00:00
stephan 8ba619b5f5 Minor wasm build cleanups.
FossilOrigin-Name: 816b503f093c4e6d92d0eb2f9fbd841acd01cc9bc89ee58d961b56c64f71406a
2023-07-14 12:21:05 +00:00
stephan f4449dc0ba Reintroduce WASMFS+OPFS support, albeit as a separate (less portable) WASM build.
FossilOrigin-Name: 7e4c594e9591101f6b727212c99f561498444e86d3d059e5864a717c614f903a
2023-07-13 18:35:48 +00:00
stephan d655290c99 Minor cleanups related to reintroducing wasmfs support.
FossilOrigin-Name: 6918a927b5ecd1df5916b70d52d8d5ce289869c3f323e577b7dfc7c939084400
2023-07-13 18:13:52 +00:00
stephan bb9549e0be More work on the wasmfs build and its test apps.
FossilOrigin-Name: 953b8557194e9451dcf9f3bb433eafb67961487325e6519e675e73fecf850bfb
2023-07-13 17:27:05 +00:00
stephan d51cefd1bb Get speedtest1 wasmfs variant running again.
FossilOrigin-Name: 0f77e031daf0121f7603409cedf57f138edbb43083cbbcf00d2a423e16de25a7
2023-07-13 16:33:45 +00:00
drh bc6d949aa3 Fix an off-by-one error in tests for cells overflowing the end pages. The
error is completely harmless for the default page cache, but might cause
problems for application-defined page caches that pack pages tightly
together.

FossilOrigin-Name: ce6793e954f291b6f5c29175baf730ce217328de1f0601b8935daac62af5f448
2023-07-13 14:49:39 +00:00
stephan cc336321f8 Significant surgery on the wasm build on the way to incorporating wasmfs as a first-class build option.
FossilOrigin-Name: 215c37fce38cf647e073480689b79d952af7eb8165ae08c7f5caed27003fecfc
2023-07-13 14:08:30 +00:00
stephan d41d454fc1 More work on the wasmfs build. Resolve the inconsistent argument type passed to sqlite3InitModule() in such builds.
FossilOrigin-Name: 4850a9e2d2b41b91e91b9ef99a6d26ddf11a161b4c970fc00d2d095606729a04
2023-07-13 10:41:41 +00:00
stephan 0f0e5700ba Minor cleanups in the wasmfs build.
FossilOrigin-Name: f64477f21040c265061ad9b7f601e74159fd6141e7e4b122c80bf244ab6ddb76
2023-07-13 04:54:18 +00:00
stephan 01bdff7a1c Get wasmfs build and its bare-bones test app working again, albeit currently in ES6 mode only.
FossilOrigin-Name: 647761ed422f196f94facc88bbddd7219a2c1a6301a5f847b0a32d3e405233a7
2023-07-13 04:26:13 +00:00
stephan 81602595a0 Replace 'self' with 'globalThis' in some JS test code.
FossilOrigin-Name: 53eeffc9b1ea8c3ee7d7cdcc220997ed893403e45b35d8ea10805ea3b0435587
2023-07-13 03:46:43 +00:00
stephan 5da49ff562 JS: change the default OPFS VFS journal mode from PERSIST to DELETE, as DELETE is faster in current browsers.
FossilOrigin-Name: a061a43e36b8f3cb7c465fd2e54cfcd9dceb07e232ee78602bbdd1818acd05ea
2023-07-13 01:51:16 +00:00
stephan a5512fb179 Remove obsolete ext/wasm/api/sqlite3-wasi.h.
FossilOrigin-Name: f661a6e0c1ac081ee402e64590eabd427440db1b717be9166bdfc025d0b0d899
2023-07-13 01:48:53 +00:00
dan fc158d3870 Avoid a case of an infinite loop in fts5 when dealing with corrupt records.
FossilOrigin-Name: 0e801f11cd2d50fc710a80c2b3b805c7801e660cff1bcc87be89133d43796524
2023-07-12 20:24:23 +00:00
dan 9f6c0d0027 Fix a crash that may occur when handling corrupt records in contentless_delete=1 mode.
FossilOrigin-Name: 74d7610a8e32ac62a608141ff16bfe517c8d2cdfd43f81ef3a5df2b2003e541a
2023-07-12 19:34:32 +00:00
drh 9faa3bbdbb Get access to SQL functions and virtual tables used for debugging and analysis
of FTS5 using SQLITE_FTS5_DEBUG and without the need for SQLITE_TEST.

FossilOrigin-Name: 383de8e2259adb3f8881a618b8b5e06c1f3b2bd7ac274bd047410ad7d2a18f0f
2023-07-12 18:38:47 +00:00
stephan b3b64179df Internal JS doc fix.
FossilOrigin-Name: 7c5ea02205a360a11fa9043be12c678eb7fe6f6a0b63b09a3dc976beae881e3f
2023-07-12 09:30:13 +00:00
stephan a17b04f840 Correct handling of the speedtest1 --without-rowid flag in the HTML front-end to speedtest1.wasm. Reduce default speedtest1-with-OPFS workload size in ext/wasm/index.html to speed up manual testing.
FossilOrigin-Name: 2daba2cdd3890c87899ff3eb0c1706c814f0ae62e72822eef3b0d4a885679a6b
2023-07-12 09:16:18 +00:00
dan b184c91076 Have contentless_delete=1 tables support REPLACE statements.
FossilOrigin-Name: 2f553a660e00564e51bc4209c92bd3628fb1266f4a52832792fbf91e4234a0ba
2023-07-11 20:57:19 +00:00
dan b92669db73 Have contentless_delete=1 tables support regular DELETE statements, instead of just the special INSERT syntax.
FossilOrigin-Name: fffb8616905501669a94231d5d9f53446bf09553353f2cdab7c43ca54bbb7fa6
2023-07-11 20:19:46 +00:00
drh 8a1104940d Reduce divergence between branches: trunk and wal-shm-exceptions.
FossilOrigin-Name: d38ed6b1fc6b572997d520c36f256c11404fb1bd5f28acab07466bfc96554a8f
2023-07-11 19:54:17 +00:00
dan d23f210377 Use a hash-table instead of a flat list to store tombstone rowids.
FossilOrigin-Name: 948267b066d0dbe667881b3d26a007fa24576da6e57c112676fadeb846c13f0b
2023-07-11 18:55:19 +00:00
drh 66422b57f9 Get fault injection for testing working correctly. Other code clean-up.
FossilOrigin-Name: ff492277ed00c1f637a5b4ccd6d8193ea22f6781f90073861588a2b7d5c045b7
2023-07-11 15:52:52 +00:00
dan 6788c7b7c0 Begin adding support for deleting rows from contentless fts5 tables.
FossilOrigin-Name: e513bea84dfaf2280f7429c9a528b3a1354a46c36e58ab178ca45478975634e0
2023-07-10 20:44:09 +00:00
mistachkin c0b6112d04 Minor style changes to the previous MSVC makefile changes on this branch.
FossilOrigin-Name: 4c950872c870a5968fa4cb8840cf60569a66c0e508811ee79992825ec9c02da3
2023-07-10 19:21:09 +00:00
drh 7994d9dab7 Compile for SEH if the USE_SEH=1 option is specified on the nmake command line.
FossilOrigin-Name: 0e67a82f6cfe681d4676ac5c92d92e95b9dcc3198df21516bccdde002f5e52ae
2023-07-10 19:13:40 +00:00
drh 5d3cc0c09b Fix a warning. Unclear yet if the code is correct this way or not.
FossilOrigin-Name: 2df4fd76ef48bb63389df8d783059af6b8f2cbb2c11d07953b408e71f50b625b
2023-07-10 18:16:49 +00:00
drh cc9380f68d Merge the latest trunk enhancements into the wal-shm-exceptions branch.
FossilOrigin-Name: f655d08d019bab0c578886e91ca24fd956d1b3dea9b9c9104812b3cf62e5e556
2023-07-10 18:05:23 +00:00
drh 7ce7f18eb0 Merge the latest trunk enhancements into the bedrock branch.
FossilOrigin-Name: 35bfb5bfee1845c6749256826fdbf4e7b2f7ebe00fdceacb3edebec1c44828c4
2023-07-08 19:16:06 +00:00
drh f578ed6838 Merge all the latest trunk enhancements into the wal2 branch.
FossilOrigin-Name: f07936535f6ad2671e4e3281214b097a0186a642055ad81e561515cd08bc4a61
2023-07-08 19:09:20 +00:00
drh 4c40b7b819 Always use the "LL" suffix on 64-bit integer literals.
FossilOrigin-Name: 07d95ed60f0a17ea13b4bc19c2ab2ec9052fedd27c9e1e57a1ec6e3a6470e5b7
2023-07-08 17:42:24 +00:00
drh bc532ae15f Fix harmless scan-build warnings.
FossilOrigin-Name: beab3c98639be531744e60440223bb9ee76bc15234aff05e5efb273c8241dfd8
2023-07-08 14:27:55 +00:00
drh 50ba4e3efd Omit the use of memmove() in sqlite3FpDecode() in an attempt to avoid
spurious warnings from valgrind.  Also makes the code slightly smaller and
faster.

FossilOrigin-Name: b409943af00e35ad05906d0a80ffa1225a66f3d807c131e00cfbf6671a03a981
2023-07-07 18:49:08 +00:00
drh afef7fc136 For the "Valgrind" test scenario, set LONGDOUBLE_TYPE=double as valgrind
does not correctly emulated extended precision on x64.

FossilOrigin-Name: d3532eaed1cc671d8149b1bd34ead2a88fb83502a2898a5f60aea0a7daf18958
2023-07-07 12:18:26 +00:00
drh bae91082dd Add option 2 to SQLITE_TESTCTRL_USELONGDOUBLE to move the setting to its
defaults.  Enhance the CLI and testfixture to make use of this value.
Base the default on sizeof(LONGDOUBLE_TYPE) rather than on sizeof(long double).

FossilOrigin-Name: 64e6bd1c25d8e1dbfe511cba1921ff052c0fa4fe410fc9ce4435700a70cb88b1
2023-07-07 12:08:22 +00:00
drh 7d0103bb06 Improved handling of integer inputs to the new sum() implementation.
FossilOrigin-Name: 31561c63bde6af2210296810fd66232b935033b4d26273a6b8a52142c54b4e12
2023-07-06 20:34:06 +00:00
drh 68c88ea04f A more precise implementation of sqlite3RealToI64().
FossilOrigin-Name: 8813317cb52990d90bd48de163b6e35ddcad2b6f3187aa28b28e2e088b4489ce
2023-07-06 18:29:15 +00:00
drh ff96718b52 Merge all recent trunk enhancements into the wal-shm-exceptions branch.
FossilOrigin-Name: e59cea2d851be68559e7d01a01cb05684c44d151d022ea9790ce99671016ab00
2023-07-06 17:55:33 +00:00
drh 3e1e7aad3d Fix octet_length() test cases so that they work when the database encoding
is utf16.

FossilOrigin-Name: ad2e39626e6115fd322ef9495ae3bcd91343d7131a725f1bf24211a51a21dcc2
2023-07-06 17:14:47 +00:00
drh 59f13119cf Use the Kahan-Babushka-Neumaier algorithm to improve the accuracy of sum().
FossilOrigin-Name: c63e26e705f5e967e14ef6aea8ce226548293ad8d25066069f29fa89673913d2
2023-07-06 15:44:38 +00:00
drh 26cd8bc1af Fix minor inaccuracies in the new sum() algorithm.
FossilOrigin-Name: e3f7a960c9bd8e84cd70f0585bb955d043604a92001d0e2bf6c1216bb1fd7221
2023-07-06 14:45:53 +00:00
drh 45d7562843 Experimental attempt to boost the accuracy of sum() using the
Kahan-Babuska-Neumaier algorithm.

FossilOrigin-Name: ebc5edd3b10c1102b07b9fb0d6837266b81e55504ef883b9b8a7ad5e8ab29dd2
2023-07-06 13:19:10 +00:00
drh 3748b7329f Use "volatile" isntead of "#pragma" to get floating point calculations
working correctly when compiling with GCC for x86 machines.

FossilOrigin-Name: 9427f42687ed6d97c474bf42d0c3e82d6f4b0075e74206adcb5699d72e32140e
2023-07-06 00:55:06 +00:00
drh efd0cf87df This is alternative work-around for the x86 float-point problem that uses
"volatile" rather than compile-specific #pragmas.

FossilOrigin-Name: 1d972a690fdc70ab40862bd38427d68b48e8802ddf8e5c301f2d58ce2178b6ec
2023-07-05 22:05:18 +00:00
drh 728650ecb3 Improved comments on the work-around to the GCC x86 floating point wonkiness.
FossilOrigin-Name: 7b4c16731e7bf6f03f5adf4fcb2008c0b19be473fb1b90b405c217c08916586a
2023-07-05 19:56:14 +00:00
drh e68899f1e4 Work around for an apparent GCC bug. See
[forum:/info/ee7278611394034c|forum post ee7278611394034c] for details.

FossilOrigin-Name: 5d9e9364808793d65925d4efbfde0f4246df258758f15e8ce1105070d1018fe6
2023-07-05 18:59:52 +00:00
drh 85ca6d7ace Make the order of operations explicit for the error term of Dekker
double-precision multiply.

FossilOrigin-Name: 28f57b34e6b11184a36e363a985e7531bddd7be48a5e039a670e5acf748eedda
2023-07-05 15:34:30 +00:00
drh 1790ccb98b Simplification of the dekkerMul2() subroutine.
FossilOrigin-Name: bfe6432a46fd41bfadf4516a73b4e2523d9b453c3e4486fe6df7e1d25275dfb3
2023-07-05 14:42:50 +00:00
drh 6aa5a42beb Add the ieee754_inc(r,N) function to the ieee754.c extension.
FossilOrigin-Name: 9a36f269451d8832f68e223ac3995493704364e7fd0cc772989c96e863a08d29
2023-07-05 12:00:32 +00:00
drh 7623795018 Enhance the sqlite3AtoF() function so that it can operate with or without
long double and provides accurate results in both cases.

FossilOrigin-Name: fd987508d5589d42b46625c97cfb2feeac70fdb1cb11224610554b07a60644e6
2023-07-03 18:32:29 +00:00
drh aa4356da68 Enhance sqlite3AtoF() so that it honors the USELONGDOUBLE test-control.
Enable the test/atof1.test module on non-intel platforms.

FossilOrigin-Name: b7850fd480535980d66f107c2746e7f7fc0a5379ec8244d84f7785533c83f276
2023-07-03 18:18:35 +00:00
drh 930e4167bd Store the fp-speed-1.c test program used to measure relative performance of
SQLite versus the standard library, in the test/ directory.  A header comment
explains how to use it.

FossilOrigin-Name: eaff376277edceb043e9aac9d2df3e1157f576605508733be8a0a0aa4d720e1d
2023-07-03 15:30:50 +00:00
drh 521f20e3d3 Add the --nolongdouble option to speedtest1.c.
FossilOrigin-Name: 6bf6018a00d61c5f0e53b37b00befaf5ad1d85323f41cf0cea25d8c6c1017096
2023-07-03 15:11:28 +00:00
drh c9f240cea4 Improved precision of binary-to-decimal conversion of floating-point values
on all systems but especially non-x86 systems that lake extended precision
hardward floating point.

FossilOrigin-Name: 32befb224b254639e756fd493f300507ff60c5222e00a7338dbcd39140133964
2023-07-03 15:03:59 +00:00
drh 1b3c6e727e Whether or not to use the long-double floating point conversion technique
is now a test-control setting.

FossilOrigin-Name: 02ae6caff1f7925c696ad263fe78b1036364d24b6373e1baec10d53aafb14a12
2023-07-03 14:56:49 +00:00
drh 0478d1d88a New test cases for floating-point conversions.
FossilOrigin-Name: 9435ed76cd9bb1be02768b3853a2323834f67bda75d3e3f76f6aa7bd0bade3d7
2023-07-03 12:58:18 +00:00
drh b884158434 Use Dekker double-precision floating point to do the value normalization
step in sqlite3FpDecode() for systems on which "long double" is 8 bytes or
less.

FossilOrigin-Name: 51f492b440728151ce64168d5436ba87ca60d84b89408e47c593fa731355afe4
2023-07-03 11:44:15 +00:00
drh 841e596ad0 Final clean-up of the idea of using Dekker double-precision floating
computations to improve accuracy.

FossilOrigin-Name: 40285d631072b79aba39cc20d4b217bdf0069f29ac9902cb97dec28d3e4cde11
2023-07-03 11:39:38 +00:00
drh bc2b65832e Squeeze as many bits as we can out of normalization.
FossilOrigin-Name: 25d1ad6df1ed460c2cc9f32e8a5106e121ae502044f83abf431c1d5b3879eb56
2023-07-03 10:35:29 +00:00
drh 5270d7424d Faster Dekker multiplication that removes the restriction on input magnitude.
FossilOrigin-Name: 2994caf5884be07c889519c78fbac4ddcf267fcfe6a3265ecb6390bcd574532e
2023-07-03 10:18:02 +00:00
drh 37b188fbca Fix one constant in the normalization logic. Improved error output
from atof1.test.

FossilOrigin-Name: d3c48807100a358a70fdd799c8935eba1b765ace2e1ddea4475fd673006cb6da
2023-07-03 10:00:38 +00:00
drh c8885d42fe Replace the dodgy error estimating logic in the previous check-in with
full-up Dekker double-double multiplication, and this idea works much better.
There are still a few inaccuracies, but it is much closer.

FossilOrigin-Name: 4fa6938dac2d3d813a37664053db31451a2a065f78dd212488f5f7f8d583ddc5
2023-07-03 09:07:23 +00:00
drh c27bda0b79 An idea for improving accuracy of fp-to-decimal conversion for systems that
do not have "long double".  Does not quite work.  Retained only for
documentation purposes.

FossilOrigin-Name: 702243333843219f8904ee0fd12283080d6c6b3fc1ffb36c534cdefb3563c00d
2023-07-03 00:40:37 +00:00
drh fecdff4354 This check-in adds the use of "long double" to sqlite3FpDecode() which when
run on x86 using a compiler that translates "long double" into binary80,
causes all legacy tests to pass.  But on a system where "long double" is just
an alias for "double", some of the tests are still failing.

FossilOrigin-Name: ce06982f880339cf46704e95c907249827c3e44af2b9420005200ca8abd3f371
2023-07-01 20:21:27 +00:00
drh 453be55b85 Reinstate the fpdecode() SQL function for testing, but only when compiled
with SQLITE_DEBUG.

FossilOrigin-Name: 07eab52e0801bb0e4743b304a06ad16e18cdf8eaa18c0faf8d47a1f5d8576ea0
2023-07-01 18:33:26 +00:00
drh 17c20bb15e Improved rounding policy.
FossilOrigin-Name: 6f1122e942b8269552daaf13d647d200d8546ec25f36310d67037c6b58d09976
2023-07-01 17:56:00 +00:00
drh 9ee9444a0a Improved response to Infinity and NaN. Remove the termporary fpdecode()
function.

FossilOrigin-Name: 76ab8ae809a47a66688e2d50c20dc87ce946d82e9ffebb3adda55c451fad07fc
2023-07-01 15:23:24 +00:00
drh 42d042e602 Progress towards fixing corner-case floating-point formatting issues.
FossilOrigin-Name: 1138b8d16edef16d2943d407e289d07d546a14e64c3efa3c94127631e06fafeb
2023-07-01 14:03:50 +00:00
drh 3f073f9ef8 Fix an issue with the "+" attribute in printf().
FossilOrigin-Name: bdfd1eb1d7b4c570a53d58789dac0bf55fdaf8eba515d5a0caf2e57e25b51952
2023-06-30 23:42:46 +00:00
drh aebeabacff Begin cutting over sqlite3FpDecode() into printf(). The code in this
check-in gets a lot right, but there are still some formatting
discrepencies.

FossilOrigin-Name: c134d423e79ba5d3343d7480229822a0c83210f67f12129e0237181f892f97a5
2023-06-30 23:18:44 +00:00
drh 002330dc48 Add decimal rounding to the sqlite3FpDecode() routine.
FossilOrigin-Name: 27871140caa833f0bc0962e44356993938e93dcf81c1074382b1560a3e1aeb61
2023-06-30 19:41:57 +00:00
dan f8be243fef Add experimental ".scanstats vm" command to the shell tool.
FossilOrigin-Name: e727640fb5c17d23b8e27972065b4acbf169c9240298d3ff7aed092b727d052d
2023-06-30 19:14:35 +00:00
dan e6d7ae24e3 Fix an error in the previous commit on this branch.
FossilOrigin-Name: 47c11ca90f98ffc4d91f07e2ab35826a604a2c903008eeddf8b814bb984971f2
2023-06-30 19:13:27 +00:00
dan aec0aa83a9 Add the "nexec" and "ncycle" columns to the bytecode virtual table. For accessing counters collected when SQLITE_DBCONFIG_STMT_SCANSTATUS is enabled.
FossilOrigin-Name: f7b163a319bee9e935a4abf0bd87e16c9974cba5ae75b0cbba63c9da168f2006
2023-06-30 18:59:10 +00:00
drh a1b0ff1735 Experiments with a new algorithm for converting ieee-754 binary64 numbers
into decimal.

FossilOrigin-Name: e923405e448385085224f9289991b303d86b02763535ea77d6fcee98ba6fc1f2
2023-06-30 18:35:43 +00:00
dan 24b368da8d Add support for sqlite3_stmt_scanstatus_v2() profiling of GROUP BY clauses that use a temp b-tree, and for sub-queries implemented as co-routines.
FossilOrigin-Name: 7afad1f759f7ceda873c6d869422fd56fe4399c2d24d47ad9bc3b84b06d830d1
2023-06-30 18:31:37 +00:00
dan 4cd3a59a79 Improve support for sqlite3_stmt_scanstatus_v2() profiling of sub-queries implemented as co-routines.
FossilOrigin-Name: 4e8718dc35dbbaf75f17265a88d14acd9750dc75efbadf41377f9c97e732009c
2023-06-30 18:23:53 +00:00
dan 07d76f7b95 Add support for sqlite3_stmt_scanstatus_v2() profiling of GROUP BY clauses that use a temp b-tree.
FossilOrigin-Name: 796eadcc50e7ea9ec348b1f7e2d6a5a3de78c675a07f665078309e6d81479e7d
2023-06-30 17:14:36 +00:00
dan 1bfe92a691 Add extra debugging code for sqlite3_stmt_scanstatus_v2() to test1.c.
FossilOrigin-Name: f936f101406069b29218c89a36581b4497226fb61906782ea368f12d943c901c
2023-06-30 16:16:51 +00:00
drh 48114d08cf Completely unwind the enhanced precision sum() from [a915f15a916af698] so
as not to offend UBSAN and OSS-Fuzz.

FossilOrigin-Name: 85be05b66ef975f02a3e7b2984bcab97d9280c7f3b6ee1e16718de549f240d46
2023-06-30 14:01:09 +00:00
drh dd8b12a6c5 Omit the doubleToReal() function in vdbemem.c. Use the
equivalent sqlite3RealToI64() function in its place.

FossilOrigin-Name: 625820e8ebfdcf513c81b1b632bbe2aa882a8fbba52001113dc6f56270fa5ceb
2023-06-30 12:59:06 +00:00
drh 60f41362cf Make the sum() function less precise and slower in order to avoid
harmless signed integer overflow UBSAN warnings from OSS-Fuzz.

FossilOrigin-Name: 1be0646a2c352dbf03d2af87fd48b6f9edfd68666790ac6863144ac95f3e0621
2023-06-30 11:51:36 +00:00
drh 2ddfa6a360 Cleanup and commenting of the new DECIMAL extension code. No functional
changes.

FossilOrigin-Name: 5124481663eb8e74a9f861be98adb7075ea911fcff0216d98c658e955acadf14
2023-06-29 23:03:30 +00:00
drh ec3e57fa92 Enhancements to the DECIMAL extension:
(1) If the argument to decimal(X) is a floating point value (or an 8-byte blob),
the floating point value is expanded into its exact decimal representation.
(2) Function decimal_sci(X) works the same except it returns the result in
scientific notation.
(3) New function decimal_pow2(N) returns the full decimal expansion of the N-th
integer power of 2.

FossilOrigin-Name: 8baf8c10aecb261751f2b154356ab224b79d07230929ec9f123791278e601bba
2023-06-29 20:28:03 +00:00
drh 500ca334bd Fix harmless compiler warnings about unused function arguments.
FossilOrigin-Name: 24927c1377314a10177da4a57191593440aa97fd0c5949fdf25a22df1d947600
2023-06-29 17:48:32 +00:00
drh 0b1a1e8c37 Improve the range of floating-point numbers that can be handled as integers
for numeric-to-text conversion.

FossilOrigin-Name: 41580ba452fdbc3f73da60d8030289d38614c4cab8d24140d7cc44a54b2da8d2
2023-06-29 17:36:18 +00:00
drh a75011a891 Further refine the dtostr() testing function in the CLI so that it takes an
optional second parameter which is the number of significant digits to display.

FossilOrigin-Name: 2f9d4444aa503102a00d6e6769dadc57d4b26a2c07f145f23f2f28e0c161246d
2023-06-29 17:26:21 +00:00
drh cd88a74115 Change the (undocumented) dtostr() SQL function in the CLI so that it only
shows the first 26 significant digits - 10 more digits than are available
in a 64-bit double.

FossilOrigin-Name: d758859f6ab94ddb9b3ee6f6f5f24b16e2b7a7712761080cfc6540d68b5a0c97
2023-06-29 16:48:25 +00:00
drh 9ea7632cca Add the decimal_sci(X) function to the Decimal extension for showing a decimal
value in scientific notation:  +D.DDDDDe+DD

FossilOrigin-Name: 61d4923913e88b980ce93db4f3f9f9d7ba3baaac724995c36c9b887c034accdc
2023-06-29 14:49:23 +00:00
drh 7f5fe1faf3 CLI enhancements to facilitate SQLite core testing:
(1) Add built-in functions strtod() and dtostr() that convert text to
floating point and back using C-library routines.
(2) Do not disable all of ".testctrl" without --unsafe-testing, but only
those subcommands of .testctrl that are actually dangerous.

FossilOrigin-Name: 669996a8ddcbf35f3de66cf466508fc1e6dd09ab269aba395ac86a11b2ec238c
2023-06-29 12:14:10 +00:00
drh 8fbb335d9f Enhance the SUM() aggregate (and related AVG() and TOTAL()) so that the running
sum is accurate to about 100 bits.

FossilOrigin-Name: a915f15a916af698e0cef46c8b3e7ed11bda19349179d2d414073cd39c4cce24
2023-06-28 12:02:48 +00:00
drh 60783f47b2 Improved accuracy of floating-point conversion constants as suggested by
[forum:/info/598d32f6135c41c1|forum post 598d32f6135c41c1].

FossilOrigin-Name: 4943e8a1819e189747eefc414d02c0485e1620deff9cf92664295b21a8a9a83c
2023-06-28 11:46:28 +00:00
drh d847c73153 Simple test cases for the enhanced SUM() function.
FossilOrigin-Name: c66ef2440e4e6c7aa17b50e5a29e543713ccab69aa0a415bac29b35b6116504a
2023-06-26 19:57:23 +00:00
drh 12b198f1a2 Use ideas from T. J. Dekker in "A Floating-Point Technique for Extending the
Available Precision" (1971-07-26) to enhance the accuracy of the SUM()
aggregate function in cases where input magnitudes vary wildly.

FossilOrigin-Name: 439fc00fee62b4db3751860485e21a99cae4fd1f5d911b2c08651a1466245ecc
2023-06-26 19:35:20 +00:00
larrybr 502c618f01 Fix CLI .sha3sum error message for keyword identifiers, reported in [forum:/forumpost/7cbe081746dd|forum post 7cbe081746dd].
FossilOrigin-Name: 9d7813ddcf67f1256e7331c5f390cb122b3f7e35cf46e0b33b610216a5396a00
2023-06-26 17:08:08 +00:00
stephan b9561384d0 Replace some JS 'self' references with 'globalThis', as reported via the npm subproject.
FossilOrigin-Name: d4e66dfc2e87112fa73a9c64db358429d88fb2ab3a08908b0eb655f6ed42f94b
2023-06-26 07:05:05 +00:00
drh d2b9cdd592 Fix typo in the README.md file.
FossilOrigin-Name: 6d435d1dace1be52bdb920168e629412b9adf55dbe59cf3db8f169c26f6e6b81
2023-06-25 20:23:51 +00:00
drh cc19bed8b1 Add support for the octet_length() SQL function, with low-level optimizations
that avoid loading large string content from disk when the argument to
octet_length() is a table column.

FossilOrigin-Name: fa5f77862c0fe0189aa4246a1e55bb7c537c28c436ec10b75f5fa141e5e4aff0
2023-06-23 11:10:13 +00:00
drh 077efc2730 Optimize the argument to the octet_length() function so that it does not
attempt to read content from disk.

FossilOrigin-Name: 8b8ea4e3f52d96cc217bd1fb27ca4a83489ef1f250756ed2f790d1b0ee529a3a
2023-06-22 21:19:37 +00:00
drh 7f9dcd664d basic test cases for octet_length().
FossilOrigin-Name: 034a5fe2a0cfe085271385da1d281515b04b311e08962519d03ad8ad156fb4bd
2023-06-22 15:26:52 +00:00
drh f3445f4757 Merge the latest trunk enhancements into the bedrock branch.
FossilOrigin-Name: 0551b79b17579a91d891c745ecd0e35b5142f944ed59eb3c95c37e5a4de8a461
2023-06-22 14:40:38 +00:00
drh b17135d81f Merge the latest trunk enhancements into the wal2 branch.
FossilOrigin-Name: 80ac494f734d9720fbbb1111741aa99fc319c6c9a32e7f13b5962001d6ace828
2023-06-22 14:20:03 +00:00
drh d6c671e626 Merge recent trunk enhancements into the octet_length branch.
FossilOrigin-Name: d35c214811aac7dec0000ca2aa77231f74a7963dd0c53cf25a65ade5ef0f8dc0
2023-06-22 13:01:02 +00:00
drh d4170ac0eb Simplification to the getAndInitPage() routine that results in improved
performance.

FossilOrigin-Name: 2e9734c2335d8c06fedc9f4cca02baaf326f7fa276bd464f3214f383715a48d6
2023-06-22 01:03:39 +00:00
drh 71aac8763f Improved comment on the getAndInitPage() in btree.c. No code changes.
FossilOrigin-Name: dc468cfdb825083b3a4b6cb95c913961e9312e22103c5a0cd923b75c83c65e13
2023-06-21 21:47:09 +00:00
drh 3a202be6f1 Add the --pcachetrace option to the CLI.
FossilOrigin-Name: 61dfa92b44ad38a7aac76a09e167819ce5d0acace3e06ba9ed17b3264cc043c1
2023-06-21 14:11:25 +00:00
drh 40ee72947e Omit unnecessary calls to table locking routines in the common case when
there is no shared cache.

FossilOrigin-Name: f94f3021cde1d46373ee8fc8e5028d7507a937240c59cf0d0d19ab22acbd3c41
2023-06-20 17:45:19 +00:00
drh c439b9e0c6 Avoid unnecessary calls to sqlite3DeleteTable() with NULL pointers.
FossilOrigin-Name: 4a35f391995a88757d7b46f52bf241f523ee2a6a7b6726df00cddaec8cee7080
2023-06-20 16:26:52 +00:00
drh f6e04b6e29 Avoid unnecessary calls to sqlite3CodeRowTrigger().
FossilOrigin-Name: 27c174dbd0e7ba029d13fb005fc950ac99eb6aee31903ee3acfc88c262d13f6a
2023-06-20 15:48:24 +00:00
drh bcc76bd419 Performance optimization in sqlite3VdbeAddOp4Int().
FossilOrigin-Name: 456a24324bd42e1e3384ffa8474b7d0760b27df95be0e92ddce5eb87339f6272
2023-06-20 11:46:32 +00:00
drh f82c8cb0d5 Avoid unnecessary NULL pointer checks on calls to sqlite3WalkExpr().
FossilOrigin-Name: 73d86a12fb068456a884c030fddd87020d6ec9d56376de541cf082b2104174d3
2023-06-19 23:27:22 +00:00
drh 1e24dc987a Small performance improvement and size reduction by recognizing that no
SQL keywords have less than 2 characters.

FossilOrigin-Name: 6b3d25b7982623ab4b25161aff9ab44778e136069043ab425543597c42c98ae5
2023-06-19 20:54:49 +00:00
dan d095b2c3d5 Add the sqlite3_commit_status() API. For querying a connection for (a) the ranges of wal/wal2 frames written after a successful commit, and (b) the conflicting frame following a failed commit of a BEGIN CONCURRENT transaction.
FossilOrigin-Name: 4b08d4dad6b254a342353e3f765066c85cbc5450fe13501665c648627cca21cd
2023-06-19 18:16:19 +00:00
drh 1be61ad278 Fix large integer constants so that they work on older C compilers.
FossilOrigin-Name: 3c94f87806a8b408d8204fc7deec16d01c085ee199ff21a1f20b6346ce816cfe
2023-06-19 13:09:16 +00:00
drh 8e5cef7bca Fix harmless compiler warnings that show up on 32-bit RaspberryPI builds.
FossilOrigin-Name: bc4d20f362925e4ce5c79f0d7a27a8e9bbac92525bd4cea2ae983798e3f8c37d
2023-06-17 15:42:44 +00:00
drh 0587a1a8bb In date/time functions, fix the rendering of "subsecond" and the
computeHMS() routine to better deal with floating point
rounding errors that arise on some 32-bit systems.

FossilOrigin-Name: e46a00ae880dd12df090105498a85017d1367f88bf27d86f0b0200cf0536a906
2023-06-17 15:22:00 +00:00
drh 56a410741b Address various harmless compiler warnings from
[forum:/forumpost/d526da8ee4|forum post d526da8ee4].

FossilOrigin-Name: 365caf2f97e8f15842f52536e8d05d359e9d6e863182e020ce14a9a9f27ee057
2023-06-16 14:39:21 +00:00
drh 2e4d526996 Fix the pager such that if the xOpen() message sends back the SQLITE_OPEN_MEMORY
flag, the pager is opened in journal-mode MEMORY, even if compiled with
SQLITE_OMIT_DESERIALIZE.  No changes to the logic as long as that OMIT flag
is omitted.  We need to better document the behavior of xOpen to describe this.

FossilOrigin-Name: da1252b29852191eccbea98e0314408c75bb83a51f9d68d589705d4971a23850
2023-06-16 11:05:49 +00:00
drh f025fc164d Updates to the README.md file.
FossilOrigin-Name: fa5cb6dab52dcf853854ad12922a125b9a5f898362e883bb31b2f789c632288b
2023-06-15 16:02:19 +00:00
drh 733ef7e8e6 Fix harmless typo in a comment.
FossilOrigin-Name: 6937986a687e4afea39168702f3a29aef0d60cebff024edab30d1aecfe159065
2023-06-15 15:43:08 +00:00
drh 0c04f2e576 Work around what appears to be a GCC 32-bit optimization problem in the
computeHMS() routine of the date/time logic.

FossilOrigin-Name: aebdbcbebff1319dd00551c9fb9ad4b08859f10e87f097295d564ae1ba188c02
2023-06-14 12:19:07 +00:00
drh e30ecbfb84 Improved sqlite3_error_offset() values for bare column errors on '*' and
'table.*' expressions in SELECT statements.

FossilOrigin-Name: 118fe600876686273f85d0a080a21267e83c11826365f3220336b1bd39562518
2023-06-13 18:10:52 +00:00
drh d34ce81867 Extended the time-diff format so that it can have a 5-digit year. This enables
time differences to span the full range of dates supported by SQLite.  Also
fix some inaccuracies in the computation of time differences and add new
test cases.

FossilOrigin-Name: d57ddbf4eeeb3ccfe31871f098045b9e58634153a3c3030f597cb58c1177218a
2023-06-13 16:55:24 +00:00
drh d97e0d0e38 Initialize a local variable in FTS3 to avoid a GCC compiler warning.
FossilOrigin-Name: 7ec4ab327decd6a5ee5e6a53f1489e17e0cdbb297945f9acc532b47d052eb7a9
2023-06-12 18:22:34 +00:00
dan 16702ec69d Fix a problem causing bloom filters to be used less efficiently than they might be. Problem reported by forum post [forum:/info/06c3f01da6 | 06c3f01da6].
FossilOrigin-Name: 56d9bb7aa63043f5db4a1e5e8eefe46762dba433145807c6e27a8316f1d03d7a
2023-06-12 13:49:37 +00:00
drh 997670f997 Check-in [70e12efd1b189560] left a path through lockBtree() that failed to
set the BTS_PAGESIZE_FIXED flag.  That omission is fixed by this check-in.
dbsqlfuzz 02c9ab74eb0619c9ae3b894ce157fae1d7f94645

FossilOrigin-Name: 30dcc872eb1c9549708009c86357826c4f15ffa17e379dab5cd4aec596643728
2023-06-12 13:31:20 +00:00
dan d679f9e5d1 Fix a buffer over-read that could occur as part of an fts5 secure-delete operation on a database containing corrupt fts5 records.
FossilOrigin-Name: a35c50eca907d0cf373b7962c359aebfce8ac7c34ce81edeb99122d4fb5804f7
2023-06-12 11:06:57 +00:00
drh 26d71f03fc Make sure temporary sqlite3_value objects used by STAT4 are reset to NULL prior
to being reused, to prevent a memory leak associated with
the custom xDel string value destructor.
dbsqlfuzz 65209ab5d5730e4fd14fff9d024325a77239036e.

FossilOrigin-Name: 88be7075cdc7000d3a1e84e49064cf8c28fc7caca9c464f44a83265223961b51
2023-06-11 21:58:20 +00:00
drh b102d62749 Defer setting the BTS_PAGESIZE_FIXED flag until after we are certain there
are no errors and the page size has been set.

FossilOrigin-Name: 70e12efd1b189560e07f59db03a20b8c1ab19224aae4fc3d7c0bdfd2568064ea
2023-06-10 19:33:02 +00:00
drh 26e136bb9a Fix an assert that can go bad if STAT4 content is corrupt. This is a
follow-up to the previous check-in.

FossilOrigin-Name: ac1d3860af4eb30e4a7444b01d7b5afc91a4b1f5e3fe5414a491c6edc7ff1631
2023-06-10 18:40:20 +00:00
drh 04b92471f7 Extra space to prevent a buffer overread on corrupt STAT4 records.
dbsqlfuzz 7128d1b41ce9df2c007f9c24c1e89e2f1b2590ca.

FossilOrigin-Name: b99135288b157044e2319833e8632c89483778f876aa45ee66e46ffb6ae42ab2
2023-06-10 17:05:05 +00:00
drh 6bbc5b3730 Enhance fuzzcheck so that it understand the sqlite3_db_config() and
optimization settings from unused bits in the database header, which
dbsqlfuzz is now using.

FossilOrigin-Name: 918708c6dea5bffab4bb1c15d655ac7356bae97b84def905479dfcf491db6c5f
2023-06-10 10:37:49 +00:00
drh 7520116441 Simplify a memcpy() in defragmentPage(). It now might copy more content than
is strictly necessary, but runs faster and uses less code space.  Possible
reasons for the improved performance:
(1) the copy is now always 8-byte aligned,
(2) fewer intermediate results are required which means less register
pressure which helps the compiler to optimize the subroutine.

FossilOrigin-Name: 6e5607ae4d872954483a8d7a5c866aa41e4af70fae9652fb7eb211b316ab724d
2023-06-09 15:54:18 +00:00
dan 6e4ff8707e Remove SQLITE_EXTENSION_INIT macros from dbdata.c.
FossilOrigin-Name: 106ec745766ac59131f975d5ab5487c8a24b9c3be1766411c018b42c6ae4672a
2023-06-08 20:49:25 +00:00
drh 2f764eb4f4 Improvements to the src-verify.c utility program so that it continues to
muddle through a corrupt manifest file, making the most sense of it that it
can and reporting "manifest" has having changed at the end.

FossilOrigin-Name: 157b5d25e0c99eabfa3c32cb867fe7e3c05031c12354f734d2cd8a4062b9439c
2023-06-08 16:19:21 +00:00
drh 8efe749ab3 Fix the amagamation generator so that it works one a clean source tree.
FossilOrigin-Name: f502610d03a1362eea936129abd03390835b7e47ba25e6799b2e7880de540640
2023-06-08 15:30:10 +00:00
drh fd5d18045a Include source provenance in the header comment of the amalgamation.
FossilOrigin-Name: e3eb5af9050aae1db6887d17a560a24e4ca58f232ad7ac7c283c861ea4944f0b
2023-06-08 15:27:35 +00:00
drh 4fa5952090 Enhance Lemon so that it remembers which -D command-line options are actually
used in the grammar and includes a list of all such options in the header
of the generated output file.

FossilOrigin-Name: c47a4dbd24b8277c57b7a83a8c0aeac2bc8f6ab75d1b65ba5e1fa83d1868d95f
2023-06-08 12:52:28 +00:00
larrybr 4eda84a456 Add spell-checking for all comments in main sources. (./src/ .c .h .in) make target "misspell" tells of misspelled words for which no exceptions have been put into tool/custom.txt .
FossilOrigin-Name: 5332abf0ad8ef42c44e9dd7045d017bb8fc9a0262a89ababa3e2c41788a6cdb8
2023-06-07 23:10:40 +00:00
larrybr 55be21647e Fix straggler misspellings and tidy the custom dictionary. Also include pickups from [forum:/info/c61fb09afd|forum post c61fb09afd].
FossilOrigin-Name: 8c291d99946eb32b20b743921202f9c7cfb716268ff526817b27adbb7942e40b
2023-06-07 17:03:22 +00:00
drh c5a5a54c51 Remove additional traces of FTS1 and FTS2 that were missed by [2bb50d5aedef0fd2]
FossilOrigin-Name: 9a12bac5d24c7a8364d2127fd12c3409a53ef83e408d10f344ddd311cdba98af
2023-06-07 15:00:54 +00:00
drh 1d7c88bf98 Remove unused, legacy shell script "mkso.sh" from the root of the repo.
FossilOrigin-Name: 57f75e48690bdeb9b6bb3f46eee39021e3eb8cc0c488d2584177baa11ac984e5
2023-06-07 14:47:29 +00:00
larrybr bc91738e66 Add a C-source spell-checking facility. make misspell (on Nix)
FossilOrigin-Name: 26c1bb4bd9e9f56613c3aa87407a7f562fd4ebde5bfd6dece02078001d9a45f8
2023-06-07 08:40:31 +00:00
drh 25e87ed1ba Fix compiler warning in src-verify.c.
FossilOrigin-Name: 2d58c77448cfaaf12887b60bcfea039d3410a750ac8955a817c45be85c00eb9e
2023-06-05 17:12:33 +00:00
drh 4246a4bf14 Update the README.md file to talk about the verify-source makefile target.
FossilOrigin-Name: aa8e79e16abf028d50370ae13b6c952a7b7f42985125bc8e7f51b20a10bfc336
2023-06-05 13:57:37 +00:00
stephan dc608d8a5f Remove extraneous console trace output from previous check-in.
FossilOrigin-Name: c9f6e36d59fc8bf2bcef6fc14ff2e9ad3dfd972de94d4a71ff3f56134810a8ef
2023-06-05 11:30:50 +00:00
stephan c81dcaabe8 Add a JS test confirming that binding of statement parameters in a subquery works.
FossilOrigin-Name: 5dfaf0bce83c3e15ad605e3f07291ce219f1a2726ce77be27779897088ee13d5
2023-06-05 11:26:59 +00:00
drh 706047470e Generalize the src-verify.c utility program and provide additional documentation
in the header comment.

FossilOrigin-Name: 7ed84046ef0c2c64031cac3a3a6b43aac3ac40aeb36838d8d046493f2d5ed122
2023-06-05 01:05:46 +00:00
mistachkin 59714faa68 Minor cosmetic normalizations for the MSVC makefile.
FossilOrigin-Name: c9fda8d6dba36841415d0f445b4081b051bfaa70428b605ac0b9ca4ae98f7d49
2023-06-05 00:57:38 +00:00
drh 979715bb45 Fix harmless compiler warnings in tool/src-verify.c
FossilOrigin-Name: 0e79ee97b18792dba9213bd9961f53e2bae765ac92df7963703d08857da74fd1
2023-06-04 22:59:20 +00:00
drh 10ca116b3e Enhancement to tool/src-verify.c to also check the manifest.uuid file.
FossilOrigin-Name: a39a569482a35610fdaa2fdfc88491d8b456dfedb4786c817e0502eb3e3a5563
2023-06-04 22:56:31 +00:00
drh 63329ce6c7 Add a prototype implementation for the "verify-source" Makefile target.
FossilOrigin-Name: ed876ff6a1a6a2d555f32b96eb78d95eaf8428ee189f968c43f0829a3065bfa7
2023-06-04 22:41:52 +00:00
drh b3d7f1c2db Prototype implementation of the octet_length() SQL function.
FossilOrigin-Name: 2db989c8635f7e89b3ea58d1fde94787fced039ac1a118d9b6362811eda73f87
2023-06-03 11:22:30 +00:00
larrybr f1967a7018 Clarify sqlite3_is_interrupted() return. No code change.
FossilOrigin-Name: b6c2d3d203100a34335aeba23a8fb82a13297b65eed494d5044ade3bed683bc3
2023-06-02 21:31:15 +00:00
drh 038158e94a Restore the LEFT JOIN strength reduction that was partially lost in
[d747afda5683ca5f] when strength reduction was generalized to all OUTER JOINs.

FossilOrigin-Name: 2d962b84dd8ac2d0968f0283f8f082c1c2a7a1f99b59053b2ad8f58745396447
2023-06-02 18:05:54 +00:00
larrybr a3f943942c Do that last CLI fix more simply, using established means.
FossilOrigin-Name: a1823b297616bc2bd64eda9df4850fb460d8da8959ae3728195db896d1f89e7b
2023-06-02 13:31:12 +00:00
larrybr 4e8a0eb4e7 Fix CLI fault on missing -nonce reported by [forum:/info/f8c14a1134|forum post f8c14a1134].
FossilOrigin-Name: cd24178bbaad4a1dafc3848e7d74240f90030160b5c43c93e1e0e11b073c2df5
2023-06-02 12:56:32 +00:00
drh 4ffcce88ad Fix the OUTER JOIN strength reduction theorem prover's handling of the CASE
operator.

FossilOrigin-Name: 73d7b14b1713d93c4ae07995e9aa6485cbad90bd6125584dbaeccb0cc9410ea3
2023-06-02 00:03:28 +00:00
drh 179c32cb57 Generalize the LEFT JOIN strength reduction optimization so that it works for
RIGHT and FULL JOIN as well.  Rename it to the "OUTER JOIN strength reduction"
optimization.

FossilOrigin-Name: d747afda5683ca5feb92866a14ccc4c5127d3378ece5cad2c3da819f2477b457
2023-06-01 20:38:22 +00:00
drh 39a7a1a9be If the filename argument to sqlite3_load_extension() is an empty string, then
raise an error.  It turns out that if dlopen() is called with an empty
filename, it tries to load the current executable.  And then if the caller
requests some other function from the current executable, mischief can result.

FossilOrigin-Name: d01688554715eb4aaa1b1fd1a6b660b84e930edb0e062156ecf5228ee81ca754
2023-06-01 16:40:28 +00:00
drh f96148c4c6 Add an ALWAYS() on an unreachable branch.
FossilOrigin-Name: a00928d48061c1169d5564996e19e7d7c2b962842100bb119846d0f696123c23
2023-06-01 00:28:11 +00:00
drh e411b69537 Fix the LEFT JOIN strength reduction for IN operators in the WHERE clause.
Further simplifications and refinement of the algorithm.

FossilOrigin-Name: 96c72dde79d4069f6c2f81467a35b617633f86f7a7dcafbda991affdaa1f8537
2023-06-01 00:01:20 +00:00
drh d8cf688469 Add a comment to explain why both sides of an AND must be non-null-row in order
for the overall expression to be non-null-row.  No code changes.

FossilOrigin-Name: 8396032ce14a75f408f0a75bcb36a6504d5188f20886e275746a2c336a74296f
2023-05-31 18:52:46 +00:00
drh 29e70da4a4 Improved detection of when the LEFT JOIN strength reduction optimization
can be applied.

FossilOrigin-Name: f544a8e47cdd5ad7233887a558489983f4f305a39391ff463c43e2e4157da087
2023-05-31 18:35:12 +00:00
drh d3b54d624f Minor enhancement to the sqlite3_trace_v2() documentation. No code
changes.

FossilOrigin-Name: fba97230e38276f2a02fc24353c3d911dfa24f3a4f75f691de9461d4daf3ae54
2023-05-31 12:42:09 +00:00
dan d8091fe979 Fix a problem causing an infinite loop in the recovery extension when processing a corrupt free-block list.
FossilOrigin-Name: 5a516c6c7a25cec4d0a0cca530017a3299f7a332319f3e1f7cf12f53c32b88ca
2023-05-31 11:03:17 +00:00
drh 8eb9c06e65 Enhancements to date/time functions:
(1) Add support for time-shift modifiers of the form (+|-)YYYY-MM-DD HH:MM:SS.SSS
(2) Add the timediff(A,B) SQL function that returns the time-shift modifier
sufficient to move B to A.

FossilOrigin-Name: 574bf370b0b9a72eb0468e0e121d4d6171fc533883b66d335f6794ef5ee1f051
2023-05-30 16:40:42 +00:00
drh 7d65f1bec0 Simplifications to the new timediff() logic.
FossilOrigin-Name: d6954259bd3f8475f3804fd02a1591d037efa9df54da45776863c024ff90e596
2023-05-30 14:46:30 +00:00
stephan 73d806dc1f Minor tweak to the OPFS async proxy to attempt to account for a spurious high-load console error message which was reported via the sqlite-wasm npm project.
FossilOrigin-Name: a61467d1fbb7a2804bfdc685a38e0b738747b9afd42c25a78a938b114a822f61
2023-05-30 14:10:28 +00:00
stephan e32fef1400 Minor internal JS doc fix. No code changes.
FossilOrigin-Name: 68abee3c1bf4e9ff632fccaa6ac7c5c4cdf55c4ff1749830ae3bd5ff6d55ebb2
2023-05-30 13:27:57 +00:00
drh cc4bfd9623 New test cases for the timediff enhancements.
FossilOrigin-Name: f14633ce6ddf28a14b03e2da623d96e439966aa3fc30c26cb0be59c0e15b3e9c
2023-05-30 11:51:45 +00:00
drh 30bb082d3a All date/time modifiers of the form (+/-)YYYY-MM-DD without the following
HH:MM:SS.SSS.

FossilOrigin-Name: fa9237a8ab803ca3f67fe753516b7bdb98be3d625e8e241ddf9b02509210e225
2023-05-30 11:13:56 +00:00
drh 36a5f1e90c Problems fixed. All tests are passing now.
FossilOrigin-Name: cff293b0a79107def7bfe54fd60ac4a721c5098c85229d9b46a4e2c4cbf34a6a
2023-05-30 02:50:32 +00:00
drh 6abbef89e6 Improvements to the timediff() function. Test cases added. Not all test
cases are passing.

FossilOrigin-Name: 2cf6518140d61e2123dc20f39d7e97e7bce17e7641405df7fd63e1cdda444ae0
2023-05-30 02:16:33 +00:00
drh 1936a1933c New date/time modifier: (+|-)YYYY-MM-DD HH:MM(:SS.SSS)?
FossilOrigin-Name: 3832b8a9ef84d9b53b48dfce9c1e724897cd7b3e05711e01675419655f894eac
2023-05-29 20:33:33 +00:00
drh 0cbc2143b9 Prototype implementation of a proposed "timediff(X,Y)" SQL function.
FossilOrigin-Name: 054a195125a273bab026ada5f07cc7c32818007027a1fd028ca59d2f179276d4
2023-05-29 18:01:42 +00:00
drh 4308b88dcd Fix an issue with sqldiff reported by
[forum:/info/9bd2155bdfae25a7|forum post 9bd2155bdfae25a7].

FossilOrigin-Name: 736a79c1491065f4ee77ce644af0e529e555ca733c78dbd55487d55d4ddbe5bd
2023-05-27 20:08:23 +00:00
dan 870fe69c72 Add extra tests for fts5, large rowids and "ORDER BY rowid DESC". No changes to code.
FossilOrigin-Name: b5b5eaeed3ef55415e27aa04561ab446f3be881b56b7cbf9b8ba3307f0b2bf69
2023-05-27 19:34:46 +00:00
stephan 39bd6a0d46 Add ability for the JS Worker1.exec() API to report the number of changes made to the caller, per request in [forum:d0b19483642e20dd | forum post d0b19483642e20dd].
FossilOrigin-Name: 6e79505df915612b60696e4eec5c9973175fe6ecf273eb3152b996e63ae54a07
2023-05-25 16:49:06 +00:00
stephan 4e8e33ba84 Minor cleanups in demo JS code, per forum feedback.
FossilOrigin-Name: 80c7c0360c08bea0733deccb8071920c60558b75df76b6afad093c82adf30ea6
2023-05-23 19:11:42 +00:00
dan 583a5cf5d2 Avoid performing aggregate processing more than once for a single expression. [forum:/info/36ff78b2a3|Forum post 36ff78b2a3].
FossilOrigin-Name: 367b4cc549a14222d54530538e692e9d1a365002c1b8c4ef80ddc0523bfdb3a2
2023-05-23 17:35:18 +00:00
drh ddf6b27579 When converting an aggregate expression to AGG_COLUMN, be sure to remove
the EP_Unlikely flag if it is present.
[forum:/info/fbfe330a2009d036|Forum post fbfe330a2009d036].

FossilOrigin-Name: 48e65f18b2ebbe786e84e208f2c6c68de6067ba86912fea5a120fd41d0af2184
2023-05-23 15:16:20 +00:00
dan 3195b88b79 Fix a buffer overread in the recovery extension that might occur on 32-bit platforms.
FossilOrigin-Name: ff4a9a2b59657116da99c748ada19dbc64b7d0fd4c920e1c517d8bda3466f06b
2023-05-23 14:05:02 +00:00
dan 9d31508a54 Add test cases for the recovery extension. No changes to code.
FossilOrigin-Name: cec49c7d93362f527f0b4744cd1ae95d44a79671d49d69baa77fda70be29f7e8
2023-05-23 11:47:56 +00:00
larrybr d294792d1a Avoid duplicate WIN32_LEAN_AND_MEAN #define. Fix a trivial nit likely never observed.
FossilOrigin-Name: 81ffcf41d69ae73ee8c037f675e18e2b46a15bee34062914640456381262d6fc
2023-05-22 20:36:55 +00:00
drh 0503cd6d11 When a floating-point RTREE is presented with large integer constraints -
integers that are too big to be represented exactly by a float - then take
extra steps to ensure that all possibly relevant entries in the RTREE are
returned, even in boundary cases.  Fix for the problem identified by
[forum:/forumpost/da70ee0d0d|forum post da70ee0d0d].

FossilOrigin-Name: bfd8d9100015f3e3fb011698963d670bd89b64ec8a8ab931e0c6c3076b029377
2023-05-22 16:35:21 +00:00
dan b838a229be Fix a buffer overrun that could occur in fts5 when processing corrupt records.
FossilOrigin-Name: 4891dbd938f4bfd345eaef01f2addea9512eaa98f860844c73abb907b6a8e0e8
2023-05-22 11:02:15 +00:00
stephan a6ab50bc42 Extend detection of the INSERT...RETURNING locking case to the DB.selectValue(s)/selectArray/selectObject() family of functions. Add tests for INSERT/UPDATE...RETURNING with those functions.
FossilOrigin-Name: 3181c50540df0eff6cb5db79bb477c469bb7b73b0692260ba600db200fcef4ac
2023-05-19 18:23:53 +00:00
stephan 02be13cea6 Minor doc typo fix.
FossilOrigin-Name: 4533be08786306a4fcb4b21458444dffd0fa2764ebb2538313f3a16d9324cacc
2023-05-19 17:59:21 +00:00
stephan a382d236da oo1.Stmt.finalize() no longer throws, but instead returns the same as the C API. oo1.DB.exec() now triggers the INSERT...RETURNING locking failure as an exception via reset() instead of finalize(). Some code-adjacent internal API renaming for clarity's sake.
FossilOrigin-Name: 4ee6b3aa531b980acea4c4b58ee256e765c5105100468928def3d4c9825fa9bc
2023-05-19 17:50:16 +00:00
drh fe19414331 Improved documentation for sqlite3_reset(), in response to
[forum:/forumpost/a72bab3dea|forum post a72bab3dea].

FossilOrigin-Name: 1dfeb3dceee8f30daf5462683f264b9de23e7068e036e70b11ee1b608ac2f7fa
2023-05-19 16:42:49 +00:00
stephan 7fe416fae1 sqlite3.oo1.Stmt.reset() now throws if sqlite3_reset() returns non-zero, analog to [f23eb5c6d365].
FossilOrigin-Name: 487ae12c9a21e5862bd590bbb1030c39734657d52136cf67b98c7545e6ecbe1c
2023-05-19 16:34:56 +00:00
stephan 416aeb16ee Add a basic INSERT RETURNING test for the oo1.DB.exec() API.
FossilOrigin-Name: d29d62cf7658aeb49f3c8a5d0b0809d945ebc9b79379a255eb88f771d2a2c430
2023-05-19 16:12:06 +00:00
stephan 2492b6c0fb sqlite3.oo1.Stmt.finalize() now throws if sqlite3_finalize() returns non-zero. This is intended to address the INSERT RETURNING case covered in [forum:36f7a2e7494897df|forum post 36f7a2e7494897df].
FossilOrigin-Name: f23eb5c6d36546ee1e181a03660e0b2dc8005bba24bee8bae594b0c78bd152cd
2023-05-19 15:54:41 +00:00
stephan 8e2fd6eae3 Correct typo in an 'extern' decl name, reported in [forum:1d4342156439233b|forum post 1d4342156439233b].
FossilOrigin-Name: 6ac18827d8c5be35442dc452af703023c759916c179f165c4e54b3e430a8f7b0
2023-05-19 12:41:14 +00:00
drh 1cb4e2e196 Change the RTREE engine so that equality comparisons of floating point
values are duplicated in the byte-code so that the comparisons are done
using 64-bit floats and consistent results are always obtained.
[forum:/forumpost/da61c4a1b5b4af19|Forum post da61c4a1b5b4af19].

FossilOrigin-Name: 4ec0f0a31b0df93ad15545fe1db117c933e28c75ad3cbc8475b56fecdb084905
2023-05-19 11:48:05 +00:00
drh 2c6c98dbf0 Fix incorrect b-tree root page reports from the enhanced PRAGMA integrity_check
output introduced in version 3.42.0.

FossilOrigin-Name: 9b538f62539bcde7d7d7ad06eccdc558ecbd3f9160db35197568f4d3b4247c76
2023-05-18 23:25:48 +00:00
drh a03be7987b Fast-path the btree transaction start for the common case where a
transaction is already active.

FossilOrigin-Name: 798c3ff20c606b5f9fde16dc67781f238370a7c450bc239c3a98d0f4ca370399
2023-05-17 19:23:24 +00:00
drh 653882ed8c Simplification, performance improvement, and size reduction in the
codeAllEqualityTerms() function of the code generator.

FossilOrigin-Name: 4fbe02651e877e442d62308d7f607e503695104cd71c5565132fcd1398c120df
2023-05-17 16:13:48 +00:00
drh 60fd5c34ab New assert() statements to verify that Expr.iColumn is never used as an
array index when its value is negative.

FossilOrigin-Name: 6084c5fb6d3fcedf35cd6c597a44ec7bf8b4a2576c7b277e5342d2a7905318e7
2023-05-17 15:46:46 +00:00
drh 9c9b33cbaf Improved assertions on substExpr().
FossilOrigin-Name: 6dbd5b3cc801a75ea23842a43eb7106b62fc1d26dd5a8477a0217325539a7c1a
2023-05-17 13:21:45 +00:00
dan 32a5feb53e Avoid an assert() failure that could occur in SQLITE_ENABLE_UPDATE_DELETE_LIMIT builds when a WITH clause attached to an UPDATE or DELETE statement created a CTE of the same name as the table being modified.
FossilOrigin-Name: 8edf56d1698c3db38250db3b25864f658488414efb3a6b3e1443283b6affd26d
2023-05-17 11:31:51 +00:00
drh 010bd47b98 Fix two assert() statements that failed to take into account
the likely() built-in function.  This does not impact production code.

FossilOrigin-Name: 8504fe812c5b4956c8acad515d537add54cb591bc81747113f9b24526d7685b8
2023-05-17 00:26:50 +00:00
drh ea1b2fa034 Increase the version number to 3.43.0 in order to begin the next
development cycle.

FossilOrigin-Name: 7874023cd460cbe05401433b8e695a77955e2c055c360a93b92d4ce6937bd136
2023-05-16 14:20:04 +00:00
stephan 73ab708b5c Resolve the JS-side corner case reported in [forum:7774b773937cbe0a | forum post 7774b773937cbe0a] by not caching oo1.Stmt.columnCount.
FossilOrigin-Name: 37d44e8c447626738a8a7f915a9bc3f35fbb95581236dd152fc119b0c3a24c2a
2023-05-16 14:18:48 +00:00
drh e36b69a347 Merge all 3.42.0 release changes into the bedrock branch.
FossilOrigin-Name: 1348c2a59027d6e260d4c7543479093eaf7baccaa3ea61a0d091c44e0dea4b8c
2023-05-16 13:04:38 +00:00
drh 5d2f5f4655 Merge all 3.42.0 release changes into the wal2 branch.
FossilOrigin-Name: 137057f95778b3c913854d2182d0fbbfd9dd117db5566dabb5a22d927a59de62
2023-05-16 13:00:36 +00:00
drh 89efa89778 Version 3.42.0
FossilOrigin-Name: 831d0fb2836b71c9bc51067c49fee4b8f18047814f2ff22d817d25195cf350b0
2023-05-16 12:36:15 +00:00
drh 89caab8147 Update the tool/mkmsvcmin.tcl script so that it prints a one-line message
to explain what it has done.

FossilOrigin-Name: 4254f086419892634c80b0b915e14edea365adc6a45bcc75eecba889da274ac3
2023-05-16 12:34:26 +00:00
drh e680b6dcf7 Fix a C++-style variable declaration in the generate_series() extension.
FossilOrigin-Name: 1d3e008905461ebbd3ea0a862672f740fa72914d4d59fcf800e1ce56f1edfc9d
2023-05-15 19:17:31 +00:00
dan aeb064c069 Limit the number of nested NOT nodes in an fts5 expression to 256.
FossilOrigin-Name: 01219e69b430c8f5fea5ab6ce511ba8c9b4c9b32b6d2d36623dde99c3d3812c9
2023-05-15 17:24:48 +00:00
drh ff4e771c0f Increase the stack size available to fuzzcheck to 8MB on Windows.
FossilOrigin-Name: 2290dc6876041bcc982f162f59fdc9fd98b0983e1058c0a33fd46937ecc4bd84
2023-05-15 17:20:21 +00:00
dan 4042d2b424 Limit the number of nested NOT nodes in an fts5 expression to 256.
FossilOrigin-Name: 0e5c1ee40a146ef8b2b3c5f53d0a45e092bc8d8e933f3819805c995819d31bae
2023-05-15 17:14:16 +00:00
drh 307db965c6 Add the --wait command-line option to fuzzcheck.
FossilOrigin-Name: 5a479a2c7637a018d1828d9d37973bf3d36a2fc4cd5b752bca61a41cd0920d39
2023-05-15 10:55:04 +00:00
larrybr b579db6d5c Make generate_series() correct on ones complement ALUs and acceptable to UBSAN.
FossilOrigin-Name: 4c5cd3e6968c44749b7663beee790969b41d247b9201b771ef90cbb03d09108f
2023-05-15 03:48:48 +00:00
drh eb4455e4e4 As evidenced by [forum:/forumpost/f3f546025a|forum post f3f546025a], the
new RIGHT JOIN related restriction on the push-down optimization implemented
by [da3fba18742b6e0b] also needs to apply to the automatic index
(a.k.a. hash-join) optimization and to the Bloom filter optimization.
Computation of the restriction is now
moved into the sqlite3ExprIsSingleTableConstraint() routine.

FossilOrigin-Name: 4902015dcf3869f08d9986e422faa231d9218a5e0fc59ba8df0f407e4eb3d605
2023-05-15 02:06:35 +00:00
drh fa746af4a3 Simplify the interface to constructAutomaticIndex().
FossilOrigin-Name: c5da16551619718bf649c517515261706843e11e131f0b99e1fd6927d0f7e238
2023-05-15 01:02:50 +00:00
dan b8d689b666 Fix windows-specific problems in test scripts sessionnoop2.test and sessionrowid.test.
FossilOrigin-Name: e4989c111ddffe6818cbb88a72bd027b30124d1e8de0cce4109c72a214ffb5b8
2023-05-13 20:01:27 +00:00
dan 5e65cd1caa Remove unnecessary call to sqlite3_dbdata_init() from shell.c.
FossilOrigin-Name: 84ca2000007d217f911a3b1586143fc9261c0697a9e9fadda6bd3dd6afdfb099
2023-05-13 19:38:30 +00:00
dan fd0791587b Add an "#ifndef SQLITE_HAVE_SQLITE3R" directive to shell.c.in, to avoid including the recovery code a second time if the shell is being compiled with sqlite3r.c.
FossilOrigin-Name: 6f1a60fb7b3ef463b4131dfecbc72b7c778c794a9aa13ce4617ab6dd895508b2
2023-05-13 19:13:40 +00:00
larrybr ea51c09a8b Mention that sqlite3_mutex_free() accepts NULL as a no-op.
FossilOrigin-Name: 603d9ad5012ca8996783996d7b8cd6a1aabf12b21604a2ccc137f4c2d99427b9
2023-05-13 15:34:15 +00:00
drh 20035253bd Merge the latest 3.42.0 beta changes into the bedrock branch.
FossilOrigin-Name: d55ba8bb85b1796d534d2db196db63312624630738712054553bd01f2ab30df9
2023-05-13 15:10:42 +00:00
drh 1adff07cb9 Merge the latest 3.42.0 beta from trunk into the wal2 branch. For some reason,
the previous merge didn't work.  Probably a PEBKAC.

FossilOrigin-Name: 3bbfbdcd9b0e03b1ed50f41b1714ada5d241b8030f73ced8d0da8fff28a268e0
2023-05-13 15:07:27 +00:00
drh 5b39dd45e8 Merge the latest 3.42.0 beta changes into the wal2 branch.
FossilOrigin-Name: 771361e19e5b7f944cd2cfecb064b39175b892148fa877b763170744bca0b0cf
2023-05-13 15:00:47 +00:00
mistachkin 273c2a9425 Fix compilation issue seen with MSVC.
FossilOrigin-Name: 67001887044da71ab50c07d8345bd950a0a1f9688cf9cd6966312ea163cf6e17
2023-05-13 05:13:29 +00:00
drh 59b6a2bcbe Fix harmless compiler warning caused by [0772ddf56713d013].
FossilOrigin-Name: f06c16a8b0e7a15ce4f7d99af3376a1bf1bfbfc0fdc048b079418ae74c619d6b
2023-05-12 19:06:00 +00:00
dan 83be5586a0 Add multi-threaded test for "INSERT ... RETURNING ..." and SQLITE_BUSY errors.
FossilOrigin-Name: 559cb1c6de384fd14160db005acaf1858b8b018d3d40d7f786a56a49d3cdbd59
2023-05-12 19:02:50 +00:00
drh 4f3dc93978 Make the regexp extension more robust against OOM errors during compilation
of the recognizer engine.
[forum:/forumpost/f50aecd5e8|Forum post f50aecd5e8].

FossilOrigin-Name: 0772ddf56713d013cd1bd44f9c75977ca14f852e3a8f038b0a6b9814f6519d79
2023-05-12 15:45:34 +00:00
larrybr fb74019a14 Fix unlikely strange result of CLI edit().
FossilOrigin-Name: ab3797e844c97fed344b36f30cfb788aca9e0d68c574fb833712219eb712db44
2023-05-12 14:51:00 +00:00
drh 69bc8c70f9 Ensure the _O_U16TEXT is defined for Windows builds of the CLI.
[forum:/forumpost/c80aa942dc6bf2|Forum post c80aa942dc6bf2].

FossilOrigin-Name: 504effa89b48c0dc6cd12b3eaf6017be3f32017c601af17759a3cc185346d868
2023-05-12 13:20:57 +00:00
drh 26a3e56649 Remove a stray CR from end of line in a source file.
FossilOrigin-Name: 48139fb904de4b7b383e7f8c29bed83ad878eb22b1dd773c366a0aee74613d23
2023-05-12 12:57:44 +00:00
drh 61791c6f4c Fix harmless typos in comments.
FossilOrigin-Name: 607c3efbe0293d7c11a955a9f21e24fb9f7f1bb758e13e60f4ba0cc0ac8b900c
2023-05-12 12:45:56 +00:00
drh 6b429dc11b Ensure that all prepared statements are expired and forced to reprepare
whenever the database text encoding changes.
[forum:/forumpost/43cbfc4d23|Forum post 43cbfc4d23].

FossilOrigin-Name: 469718f106e1cfa7f8f4714a9e743108c361af81e0258061c2b76880a7c352ae
2023-05-12 10:52:12 +00:00
drh 78b404d976 Add [/info/cec6a06508239c09e363?ln=5525-5531|restriction (5)] to the
omit-noop-join optimization.  Fix for the issue reported by
[forum/forumpost/49f2c7f690|forum post 49f2c7f690].

FossilOrigin-Name: 3e9c9bbdb59b9d500ff218db538c047c83da7ac18ebb95c3ee7629ab15e0b43a
2023-05-11 21:15:55 +00:00
drh 5ca40d84ad Fix a minor typo in a comment. No code changes.
FossilOrigin-Name: 607cfb5bc5c0fb8b789944c2326cfdebf0629e45fbf0a61dd5f667ed685a1bbe
2023-05-11 18:03:11 +00:00
dan 418d212308 Fix the rbu_exclusive_checkpoint=1 option so that it works with a zipvfs database in pass-through mode.
FossilOrigin-Name: caca67d30670913914335aa3edff784e1c3d26aed70ae32efb5607ccf31ad142
2023-05-11 17:25:14 +00:00
stephan 0c7bcf4c0e Doc typo fix. No code changes.
FossilOrigin-Name: e841b142f583e49350c7cdd14dc4da72434cc67c7b521ae741e03e37e785d7e0
2023-05-10 21:08:01 +00:00
stephan 161f742856 Resolve the JS-side corner case reported in [forum:7774b773937cbe0a | forum post 7774b773937cbe0a] by not caching oo1.Stmt.columnCount.
FossilOrigin-Name: c3d25c3a25e79e01c4ad0cf11f7dc3b0fb1932f3bcd04935a728ef62f7e07cf1
2023-05-10 21:06:02 +00:00
drh 28f4e7b286 Enhance the split-sqlite3c.tcl script so that it uses single-character
extensions on subordinate source files, even when the number of subordinates
exceeds 9.  This is not yet needed, but might be soon.

FossilOrigin-Name: 4150e416263f24efcbfe68d5b1e15ec7e79df41dff0b6dfdc59f06ee9e205049
2023-05-10 16:04:04 +00:00
drh 0051a56dec Fix harmless compiler warnings from the CBMC compiler.
FossilOrigin-Name: 5bc17cbccdd369486fca049be9d7457e18f162c0e2f5475809ffc8f01c5fa9d2
2023-05-10 13:56:32 +00:00
drh 5af043c2e7 Rename sqlite3IsTableConstraint() to sqlite3IsSingleTableConstraint() and
improve its header comment, in an attempt to make the code easier to reason
about.  No functional changes - should generate identical machine code.

FossilOrigin-Name: 5dae897431a0a9dbb354c4a8a48f935ea7438035d96f90b83dd81eae434c8277
2023-05-10 11:05:59 +00:00
drh d733ee5cca Add a missing comma to the documentation for the OP_Jump opcode. No changes
to code.

FossilOrigin-Name: 7e2c2b1cee760af68cf303ce9ec1babbaf607fe54e043893c046d2d37cd0ab46
2023-05-10 10:03:28 +00:00
drh 88a99437b9 New restriction on the push-down optimization to prevent an ON or USING clause
on the left side of a RIGHT JOIN from being pushed down into a subquery that
is on the right side of that RIGHT JOIN.
[forum:/forumpost/a7d4be7fb6|Forum post a7d4be7fb6].
Also add comments to describe previously undocumented push-down optimization
restrictions.

FossilOrigin-Name: da3fba18742b6e0bd5290bee9d86a2d5cb1ff2de25d737ef93060d7c1143273f
2023-05-10 00:33:21 +00:00
larrybr 5fbb2adf7d Fix bug in fall-back logic of CLI .clone
FossilOrigin-Name: 0dcd9785c11e33e6393b7c1be1ff8fe7129f280439aed12de22d4687609a8bac
2023-05-10 00:20:42 +00:00
dan d9be387361 Fix recover1.test so that it works with DEFAULT_AUTOVACUUM builds.
FossilOrigin-Name: 4ffae48e831eedf8f5e61dc6d38817c0fdccfb2f4f1189d07f9722b9e3a48b5e
2023-05-08 16:26:22 +00:00
drh bbd0ae2e87 Improved error handling when ".open" fails in the CLI.
FossilOrigin-Name: 38544b11f0e19cc6c6f8230a89d28e36c7c3587481deaac6cedbf82338ca0d47
2023-05-08 13:47:38 +00:00
drh addc7e0e5b In editPage(), if an attempt is made to edit an empty 64KB page, then take
the slow path through rebuildPage(), in order to maintain strict invariants
in the editPage() code.
[forum:/forumpost/0f37727c0d|Forum post 0f37727c0d].

FossilOrigin-Name: a865e574e510c1ca192d0bd5293fb583def90f7ed086af2c89ca856ce93d115b
2023-05-08 13:38:14 +00:00
drh c42bbfc550 Ensure that the label for a JSON object really is a string and not an array
or object that just happens to end with a string.
dbsqfuzz 32f62ed8676d0a153dbe756aeae3202d3670fe50.

FossilOrigin-Name: c335737c2889722ea05b732a5efff8a2f31140c18ca038c41e05c2a26e0db5bd
2023-05-08 11:11:08 +00:00
dan 6d841fdc89 Add tests for recovering data to a lost-and-found table from a utf-16 db. No code changes.
FossilOrigin-Name: 832f6e9f797eecb9a3dae4e7e20c70eda1e91372e9d3ce4223d57f26582cdbd2
2023-05-08 10:51:35 +00:00
drh 28f32bedd1 It turns out that pagerExclusiveLock() can be called with the lock state
already set to RESERVED if the SQLITE_FCNTL_PERSIST_WAL setting is set and
a specific sequence of multiple journal mode changes occur.
Enhance pagerExclusiveLock() to deal with this.
[forum:/forumpost/8130545bc6|Forum post 8130545bc6]

FossilOrigin-Name: 2bb8d977392f635515aa4a36f6f763a2e4858f7adc1120519e2e74c04a9749b5
2023-05-07 03:23:32 +00:00
drh 4e73863fd4 Fix a comment in the JSON logic. No code changes.
FossilOrigin-Name: 3f30ea5ec9d849c64d8c38f4fffbc06c749ecff91ab90bc29b3a3cc9ec5b9df0
2023-05-05 20:22:06 +00:00
dan 77eb3e305c Fix a buffer overrun that could occur when using the format() function to format a very small real value with the "," modifier.
FossilOrigin-Name: 910e770ad4d8e8e45bf069af963f2e975bfcfb882578dc5fe714cd2396258934
2023-05-05 19:36:13 +00:00
dan ed96436f23 Reduce the maximum depth of nesting in json objects to 1000.
FossilOrigin-Name: c7697a0d45bfab20ec09f17ad65e375ddb43af6762278481c13a65c9a784978e
2023-05-05 15:52:44 +00:00
dan 3e49355e17 Do not attempt to run json502.test with SQLITE_OMIT_VIRTUALTABLE builds.
FossilOrigin-Name: 6664850647cd314c076842df5bf94e4f12d9be7fb56795b2af25f15c1267fa4d
2023-05-05 15:28:35 +00:00
drh e030619109 Updates to compile-time option usage and especially to the compile-time option
test script in tool/omittest.tcl.

FossilOrigin-Name: fece588b186c4f9f76d626313e35336fd5681e966e9bd0fa1053b147c4e3c315
2023-05-05 14:16:31 +00:00
drh fb18d81038 Fix the pager2-3.1 test case so that it does not run if a required
dependence (shared-cache) is omitted from the build.  Fix for
[forum:/forumpost/7a028538a7|forum post 7a028538a7].

FossilOrigin-Name: a77c8a780f884d0cb01d20912cf1aeec0bb7f3999d23117fe5bd82bc4405d483
2023-05-05 14:14:10 +00:00
dan cad6aae45e Allow virtual tables to prepare writes to shadow tables in defensive mode from with xRelease() callbacks - even if the xRelease() is invoked from within sqlite3_finalize().
FossilOrigin-Name: a17106494a8a67bc92cfe27ee8ef9bf160517d6930d796980ae12287b98c1cfe
2023-05-05 11:47:31 +00:00
drh 0498caec89 Improved formatting of comments on SQLITE_CONFIG_xxx defines as those
comments are used by documentation.

FossilOrigin-Name: 2cb840bc9df81f76699f742e5db03835f4c272548cca0e6d24412503db3d76b3
2023-05-05 11:33:59 +00:00
drh d2bef86709 Add the "subsecond" modifier to the date/time functions.
FossilOrigin-Name: 6fcb5a941fba0449d017954590bde7da74e5a1f8fd9b3e1febdff9bed1f22051
2023-05-04 20:28:19 +00:00
larrybr 88813681c2 Add some tests of subsecond modifier for date/time functions. Sync with trunk.
FossilOrigin-Name: 6499ebff545e663198bd0534be205a4e9ca68c7fb20fdcfa54fae4d9b79bfe3a
2023-05-04 20:19:33 +00:00
drh 6242c9386f Enhance the format() function so that the "," modifier works for floating
point numbers in addition to integers.

FossilOrigin-Name: 83683e108bce83c105bbcce1aa62880d8f14ebf3383d87d83a1e5c85026a7817
2023-05-04 14:44:53 +00:00
dan bf1305334f Merge latest wal2 changes into this branch.
FossilOrigin-Name: b2e0800b24f8c676e189d63abd77ca45a972de9722ece5de6efe2db6ede75cbd
2023-05-04 14:41:10 +00:00
dan d17b9114aa Update this branch with latest changes from trunk.
FossilOrigin-Name: 49777032f29517d23c8c7483536f8ba828e7000dc303415da6881cc458620be2
2023-05-04 14:31:17 +00:00
drh e8468098a0 Add support for the comma (,) modifier to %f formats in the format() function.
FossilOrigin-Name: 7080e196a1f887640ff51ddc508ec6796ce12874c2944855702753b64a8e5e50
2023-05-04 13:07:49 +00:00
drh c1e40a3a02 Fix a bug in cursor hints that can cause references to tables that have not
been opened.  Cursor hints are intended for use by COMDB2 only and should not
appear in production builds, so this should not be a factor for the vast
majority of users.

FossilOrigin-Name: d3370d59cffb7ab982d6c620c93d22aa6a9dc786e1c4af95ca8d45ff0b9b7d6f
2023-05-04 11:29:15 +00:00
drh 62b28db5c7 Add another assert() to FTS5 to demonstrate that a pointer is never NULL.
FossilOrigin-Name: b8a84a1bf53fa07ef01c57c6db6329ba439774a262b3adcbe94e7bd77525e296
2023-05-03 14:10:05 +00:00
dan 3f23eb6813 Add an assert() to fts5_config to ensure that a potential OOM is being handled correctly.
FossilOrigin-Name: fe9c207657400f9d9f4e822eb658157bc147ed538e2701322f6f973933f023ed
2023-05-03 13:57:57 +00:00
drh 3442306989 Protect a macro argument with parentheses in FTS5.
FossilOrigin-Name: bc07fe51fe0c6bb50ca8ae1baefcc35c8f5395b2d0de641bf0b0cedc92d754d4
2023-05-03 13:48:33 +00:00
drh dda2833a4e Add a comment to the [0512f82a2cde7447] change to link it to its TH3 test case.
FossilOrigin-Name: 1281d4d9d799ef83e6eb51d656ede82b4503d39adf169deb936b30ebfddd88b1
2023-05-03 13:33:37 +00:00
drh 69e2422973 Enhancement to showwal such that it shows what the page hash should be
on a hash mismatch.

FossilOrigin-Name: 960dc97083ed9a7bfa912010b9ea19114f51dd10e73d5c5bf06cc642ebd1a2e7
2023-05-03 13:03:30 +00:00
larrybr 6539977cfd Conform CLI double-quoted escaping to what its doc says.
FossilOrigin-Name: 910535ba518f35ecc64c980ad67e3af69008d1995235f52ab12db6cefdb6602b
2023-05-03 11:27:15 +00:00
drh b4824b055c In the zipfile extension, leave the catalog list in a consistent state when
the last entry is removed from the list.
[forum:/info/f03f1e4c5a5c9959|forum post f03f1e4c5a5c9959].

FossilOrigin-Name: 1b489d008b7bda12f9bf92e246bc613ef61e9c74ace0652d6d088e5a6e0696f6
2023-05-03 07:30:15 +00:00
drh ce2aa8bebe If the page size is wrong on the page1 content inside a WAL file, then
running VACUUM could trigger an assert().  This check-in changes that assert()
into a branch that returns SQLITE_CORRUPT.
[forum:/info/4ef9b37d74d2d2e8|Forum post 4ef9b37d74d2d2e8]

FossilOrigin-Name: 0512f82a2cde7447cb348d9fd620f39f2f0e359141903cde20f7ba30b5d34af3
2023-05-03 06:48:08 +00:00
drh 681307dc80 Improved detection of excess recursion on arrays and objects in the JSON
parser.  Fixes a problem detected by dbsqlfuzz.

FossilOrigin-Name: d40fd5924adaa8d6b1dd6b9a4087f64d496cf60096ae11c9229c59309c0d4844
2023-05-03 06:38:03 +00:00
drh 790adfd8ec Do not overflow the Index.aSample[] array if the same index appears in
the sqlite_stat4 table under multiple names because it is a WITHOUT ROWID
primary key index.  [forum:/info/537d8ab118df7edd|Forum post 537d8ab118df7edd]

FossilOrigin-Name: 9350a25ac0b55a6b901bc50e4db6d4e883c2617e1d2a8fdc90effabe52bb0012
2023-05-03 05:00:10 +00:00
drh 8777bdb78a Do not even attempt to load the sqlite_stat4 table if the use of STAT4 is
disabled using sqlite3_test_control().

FossilOrigin-Name: 0bf94c77d97582be5368bcfd149f3db7b9f928b4684aaa2626d98a2bdee8f96f
2023-05-03 04:21:31 +00:00
larrybr 164b332d80 Fix CLI mishandling of OOM during .import _of_ a DB. [forum:7ed4a250d3|Reported by forum post 7ed4a250d3].
FossilOrigin-Name: 30da3f0ebd944754881bce678c61289fdaf1e32198d241cc6b5c003e4fb33c0e
2023-05-03 02:32:42 +00:00
drh b504623802 In the CLI, ensure that the argument to quoteChar() is not a NULL pointer
due to a prior OOM.

FossilOrigin-Name: 776d1a47aae3c3679a2d158595e84facf118adf2d1bca0f0642311dd346479b8
2023-05-03 01:25:37 +00:00
dan 9cf09b5672 Fix a problem with the sqlite3changeset_size() API and rowid tables.
FossilOrigin-Name: 92ade220dcf5c1b76a5db605b1467a01051271957cd26f790755904fd82772eb
2023-05-02 20:34:15 +00:00
drh 6c9cc2c4f8 Merge the latest trunk enhancements into the wal2 branch.
FossilOrigin-Name: 0215dcc76d7b3a1dad10b8e6fd8b54f2ab5006a156db5cee992ab2dac98d9284
2023-05-02 19:58:35 +00:00
larrybr d8200de04d Give CLI quoted args hex escapes (as documented.)
FossilOrigin-Name: 90e434a6ae370a35109802fc336b98298d345f3d23125ef2cc68c31d8e69bb50
2023-05-02 19:33:27 +00:00
larrybr 8594f02c5d Cure CLI hang on .import with high-ASCII column separator.
FossilOrigin-Name: aac8ac631de5c5b439c48c13f5fd271d76c11b1a5fd0919dfe4506ef7725f5ab
2023-05-02 18:03:25 +00:00
drh bdb40cb18f Allow for lower-case "nan" to mean NaN in JSON.
FossilOrigin-Name: 6bee0a19e2efb6e39097fb22602556791d3d3962c50a1a0471c39a0c61db632e
2023-05-02 17:33:34 +00:00
drh d6b3a02d21 Interpret negative arguments to sqlite3_sleep() as zero.
FossilOrigin-Name: 2b542326aa8e7e1f6ed9d999b95ae5340278e0821277a17de3cc803b4af793ae
2023-05-02 16:34:15 +00:00
drh 8c79ea8447 Improved handling of NULL arguments to json_valid() and
json_error_position(). [forum:/forumpost/06c6334412|Forum post 06c6334412].

FossilOrigin-Name: ab78e6946ba8125a83ff155561adf9c8df4d1c09a645fce08c8245ddb08bfa3b
2023-05-02 11:12:01 +00:00
drh 67a17779cb Fix typo in comment. [forum:/forumpost/3da7d9c445|Forum post 3da7d9c445].
FossilOrigin-Name: 41a8a15c4b6f6157b12d43976e300763ef8a4963986d2806557ad0d8b0a784cd
2023-05-02 10:22:05 +00:00
drh 53ffe6e21d Minor code change in the CLI to make a UAF warning from scan-build go away.
FossilOrigin-Name: ad6aae768ad0816bfd1c8e9db5c0153226982d3af9a089f31e608223339e9829
2023-05-02 00:33:45 +00:00
drh f9b58eb4f8 Fix harmless compiler warnings.
FossilOrigin-Name: 342af5b4fa0bd7c699e5497161db13d0cf795c7a5875ae30d666122e518f213b
2023-05-01 20:42:15 +00:00
drh 0f97dc28b0 The gcc definition of the SQLITE_INLINE macro is not compatible with
string ANSI, so disable it if the -std=c89 command-line option is used.

FossilOrigin-Name: 62d703d83cf8cf3358715792347c49315a82c659e475158e385746f4329a4f39
2023-05-01 20:09:52 +00:00
drh 1f02cd6cd6 Fix harmless compiler warnings in FTS5.
FossilOrigin-Name: f6210017b3143ccbaea23655852f947f779bb438ad4fbf582f5da42f7cd1d9f5
2023-05-01 19:59:10 +00:00
stephan 58d40363b9 Remove xFileControl() handling from the OPFS VFS altogether, re-routing all syncing through xSync() instead. This eliminates superfluous syncing introduced in [a371374148a2], as discussed in [forum:647d2f811dbc2dfe|forum thread 647d2f811dbc2dfe].
FossilOrigin-Name: f809de7f232c8c2731a877c7bf1ad39fda5b326b2e45e4748cab4dc5e5b535bd
2023-05-01 19:28:07 +00:00
drh 59fe13e39a Fix harmless compiler warnings.
FossilOrigin-Name: 0a926994691ccb02ea4f0ba3c2e8f728871ae67d24c705fbc0f7af81f38d5f47
2023-05-01 18:52:35 +00:00
drh abdcfcef83 Add support for JSON5.
FossilOrigin-Name: f8c3ed23a6931b1da3b93b3274b132387078112a5c8e8d06b5312c47987d3937
2023-05-01 18:28:48 +00:00
dan 5ecee3dae3 Fix typos in comments in sqlite3session.h preventing documentation from being correctly generated.
FossilOrigin-Name: ab75170d5609c477613466e8880f20b74b0069281db6536fe09db06ea9d7cff3
2023-05-01 15:59:20 +00:00
drh 4cc6f869a4 New enhancements to the sqlite3_db_config() constants documentation.
FossilOrigin-Name: d74011a3c495719fe1816e15251269824396ac2a40e41f7b96f0dd507c9be609
2023-05-01 15:57:46 +00:00
stephan 043353483e Expose the SQLITE_DBCONFIG_STMT_SCANSTATUS and SQLITE_DBCONFIG_REVERSE_SCANORDER sqlite3_db_config() options to JS.
FossilOrigin-Name: 0a7024af3f0e1dddc1a60c67bc4084e46b297dc16fe753bd5e6a45a59fd1cfa4
2023-05-01 15:42:59 +00:00
stephan d45bbbb05c Implement full xSync() for the OPFS VFS. The previous implementation was not correct for journal files. Reported in [forum:647d2f811dbc2dfe|forum post 647d2f811dbc2dfe].
FossilOrigin-Name: a371374148a2874be6e48890781aa5952229056b146a50fa4d035693341c5636
2023-05-01 15:15:11 +00:00
drh b5fab68162 Fix typos in documentation comments in sqlite3.h.
FossilOrigin-Name: 6a8b00788e7911287a8287277eb603e983e3324c9d62de0889ce6f0b2d848f4a
2023-05-01 14:50:54 +00:00
drh 908dec7404 Do not apply the "AND false" optimization if either operand comes from the
ON clause of a join.  Fix for the problem identified by
[forum:/forumpost/96cd4a7e9e|forum post 96cd4a7e9e].

FossilOrigin-Name: d095da0e7a24e3bcab6495d964f76a86d7a5910d2d6edddc6e8092bfa6084fe6
2023-05-01 11:24:35 +00:00
drh e210c93903 Fix abbreviated paths for objects such that they work even if the object
key begins with '$'.

FossilOrigin-Name: 1b991c78141a9915ae9350ecb347a758e50d7d25c8a0f4cc098ae10d47c27043
2023-05-01 03:56:48 +00:00
drh 5196d93505 Fix a problem with json_patch() when one side or the other is JSON5.
dbsqlfuzz bc10593a4ba8e7a7862593532285be31f00f8e41

FossilOrigin-Name: e18c0899cc774f6f20d2a73381fa3ab3af9355cf4d108c7612db259eadbb8b96
2023-04-30 23:52:55 +00:00
drh dae7ae359e Accept the full ECMAScript 5.1 IdentifyName syntax for keys in objects.
FossilOrigin-Name: 9be2c87518b33713210e3e2fa56924888e19415c71329141d18150b275f6f25e
2023-04-30 20:37:49 +00:00
drh 4a398c317d All JSON to understand floating point literals "Inf" and "QNaN" and "SNaN" in
any case, without the SQLITE_EXTENDED_NAN_INF compile-time option.  This
extension is always available.

FossilOrigin-Name: fb551145e0d84213b3343dc1bc7db70c898b9dea24a72b968240617f4b52d821
2023-04-30 19:45:25 +00:00
drh 7be1473ccb Omit the json_valid() function. Change the name of json_error() to
json_error_position().  Use "NOT json_error_position(X)" as a substitute
for "json_valid5(X)".

FossilOrigin-Name: 34c4e900a9cc51630eeaf01deef74bf5b18d66e0ab1dc61a2023ac8f837a5197
2023-04-30 19:34:41 +00:00
drh 5a5cba0692 Merge all the latest trunk enhancements into the JSON5 branch to
facilitate testing of the JSON5 branch.

FossilOrigin-Name: 30d12edebad9b097cd5f0da355304d1cb2f8b70d7c7dff378fd7ad7c8ebf9279
2023-04-29 18:40:48 +00:00
dan e01e3b5b94 Fix a crash that could occur in fts5 'secure-delete' mode when operating on corrupt records.
FossilOrigin-Name: 2e85b0e3dcae0915aa6472a3654c8ac72a6b2083c11747f3f657c79bbdaf530b
2023-04-29 18:31:44 +00:00
drh 440f7b4833 More ALWAYS() macros.
FossilOrigin-Name: 770b09f7a795956be63a06847059370db8dfc88654878d742b7b826947029962
2023-04-29 17:35:29 +00:00
drh f8ae2d2a2a Add ALWAYS macros on unreachable branches in the new JSON5 logic.
FossilOrigin-Name: 91e15ed9d93e51b750172a074dbbcf206128e9476571ce6189b8ea0ba5ab4135
2023-04-29 17:13:39 +00:00
drh 399875f6c5 Simplification of the logic that normalizes JSON5 integer literals into
canonical JSON integer literals.  Improved reporting of OOM.

FossilOrigin-Name: 01ee613c07fcb87e7d7b7f1b1387982715d1343418f37f4a1dc90e43a76d20e8
2023-04-29 16:31:08 +00:00
drh d76d954de3 Do not allow leading zeros on non-zero numeric literals in JSON.
FossilOrigin-Name: 3e91494390ba88498eb243f61ce4ef4efa23b58326108a769bc72331d7d7d75b
2023-04-29 16:00:20 +00:00
larrybr b608d7e2c6 Cure CLI generate_series() overflow bug (forum:754e2d4db2a5|noted by forum post #754e2d4db2a5) and bring behavior with negative step arguments closer to as-documented and eponymous function in PostgreSQL.
FossilOrigin-Name: 07383758d68e05021ccd393a69c1fa94836dfb02331326c082495a531a384363
2023-04-29 15:42:46 +00:00
larrybr 29c2f0366d Replace duped generate_series() test with another corner case.
FossilOrigin-Name: fb2f08783fdfef4d5123ad66792de2265d39272d40d7bf1be7dd21876e8ffd73
2023-04-29 15:29:29 +00:00
larrybr e0a3e4bd51 Sync w/trunk, improve generate_series() variable names.
FossilOrigin-Name: ecbf7e13e6454ee879c178c53ffa6face789e93ccbebb0cfac8fde72cb3a4952
2023-04-29 12:29:15 +00:00
drh 64953f0fb8 Minor fixes to to the JSON% parser.
FossilOrigin-Name: 2fe684cdcdc3cab4ec3348ca5aa5948e4472c562b739c29faebcb77397f8d969
2023-04-29 12:13:27 +00:00
drh f176776bd7 Correctly recognize an isolated U+feff as a space character in JSON5.
FossilOrigin-Name: 4473dc8e3ad18bb6185ed4a819baf881a7d26ac74bc70fae6ba23a0030be8316
2023-04-29 00:59:22 +00:00
larrybr 7667f5f12f Add tests for generate_series(), and change an existing test that enforced varying rowid-to-value mapping with query ordering. (That no longer varies.)
FossilOrigin-Name: 9605db4ae37ef383f8e5568af4543a9e6f336bace6ed80ff80fe4c0b0f437078
2023-04-28 23:39:16 +00:00
drh 8f3fe2ead2 Bug fixes in the logic to skip over JSON5 comments.
FossilOrigin-Name: c736b77a2fc5f8f2ba8f770adf05443dfdc3a9847ee519f411e661f76756d611
2023-04-28 23:38:54 +00:00
drh 6e2de57963 Bug fix in the offset computation for json_error().
FossilOrigin-Name: 8f7ab5d9210490eb451534df246b82081a5aa252a1bd662659df3da429290837
2023-04-28 23:19:11 +00:00
larrybr b0d46fcd72 Revise generate_series() extension (in CLI) to address overflow reported in [forum:754e2d4db2a5|forum post #754e2d4db2a5] and to make behavior better match the like-named PostgreSQL function.
FossilOrigin-Name: beeea3e1b010dace9789f27172462b912819d0f8142a67e3e1e7335211e0e9a8
2023-04-28 21:25:03 +00:00
drh 8e7efe3764 Improvements to the accuracy of json_error(). Add the extension SQL
functions random_json(SEED) and random_json5(SEED).

FossilOrigin-Name: 8d09dc1c45a8026b94f70273d064e47939f30cadedc17548b5a26ba054a8d3a7
2023-04-28 17:38:35 +00:00
drh 272ae627c5 Add the json_error(X) function that returns the 1-based character offset to
the first syntax error in JSON5 string X, or 0 if there are no errors.

FossilOrigin-Name: 901ad995d5a722ca2672516205ff488e9acd703a828ca5fc43f11fca5f2af120
2023-04-28 14:48:11 +00:00
drh 016f46666b Fix indentation and omit trailing whitespace in the random JSON generator
script.

FossilOrigin-Name: 629db09fceb7bf37561b52ccee06ebf4df261291e9a8ffcca82b243f6db5ff07
2023-04-28 13:25:35 +00:00
drh 1170fb52e1 Fix json_tree() so that it is able to deal with the JNODE_RAW labels of
a JSON5 object.

FossilOrigin-Name: f56528d413d8e622f7c4f18b2f9f2e620bfb441c020461299b35a90072ee6c13
2023-04-28 11:02:01 +00:00
drh ea2529528e Merge all the latest trunk fixes and enhancements into the json5 branch.
FossilOrigin-Name: b5ca15cfc19380cf870b70be6a86e70f2026cc3d6d89005b45891d58c4f11c2d
2023-04-28 10:23:01 +00:00
drh 90f549a0cf Do not assert() a bad string representation in an sqlite3_value after
an OOM.  dbsqlfuzz c822a17a23c524a0ac7cfb203c7198209da15de8.

FossilOrigin-Name: 91fee79a01971259b21478e60a069a711a00efc79ddfececa6224a152cd8d09a
2023-04-28 10:10:52 +00:00
larrybr fb016bf257 Make CLI .load slightly harder to abuse.
FossilOrigin-Name: 2779f9270cc431786d0e16ef05ec05b0f22dda2307f3bf17467ad0b25304e160
2023-04-28 00:28:06 +00:00
drh fe03773305 In the CLI, early out from the output_quoted_string() routine if the
input string is NULL.  [forum:/forumpost/8a22c0bfa7|forum post 8a22c0bfa7].

FossilOrigin-Name: 2881978d7b7014738a21c590b0430c4fa40e6b00b9547d0284b7656124523ae9
2023-04-28 00:19:55 +00:00
drh 93b4c3beb8 Use a new technique to detect fresh OOM faults in columnName() that does not
rely on there being no OOMs prior to entry into columnName(), as
[forum/forumpost/fb6811c2f9|forum post fb6811c2f9] demonstrates a technique
which could cause an OOM prior to entry into columnName().

FossilOrigin-Name: a63346d6a0c0ca7ba4c87499de2e461be9c77e9b5d98f2bebf308cdb6599f33c
2023-04-27 23:59:51 +00:00
drh bfd28f9e01 Remove a faulty assert() from the CLI.
[forum:/forumpost/726c4f7db0|Forum post 726c4f7db0].

FossilOrigin-Name: 4bbebb6bfb9910265d91b777c1711b3b8e0732bcf299f7459b20c4ea110422bd
2023-04-27 23:44:38 +00:00
drh d5de1f39b5 Remove an ALWAYS() macro that can in fact sometimes be false.
[forum:/forumpost/f0e872fcee|Forum post f0e872fcee].

FossilOrigin-Name: d85880e4ed89e2198beea642818051f7a91a71fea9929be0a7ed0603b493ce47
2023-04-27 23:40:03 +00:00
drh 52da6d2607 Test cases added, and some bugs fixed.
FossilOrigin-Name: bc84a82e4ddc1b71025c56c49e62a44f0b12fa87a6417ad61967d9d3121a0d4e
2023-04-27 23:29:09 +00:00
mistachkin 263b86c54c Fix compilation issues seen with older versions of MSVC.
FossilOrigin-Name: 4ba2a651120391d779a3db9881c0fb17e1322cc9d8738f915a46e31a2f313578
2023-04-27 22:17:12 +00:00
mistachkin cf5dce045f Adjust OSTRACE usage so that it works with the changes in the previous check-in.
FossilOrigin-Name: 1eb4573e9092d350d00d05ba716cb0272e2dbb1ecbaadc2a7cc22de0b8cc2bda
2023-04-27 21:31:21 +00:00
drh ae31f48826 Further optimizations to the JSON parser.
FossilOrigin-Name: bb8f1c16f244f893170f3d03bc445bd15fc337804c7c3e76c548397f5b95b39a
2023-04-27 19:30:53 +00:00
drh 1371df2117 Faster implementation of numberic literal parsing in JSON.
FossilOrigin-Name: 58398292e8473b0b9e4b77dd5ed27334bc24a85d0c399e8d0b86de6fb59dadce
2023-04-27 19:13:20 +00:00
drh 49d8e0ef97 Add support for hexadecimal integer literals in JSON.
FossilOrigin-Name: 85e00c9e68d0695592e8f72555ee133c096bfca5a860a8e21d1e0ef756705aaf
2023-04-27 18:28:10 +00:00
drh 47dd0e735d All floating point literals "NaN" and "Infinity". Additional variants
of these literals are available if compiled with SQLITE_EXTENDED_NAN_INF.

FossilOrigin-Name: c13346afbecb92275e741252897d00478dab4be2d158889bc735e80efd9444f5
2023-04-27 17:32:29 +00:00
drh 8ded164200 Fix handling of reverse solidus in string literals. Allow decimal points
in floating point literals to occurs and the beginning or end of the mantissa.

FossilOrigin-Name: d92a6ab2871095ac66c60cfa15dbafa7b762f83d287d452f61792eb30cf5b26b
2023-04-27 16:57:14 +00:00
drh dbcea23976 Fix the handling of escape solidus in the JSON routines.
FossilOrigin-Name: 676877aca235e620ee12d10235dd6ad009d4968455ec170daeb1998b94a7e0a2
2023-04-27 16:24:12 +00:00
drh 13f8a80d94 JSON string literals may span multiple lines by escaping new line characters.
FossilOrigin-Name: 66da4bd4a30c390fa1a7960ce2edaef82e63971ecf33ffb6b906db9f278041c5
2023-04-27 15:48:58 +00:00
drh ef3a162be6 Allow the labels on JSON objects to be unquoted identifier names.
FossilOrigin-Name: fb428db3f64f148ab9a3478fdcc8b3733e58102b3c8895a482e2551d974d5661
2023-04-27 15:11:26 +00:00
drh 0d8579120a Performance optimization in the JSON parser.
FossilOrigin-Name: 5a88ba743f55d45b1c0ce0090bb3b396bcf7fcf7b3bcb989aaf30b8bb772599e
2023-04-27 14:38:51 +00:00
dan 2e225a35b4 Fix a problem on windows sometimes causing SQLITE_NOMEM to be returned if an error occurred while opening a temp file.
FossilOrigin-Name: c22a39a6b28e3039fe453c0c0db98744b8a63ef9063cc3f6e0ed67d2506d34be
2023-04-27 13:55:29 +00:00
drh a549e7a527 Permit JSON5 whitespace in all contexts of objects and arrays.
FossilOrigin-Name: 93f3ab26b57c0469862a56e97d4b3c796b27f9f582046fcff1f2aa8d8910c550
2023-04-27 13:44:26 +00:00
drh f59c01e24c Translate JSON5-only string literal escape sequences into the JSON equivalents.
FossilOrigin-Name: 14e82f36eed31af1237898728bf353b968523c62b1f8d1d90dbbabd92d0c2834
2023-04-27 12:24:00 +00:00
drh c5ee2d8e80 Implement some of the JSON5 enhancements to string and numeric literals.
This is an incremental check-in of work in progress.

FossilOrigin-Name: 9508efa9d61c0ff0eb73100dd52889dadc5fa2a2091b944a9b8a74e8a5c50e82
2023-04-26 20:26:14 +00:00
dan 6a9ba9a8c2 Fix a typo in a printf() format string the sessions module.
FossilOrigin-Name: 14d1c4a828660bc137dcea1bb4cc439e1ed3872867a0dab486f6a639ef3bcc45
2023-04-26 18:50:49 +00:00
drh d924af2325 When changing a large integer into a floating point value, cancel the
string representation.  Fix for
[forum:/forumpost/5c74a3bc4a|forum post 5c74a3bc4a].

FossilOrigin-Name: 3e2da8a7e35c839128d26aac575605e1e34889e8ab3484440bdd65c4d752c6bb
2023-04-26 18:23:47 +00:00
drh f14b2e35fa Partial implementation of JSON5 numeric literal extensions. Use a switch()
statement in the parser for better performance.

FossilOrigin-Name: 78404dc37024cad5fe7eacf78ea85b56f08b129a1b9a046c3e1b11275068a485
2023-04-26 17:30:28 +00:00
drh 5255102e04 Improvement to the way the JSON performance measure scripts work &rarr; keep the
test database in the directory of the test, not in the source tree.

FossilOrigin-Name: ac411dbdcbf0b9040fb5b7de173271f383a6aa303d57f22ebd200809a5b4a6d3
2023-04-26 15:58:08 +00:00
drh f62518f65d Work toward implementing JSON5 whitespace. Untested and incomplete.
FossilOrigin-Name: d262c059455ebe0650a45a6c1c04d1baf9609c635df352732dd192426e1bdc39
2023-04-26 15:19:19 +00:00
drh ddc8b9ee0e Fix to the json-speed-check.sh script.
FossilOrigin-Name: d839c9544d7f28c1abc779eb2d40f95c1a9386984656fbd29d19b1e7830171bc
2023-04-26 13:52:20 +00:00
drh 02efa7be96 Add scripts for JSON performance testing.
FossilOrigin-Name: 3051d6a7c12cbf86634d8fab64f3f3bc9a1eb184260e56a82eb543cb634ba7ef
2023-04-26 13:25:30 +00:00
drh 058f3dbb27 The json_valid() function only returns true for pure JSON. JSON5 (or
at least that subset of JSON5 that has been so far implemented) is accepted
by all routines, but json_valid() still returns false for JSON5 inputs.
The new json_valid5(X) routine returns true or false if X is or is not valid
JSON5.  All of this is experimental and subject to change.

FossilOrigin-Name: 5d33ab77800765c8b3a13ffcc02ba8a348d71b2b425924560418b517d723494d
2023-04-25 21:24:20 +00:00
dan 69d98d9f93 Allow the sessions module to be configured to capture changes from tables with no explicit PRIMARY KEY.
FossilOrigin-Name: 5d4431bc44e206f48f3bcc1f51399b38bd4cb418e7e50a37495475fb8da04646
2023-04-25 20:41:19 +00:00
dan 77a159212b Remove unused variable ltoj accidentally added by [1783655e].
FossilOrigin-Name: faa1575ce0a39a2335a77057e8769a4e67aedd6326b4627988d03f0cb21fc39a
2023-04-25 14:54:18 +00:00
dan 0941ea8a62 Avoid assuming that an expression in an ON() clause that evaluates to zero implies that the query will return zero rows when the query contains a RIGHT JOIN. [forum:95849acbe1|Forum Post 95849acbe1].
FossilOrigin-Name: 1783655ea422185e75593b89e4ef452a6f5496aefd389f88ce7fe4b7d41d6a98
2023-04-25 14:37:12 +00:00
larrybr 38c2052767 Handle newline-trimmed input TEXT correctly in base64, base85 UDFs.
FossilOrigin-Name: 8f637aae23e6638c064a34262dcf16a3cdfd000fb1fa1b2a834b292fe6659408
2023-04-25 04:28:39 +00:00
larrybr 21461407f6 Check for OOM sqlite_value_x() returns in base64, base85 extensions.
FossilOrigin-Name: e6f9c0b1f963033a8e17d13935c5c6b12d263fe10c585035a3d1f1154c6ba5d6
2023-04-25 02:44:53 +00:00
drh be5bada4d0 Allow trailing commas in objects and arrays of JSON.
FossilOrigin-Name: 4031b231c223db598b45dbd192b027a99a9b82a981d43c75f723a3fb06720b82
2023-04-24 23:14:34 +00:00
drh 7eff609d8a Update the compile-time detection of architecture byte-order in the RTREE
extension so that it is aligned with the latest enhancements in the core.

FossilOrigin-Name: 122431d3a7267ec83768316ab146c0557fb6c0577a4a47ac6ed3d7aa6811ca9a
2023-04-24 19:23:42 +00:00
dan 655a1b7b58 Allow the sessions module to be configured to capture changes from tables with no explicit PRIMARY KEY.
FossilOrigin-Name: 8a612f0860126c0c8473b1e65fcabb9a8821d8bf09fdf3f6018acfc99df9af71
2023-04-24 19:22:21 +00:00
stephan 92df71a360 Add a note about the journaling mode in the OPFS VFS. No code changes.
FossilOrigin-Name: e79c95fc130fc302719690eb6391d96070aff825b2b51ef6c4ad459d9a8918d7
2023-04-24 04:25:42 +00:00
larrybr 022a042897 CLI to have "undocumented" dot-commands, not usually shown by .help
FossilOrigin-Name: 17f5dd2d2ae02a95180b9208b7de805922ba20271d3263e3193f0d46f4ec324c
2023-04-23 00:12:23 +00:00
drh 223c6b48a9 The sqlite_stat4.idx field is case insensitive. It should work even if
some entries use a different case than others.  Fix for
[forum:/info/6c118daad0f1f5ef|forum post 6c118daad0f1f5ef].

FossilOrigin-Name: f097ca70b5b967d1aadebd74ac5020813e00b7c30cc543814dbf5f359d1328f1
2023-04-22 22:32:19 +00:00
dan 4d24eff8dc Avoid a crash in fts5 when combining an empty expression with another.
FossilOrigin-Name: 33681ff3d29c9f096dbd8c5d885993f780e93974a7653adc00fa41a158356d7c
2023-04-22 20:47:08 +00:00
drh 2f251f4929 Enhance the ".tables" command in the CLI so that it is able to deal
gracefully with an OOM situation.

FossilOrigin-Name: 5157fd1cf753b1c01c6c00e9ac2712da2631a673837ed15ec3420ec869ee66aa
2023-04-22 20:22:17 +00:00
drh f661db3050 The assertion-fault fix in [53a61f7423a7f057] was not quite complete. This
additional change enhances it to deal with WITHOUT ROWID tables.  No changes
to deliverable code.

FossilOrigin-Name: 4d5af42c651bc1655a4f09b743f2a1424f19c1e4248e041b3fd4a042fd1f9799
2023-04-22 20:07:48 +00:00
drh 57114c256f Add the --unsafe-testing command-line option to the CLI. Without this option,
features of the CLI that are intended for testing and that might result in
incorrect answers, assertion faults, and/or corrupt database files if misused
are disabled.

FossilOrigin-Name: f28256a96af4263c0caded79b7481b88e9c7e7f1af7dbb3fd5e22ff07fbedaf3
2023-04-22 17:41:53 +00:00
larrybr e945b29359 Add --unsafe-testing invocation option to CLI. Needs some tests added and changed.
FossilOrigin-Name: b3d9ac052d5c2dd1afeeeb5c9cfac9dd91a1b8d6a74a2ef10aa2037ca505abce
2023-04-22 16:46:38 +00:00
drh a9c4c82ef1 Fix an incorrect assert() statement in btree.c as found by
[forum:/info/d03345d572713fe6|forum post d03345d572713fe6].

FossilOrigin-Name: cd485b302c54aef066d751a153df34eea0cd23e54a414e291f636ad9929fe78a
2023-04-22 12:47:16 +00:00
drh fb50f96121 Fix a incorrect assert() statement in the pre-update hook logic. No changes
to production code.  Fix for the problem reported by
[forum:/info/19b217bfe709a072|forum post 19b217bfe709a072].

FossilOrigin-Name: 53a61f7423a7f057568a1d8a1e63be01a2328997850dc32e3d8c6d51f1c5ab0b
2023-04-22 12:37:41 +00:00
drh bbe1986761 Fix harmless compiler warning in [1489e7f53a4863b3].
FossilOrigin-Name: da907dbc60b4f754845ae0bedc05cbda4cbfed9f468be3ffdc18860c763d9324
2023-04-22 12:11:31 +00:00
larrybr 1a2c7f9029 Also fix no-length string intolerance for CLI json mode.
FossilOrigin-Name: 3ac1984039a50445ff65db97468a5313d7f09db7b604755e444b77ac9346ea76
2023-04-22 11:33:27 +00:00
drh d9eb39e625 Early out from sqlite3Prepare() following an OOM to avoid possible
problems further along in the parse.  Fix for the NULL pointer dereference
reported by [forum:/info/2e5131839365682a|forum post 2e5131839365682a].

FossilOrigin-Name: f35ce7c122017009496b777f33e641d0be086abee6ea8e25aab4c3e0fb45731a
2023-04-22 11:29:36 +00:00
larrybr 8bfa190e9a CLI to handle absurd string length limit better. [forum:5180af725f1cc375|forum post 5180af725f1cc375]
FossilOrigin-Name: 1489e7f53a4863b316af6dee10d9c66491d1bed6cf52a3538ac118c03b1d7ef9
2023-04-22 11:24:52 +00:00
larrybr c40a7c82fa Avoid need for cast in CLI shell_check_oom() calls.
FossilOrigin-Name: c88550ef6711b03a81e2e6d121e2eb99ccff6e925c025dd6e85dd81088508400
2023-04-22 09:26:36 +00:00
larrybr 5870e942c2 Fix unchecked return in CLI .dbi command. ([forum 79e2cd1bc](forum:)79e2cd1bc)
FossilOrigin-Name: 81a74ee9d66ba23dfadd875e372ce96491dd6bf146e03e7fec8c3fd98e6be825
2023-04-22 09:14:22 +00:00
stephan 0e38c6e696 Change the return type of the ts_read/write() family of functions from int to ssize_t, per report in [forum post 947169d5e7](forum:947169d5e7).
FossilOrigin-Name: 7809e7ce6a70657b8ea239eb4778698f7986a658e9177a57b2fb7814c069c936
2023-04-21 15:37:33 +00:00
drh 11d5efde75 Add a new modifier to date/time functions: "subsecond". May be
abbreviated as just "subsec".  This modifier causes functions to try
to show fractional seconds if they do not already.

FossilOrigin-Name: 03f2a15e8779d8ca76510badeabde62a0539ce8a6368d7a9688f5aadc215b940
2023-04-21 15:30:47 +00:00
larrybr 88ed723490 Increase CLI tolerance of arifically low memory.
FossilOrigin-Name: 13f2638622871db9b91db547dcfd38f61c03e7696e4df5144adefcd86140aeba
2023-04-20 20:07:14 +00:00
larrybr 1829f34f46 CLI .ar made to enforce getting a proper subcommand.
FossilOrigin-Name: 104ab105ac0e96c0fc61c6d69179bf984fcfe5e0efb2c82f3742456c1b5fe5d1
2023-04-20 16:15:20 +00:00
drh 5eca6ffe5c Fix a function declaration so that it correctly appears as private in
the amalgamation.  [forum:/forumpost/ec8c77516c|Forum post ec8c77516c].

FossilOrigin-Name: 1864569bb9fe28a5e77f34b199f26ac272dbbb1a6642dacd7e50ddd29aa567b6
2023-04-20 10:40:37 +00:00
dan 9549245509 Fix a broken assert() in the recovery extension. [forum:/forumpost/82a88f5812|Forum post 82a88f5812].
FossilOrigin-Name: 8a1ad137606ce5bddbed62f89ff9a5cd922c6530db84c12b8b27c08df3c66bc6
2023-04-20 10:26:56 +00:00
dan 8cb4f0b1b2 Add tests for the progress-handler callback with the fts5 'secure-delete' feature.
FossilOrigin-Name: 48505ad950bc0902d58210be066d4672e6085eb27c525ba2bc663fde7e932ff7
2023-04-19 20:29:26 +00:00
drh f23adcbaa4 Improved rebustness of the pcache tracing logic.
FossilOrigin-Name: 0a43235b8368a0996d1856c4e956786a5321275458cb9179ce818aa47bb824ed
2023-04-19 18:36:49 +00:00
drh 2cc723a2da Remove unreachable legacy code.
FossilOrigin-Name: e3e7fb87d90690c29b70da261887c8f7d87b62188d15577e48db91232e13608c
2023-04-19 18:32:32 +00:00
dan 3115b36d67 Fix a segfault that could occur if a non-empty in-memory database was the destination of a backup operation from a database with a smaller page size.
FossilOrigin-Name: 020968f857d7b90bab3525fc9d6b859a019f6a80422c3d5ffb88f5bdb8e02a11
2023-04-19 17:07:35 +00:00
drh 2c5c124064 Do not remove pages from the cache of an in-memory database due to a
failure to acquire the page due to it being larger than the maximum page
size. Fix for [forum:/forumpost/a19bb49140|forum post a19bb49140].

FossilOrigin-Name: 982b35563da685dfdf50cbe4a7ae829d3b428e03587028df7efe520f819b1dc2
2023-04-19 15:35:45 +00:00
drh ec0ed68846 Fix ambiguity in the header comment to the sqlite3PagerUnrefNotNull() routine.
No code changes.

FossilOrigin-Name: d419e65eef2954ed759121254024028854b55e8c4f52ff7a8896c8cadfc57f53
2023-04-19 14:26:43 +00:00
drh 4c0fca00d3 Fix a debugging printf() in pcache.c. This only comes up on custom builds.
FossilOrigin-Name: f2d6756731caadf9a30214ab7d19717a2ca1dae2652bcc6b211f6b0830362240
2023-04-19 13:56:47 +00:00
stephan b669956637 Correct use of an undefined symbol in one of the OPFS VFS loading failure error reporting cases (copy/paste error).
FossilOrigin-Name: c81c968b458977b2d6dd2fdd3c3d667f0e7cee069f3715363d0e8aed4e6e8377
2023-04-19 13:39:13 +00:00
drh d9ae63144e Add pager debug tracing of truncate operations.
FossilOrigin-Name: 0daadf36229816abe73ff684b55ef4f8045ed314347f3a694404a2084decb355
2023-04-19 13:30:17 +00:00
drh e61aa230ce Attempt to suppress harmless compiler warnings reported by
[forum:/forumpost/fc98845c06|forum post fc98845c06].

FossilOrigin-Name: f6e6c5f94cef83645288568a344a0ff64e4ef17b10fab28996567a94db0a002a
2023-04-19 12:08:46 +00:00
larrybr cbba7e61d6 CLI to cease dumping back to OS shell on single ^C when interactive.
FossilOrigin-Name: ba8e4378f21d529a248d2d60b60974fef0d6612dcaaa1f6b7f8a6946fc46b39b
2023-04-18 23:05:34 +00:00
larrybr 837b661f2c Add comment on CLI ^C fix (because it is a bit subtle.)
FossilOrigin-Name: 19c062f1cd814a33bf9c10928217464cddaea7c3291ff44e605901a86b82a06d
2023-04-18 22:53:30 +00:00
larrybr 7b1a49c993 CLI to cease dumping back to OS shel on ^C.
FossilOrigin-Name: 7b1f635268bd5b093b22f832c3ac9e5d642d1aac5ebc620619be5a24930feb83
2023-04-18 22:28:48 +00:00
drh 359c5ac1fb Ensure that the VACUUM command is not confused by alternative encodings when
it is the first command run on new database connection.
[forum:/forumpost/09503b4d33|Forum post 09503b4d33].

FossilOrigin-Name: 8b0fe63f873661037351fc1eba6071b98b5c97a0fc8bd8a504a90d047f78bc0d
2023-04-18 15:21:55 +00:00
drh f84aeac274 Another fix to the indexed expressions in aggregate queries with GROUP BY
enhancement of ticket [99378177930f87bd] and implemented by
check-in [b9190d3da70c4171] to address a problem described by
[forum:/forumpost/f34e32d120|forum post f34e32d120],

FossilOrigin-Name: 5acc3ef83e16a9f2803981a6e61eca62fe177cac4eed1befe2ade3a5977a1eab
2023-04-18 14:13:54 +00:00
drh 053a29c291 Add the --fullsync option to speedtest1.
FossilOrigin-Name: 960fe54c8227a376c97bb2a3ba604d97e79e27e51a9798be9d318cf56c04fa99
2023-04-18 11:35:09 +00:00
dan a5eab9579a Fix an assert() in fts5 that could fail following an OOM or IO error.
FossilOrigin-Name: 49595083ba58e13d404b6a11c28f53768a257c59f3f36be3591438127df7e875
2023-04-18 11:33:35 +00:00
larrybr ab4af9c368 When CLI -utf8 active, do no translation to MBCS for output to Windows console. plus code style improvements
FossilOrigin-Name: 25edf6089724bf9f97c796e5abf838ec508ebc132895d8325620c16d63996fef
2023-04-17 23:59:15 +00:00
mistachkin acb4050cfc Minor coding style changes.
FossilOrigin-Name: 481aa3ccf6f12c2c2fd11a1a20d8fb59f4f9b1cd8dcff0586b4f52222463d720
2023-04-17 22:28:02 +00:00
larrybr 690ebc6908 When CLI -utf8 active, do no translation to MBCS for output to Windows console.
FossilOrigin-Name: cc1d4296d71ee6e2ba133edc581385e201c89657911c2ed39108843c776633a4
2023-04-17 21:18:53 +00:00
dan 68c0da6f66 Fix some problems with using fts5 options 'secure-delete' and detail=none together.
FossilOrigin-Name: 4d3f27ba90aa59837e49ceaae9f36cad426af3a33707d85d1bb848fc1bda6fed
2023-04-17 18:32:22 +00:00
dan 48824f23cb Fix problems handling corrupt records in new fts5 code.
FossilOrigin-Name: 2f63d4a8bac6dc72197df32efd84c148adbc4794943088dff08da57b15dd7900
2023-04-17 11:41:42 +00:00
dan 5c45a9fbaf Fix an assert() in fts5 that may be true when dealing with corrupt records.
FossilOrigin-Name: e8de2774463c07b76cd6b9a44ab32a583fe973f42a21b21af46fe82ab177c1b8
2023-04-17 10:39:51 +00:00
dan ae6f6ef3b0 Change a memcpy() to memmove() in new fts5 code.
FossilOrigin-Name: 8de4542553ae20947787b4daf5e88bbe69f815679960e700e19f9c1148234b08
2023-04-15 21:00:55 +00:00
dan 0e429cf934 Fix a use-after-free that could follow an OOM or IO error in fts5 when using 'secure-delete' mode.
FossilOrigin-Name: 904be83d9f3dd4b21a3aa47dde337b5c045b925ffdde4bccd1f8e8fdbef68c18
2023-04-15 19:58:58 +00:00
dan e45317a50a Avoid dropping the error code following an OOM in fts5 when using 'secure-delete' mode.
FossilOrigin-Name: 9ee24a499c7b4a048f269f1c03c7c0b22b2898a3c23ea3e2511c2ba0342afa30
2023-04-15 19:13:51 +00:00
larrybr a8e9a7763d Add -utf8 option to CLI, for improved console behavior on Windows.
FossilOrigin-Name: 414010d2366477285cb238f7af1e3d511ab4cda4bd4493441bd9868387dd5b66
2023-04-15 17:56:43 +00:00
dan 7dcc3dfc04 Add the 'secure-delete' option to the fts5 extension. For configuring fts5 to delete old entries directly from the full-text index instead of using delete keys.
FossilOrigin-Name: 394980e4fe12125460ab14da41edae9089a4da332a46f3124bf0b9778793791f
2023-04-15 17:47:32 +00:00
larrybr 9726db69e6 For CLI -utf8, set output codepage too. Adjust PP vars so that the code can be entirely omitted for targets pretending to be "WIN32" but not quite doing so.
FossilOrigin-Name: 543594a7277b12d1c58af8e3cba3841667cf924172995536489ce7da704a11d7
2023-04-15 16:12:46 +00:00
larrybr 5f347a426d Fix stray edits in comments in last check-in.
FossilOrigin-Name: fa3ce7f013d40147f012e26d0ede75b134174f144d314336310c9ac585e23fb7
2023-04-14 21:34:58 +00:00
larrybr cdb8a698c1 Set CLI -utf8 option and build with line-editing package to be mutually exclusive. Integration of console-invasive UTF-8 handling with line-editing takeover of console may come later.
FossilOrigin-Name: 047344a91583f273a55fe3659bb8020ffc947c05c4274fbda54ee0608e62adb1
2023-04-14 21:23:08 +00:00
larrybr 67f678138d Cure CLI double-prompting (by ditching gcc fgetws()), general cleanup. Work remaining is to avoid effect of -utf8 when a line editor is linked/used as part of CLI.
FossilOrigin-Name: 73a5f54231e9f6ad8f013df3987ea48c516080f9193ed873b56f982ee75658c2
2023-04-14 19:56:32 +00:00
dan 112500028e Further tests for fts5 secure-delete mode.
FossilOrigin-Name: c5a47063b7971868ebbb3f0718434dd6bd22d01c43697f48941df9aae3885abc
2023-04-14 18:00:03 +00:00
dan 3fc123b7bc Add extra OOM test for the new code on this branch.
FossilOrigin-Name: 846ae7e099ce67dab6b5df0ad5648a01d7bf7f2acf8312d817ed8505d4ea5aec
2023-04-14 17:00:29 +00:00
dan d36106c954 Add tests for the new feature on this branch.
FossilOrigin-Name: 0268d1a59316d169927cbd4c562725e46a9023b7d7123ade911c12203e222adf
2023-04-14 16:11:05 +00:00
dan 41334cf960 Fix SQLITE_OMIT_VIRTUALTABLE builds of testfixture.
FossilOrigin-Name: 430ec1a61507d8afd8adc161f577f32856dba21506db9717fae3d14570786439
2023-04-14 14:36:34 +00:00
dan 95dc3e08bd Update test script recoverbuild.test so that it may be run using old test harnesses that run more than one test script in a process.
FossilOrigin-Name: 5135aab0c0ab39f3b9cc84efbbc57f7266b42d98134fc0c5dace06f707374195
2023-04-14 13:34:26 +00:00
drh deabd6969b When setting the column types on a subquery, ensure that the COLFLAG_HASCOLL
flag is cleared from column names from when the collating sequence name has
been removed.
[forum:/forumpost/6916dacf83|Forum post 6916dacf83].

FossilOrigin-Name: 8d9dcd7cfdd53034e21cdb2cb997b6f5fe3207ca5d0d7a0b841b8a68e6ce8624
2023-04-14 10:40:50 +00:00
drh a73f51447b Fix a cosmetic indentation issue.
FossilOrigin-Name: 90deb84486494b903bab065fbf0174b56483c190353992b907359661393f2c54
2023-04-14 10:35:29 +00:00
drh 2142b7c0a6 Avoid double de-quoting of table names when processing RESTRICT actions
in foreign key constraints.
[https://bugs.chromium.org/p/chromium/issues/detail?id=1405220|Chromium 1405220].

FossilOrigin-Name: bb2b5ab172f0751c00343facf36fb12db10c88220caece31849f2711f12293d9
2023-04-14 00:20:16 +00:00
drh 731a1aaeb2 Fix an obscure issue with ALTER TABLE RENAME that comes up with triggers
that have UPDATE statements that contain errors.
[forum:/forumpost/ff3840145a|Forum post ff3840145a].

FossilOrigin-Name: c4845a7c5f7f219848d3ee32eef0f9c69ad6dc6e8509da84d612f41e1e05f007
2023-04-13 18:44:59 +00:00
dan 8efa288f99 Add extra test cases to window1.test.
FossilOrigin-Name: ebc844fbfb046c5789efe21fd607c9301cb7b3d78edef72b2926b8c889b048a9
2023-04-13 14:53:19 +00:00
drh 017bdf7838 Fix a code-generator issue associated with very unusual use of window
functions.  Both the expr.c or the window.c changes will each
independently fix the problem.  They are both included in this patch for
defense in depth.  [forum:/forumpost/0d48347967|Forum post 0d48347967].

FossilOrigin-Name: 1ba22631a7831e3562eda0eb6a5edf7f009c85c7ab4451d9eacd13ef0fb6036a
2023-04-13 14:50:50 +00:00
larrybr 4e87950b48 Get CLI utf8_fgets() to not consume more input than it returns. Get console setup restoration to happen for all non-crash exits.
FossilOrigin-Name: b4fa233d3dda54fa83771844cf5156bf1275c687925340af17a7713a9400dfef
2023-04-13 14:14:27 +00:00
drh 3f4a319934 Unwrap the loop in the WAL hash function.
FossilOrigin-Name: eb94ae13206762a42ddad12e0d1461df83e9759e44a369f188cd3aad82f211c4
2023-04-12 20:23:03 +00:00
drh be0023faba Automatically set HAVE_PREAD and HAVE_PWRITE on linux, as has been done in
MacOS for a long time now.

FossilOrigin-Name: 2f7a36d2c374100019bd9d38aea51ea10cfcf9b1f0330ba084a75f0773df8ebf
2023-04-12 19:40:00 +00:00
drh 6b507423a0 Small performance enhancement to integer-to-text conversion.
FossilOrigin-Name: cfb3dba9b015ce7a75857978bfd2540b4d5be985d9d2ec4a5842b3089ed3da60
2023-04-12 18:57:50 +00:00
dan 7877dca93f Avoid leaking a database handle in test script rbuexlock.test.
FossilOrigin-Name: 2edf98d128cabc0c9de1295470d7f87eb64e4cc1f13344fe3c198fcf9c7bf019
2023-04-12 18:18:28 +00:00
dan 02877e9558 Remove temporary debugging code accidentally left on this branch.
FossilOrigin-Name: 0a0f64870feaf95d2673efc8884ad1236d9bbf9bdb94364f8d2602221bd36ff3
2023-04-12 18:06:43 +00:00
larrybr ad5a7da489 WIP: CLI option to take control of console on Windows and make it support UTF-8 input pasting (or typing). Needs work to become robust per "ToDo:".
FossilOrigin-Name: 824382393d92d9eb6df8701de7c263280150569a708759c4a539acc6d8d38821
2023-04-12 17:54:52 +00:00
dan 015020cd1a Add the 'secure-delete' option to fts5. Used to configure fts5 to aggressively remove old full-text-index entries belonging to deleted or updated rows.
FossilOrigin-Name: 4240fd09b717dbc69dffe3b88ec9149777ca4c3efa12f282af65be3af6fa5bb0
2023-04-12 17:40:44 +00:00
drh 67f3765876 New #ifdef to enable building with -DSQLITE_OMIT_WINDOWFUNC.
FossilOrigin-Name: e1ff83fa2565334b28bd0d6582088c4ae0d2d9a590d973615a4a598683fe419c
2023-04-11 19:38:47 +00:00
drh 82aacda7eb Remove an ALWAYS() that can sometimes be false. Add a test case that makes
the test false.

FossilOrigin-Name: c8fb143d64d8e823684cd26799080da4b42bef121ca3c6315b1803a593490926
2023-04-11 15:06:20 +00:00
drh e4d8e7e5df Better handling of OOM errors in the cursor-hint logic.
dbsqlfuzz 60cd5fff91974af91c2c3692beb4a2d7fdafef46

FossilOrigin-Name: 68fcfb58df837b40e401dfa4e505f4f0b87554b9cfeb2c527f3558348b73c1be
2023-04-11 02:10:34 +00:00
larrybr 69881887b8 For sha3 extension, mention NIST standard implemented.
FossilOrigin-Name: 529ab138a438379f0cbd170bff5211aee1c327a1b3119a714ee15f68fd2239f3
2023-04-10 23:21:19 +00:00
drh ed36917835 New assert() statements to validate the parameters to
sqlite3BtreeCursorHint().  Fix a problem with the construction of those
parameters discovered by
[forum:/forumpost/0b53708c95|forum post 0b53708c95].

FossilOrigin-Name: 4c5a3c5fb351cc1c2ce16c33314ce19c53531f09263f87456283d9d756002f9d
2023-04-10 18:44:00 +00:00
drh 8346f8dacc Sync the vt02.c test virtual table with TH3, in order to pull in the fix
for long delays when there are huge OFFSET values.

FossilOrigin-Name: 49ba030080dd00b4fdf788fd3da057b333e705fa0fe37d653e2461bf96ca3785
2023-04-10 13:20:51 +00:00
drh d27c18818c Fix an assertion fault that can occur when compiling with both
SQLITE_ENABLE_UNKNOWN_SQL_FUNCTIONS and SQLITE_ENABLE_STAT4.

FossilOrigin-Name: 9ff69e599817610d14948a3eebcc4d3c18245f78a1433c23ae659211cbbdb017
2023-04-09 20:44:09 +00:00
drh d4fe3c77e6 Fix a faulty assert() in whereRangeScanEst() that should only apply if there
are not prior errors.
dbsqlfuzz 567ad91132879cbab8172b38c6a68ad40fa8d650.

FossilOrigin-Name: 8ba9b08bd976ed2c024f5f378b4dd40a55226e950af77c476c21273fccaf7e26
2023-04-09 10:09:28 +00:00
mistachkin 1a2f490fbc Fix a couple harmless compiler warnings seen with MSVC.
FossilOrigin-Name: 1b864a370fd04bba11487e3dab0388394bc71e1f3f006f7bd9d64eefe04e8b61
2023-04-08 19:27:03 +00:00
drh df3944ecb8 Fix a typo in [83e84531b46814ae] that changed the value of
SQLITE_DBCONFIG_STMT_SCANSTATUS.

FossilOrigin-Name: b71776539183e5fd8678ce8f3272180d3dbb8214ed7b7bb24d35c0ba027a6ac5
2023-04-08 19:23:13 +00:00
drh 9e0181fcf7 Faster implementation of keywordCode() - the routine that determines if an
identifier is really a keyword and if so, which keyword.

FossilOrigin-Name: 0ff3d3d53709b7f18bf01ded1f988e41b7f8471072cf4f2702a3a8b79964be3f
2023-04-08 16:51:08 +00:00
drh a9b20c0937 Fix a harmless compiler warning.
FossilOrigin-Name: c9559ba62191fe7fa2a718233afaa841e2594d1fc833314bf5b0a6b775e87c35
2023-04-08 13:31:17 +00:00
drh 2da07d9025 Optimizations to btree.c save about 4.5 million CPU cycles:
(1) Clone insertCell() into a separate insertCellFast() routine for
use by sqlite3BtreeInsert().  (2) Mark allocateSpace() as always-inline.
(3) Improved coalesence of adjacent free blocks in pageFreeArray().

FossilOrigin-Name: 5c12c400fe8eb4e86e14c69a6c34d0d78d9861e5d40a36c6a596a81c6dd65977
2023-04-08 13:01:34 +00:00
drh 65aae44b87 Guard against oversized cells in the newly enhanced pageFreeArray().
FossilOrigin-Name: 2dcdbb50356edbd3a79e53fa0bee4e700c2bdea78e27173b62ddabe44b066726
2023-04-07 18:27:32 +00:00
drh 3da5e2a9f3 Add NEVER on an unreachable branch.
FossilOrigin-Name: 9b3febbd988be05807ada20146d3e196ae17c966722fff049feb32292157bff2
2023-04-07 16:30:33 +00:00
drh 81e31c9c2b Fix an assert().
FossilOrigin-Name: 7eff46ba97dd8a3d6a5d01d5d61e98a2805deceafa47335eded7d784a8304525
2023-04-07 15:49:42 +00:00
drh 706c33d9ba Tweaks to the new insertCellFast().
FossilOrigin-Name: 203a581a9177c1083e8d5b49e8ff026af33b5c5e3e144aeda126f07a3a2953bf
2023-04-07 15:07:58 +00:00
drh 34ceb7e622 Clone insertCell() into insertCellFast() for use by sqlite3BtreeInsert() for
a substantial performance increase.

FossilOrigin-Name: f225afd90c8e65661d8b855050f0ee1a8fe4c0f3bcec824aa5a66d906f3c7119
2023-04-07 14:33:33 +00:00
dan 6d47af64e4 Fix a problem causing the rbu_exclusive_checkpoint=1 to be ignored with zipvfs databases.
FossilOrigin-Name: d8f50b31e8d64fce5141da4a016767a15482703364692a55df346f059fc9d30c
2023-04-07 14:03:30 +00:00
drh b39f037ca5 Small performance improvement in freeSpace().
FossilOrigin-Name: 8dc5292ee592f16451441e33ad0800ba10a21ecd63f1f9926d6915a59a1552d3
2023-04-07 13:21:20 +00:00
dan 1e24f9f1f3 Add further tests for the rbu_exclusive_lock=1 URI option.
FossilOrigin-Name: c07b62bef96bb69e9b1ce08f0084fdce8de981f4d8ea9689c87a41f1e4451ac7
2023-04-07 11:18:08 +00:00
drh 50dc8d9720 Increase the size of the cache of free blocks inside of pageFreeArray() to
reduce the number of calls to freeSpace().

FossilOrigin-Name: 27c59f1ea789f3ff245f23e79ded5cd71c48e3a51ffbb8c220b51101a4e69fd7
2023-04-06 20:14:10 +00:00
drh 82412051db Work around a harmless assertion fault associated with
sqlite3VdbeMemAboutToChange() such that the detection of stale values
in registers is preserved in debugging builds, but we avoid a false-positive
assertion fault in cases involving a virtual table with a LIMIT clause
in an IN-operator loop.
dbsqlfuzz 3fd70d4ab4950acf1deb8f610a7a7c67cd38713b

FossilOrigin-Name: 56ea2c2fe6108d39833ac40957afab59ade01a216639d5bafdeeca53bbf4cd67
2023-04-06 17:29:38 +00:00
drh ebec9c475e A prepared statement that aborts due to SQLITE_SCHEMA should not invalidate
cursors in other prepared statements that are already running.
See [forum:/forumpost/cae4367d9b|forum post cae4367d9b] for the original
trouble report.

FossilOrigin-Name: 857d0f5e16ba69ac9e5ee581befca2f7ed933edfe6e36396b10dae7979b44a57
2023-04-06 13:35:42 +00:00
drh 5482afd6cf In the CLI, during error processing while looking for a word boundary,
avoid being deceived by malformed input that has a very long sequence
of 0x80 characters.
[forum:/forumpost/ab93a23ba1|forum post ab93a23ba1].

FossilOrigin-Name: 82609d5a2d4eba741d48ea265f4e749578964961903c072c7b222ffe2aefaa3c
2023-04-06 01:05:52 +00:00
drh c6349ce205 In the zipfile extension, defend against corrupt ZIP files that contain
a zero-length filename.
[forum:/forumpost/b15f5e3ad8|Forum post b15f5e3ad8].

FossilOrigin-Name: 46db2e42a5f9b18da9661ccedca68cb70257ea5c58b33b401db2a5e030c1346a
2023-04-06 00:59:41 +00:00
drh 1ba1806637 In the new .scanstatus command in the CLI, make sure the database is opened
before invoking sqlite3_db_config().
[forum:/forumpost/6e26dcf544|Forum post 6e26dcf544].

FossilOrigin-Name: 1cd993c45cd6b60e00d1426dd01d63efad13f7258636b5fa694f21499e77955a
2023-04-06 00:18:31 +00:00
drh 072a02ce2c Add a test case for the ALWAYS() macro removed by the previous check-in.
FossilOrigin-Name: 68a1a837493a0bc5e0e0f2373ac76cb575078cec08990c017fdcb51a4ba363a1
2023-04-05 02:55:08 +00:00
drh 9d10ba8b1f Remove an ALWAYS() that might now be false due to the prior check-in.
FossilOrigin-Name: fc68993501aaa7180f5457dcb3c296e5b199904a385d98e2bcad7854e34d428e
2023-04-05 02:50:00 +00:00
drh 8907cb2f88 Fix the function that determines the collating function for an expression
tree to handle new cases that arise as a result of the recently added
ability to use indexed expressions in aggregate queries.
[forum/forumpost/0713a16a44|Forum post 0713a16a44].

FossilOrigin-Name: cc5041f3f067cf610adffb868b4e2d1b5d248dc5a0ecc551339b670800ecb0ff
2023-04-05 02:21:57 +00:00
larrybr c6c4bae80a Shell to use SQLITE_SHELL_HAVE_RECOVER consistently (correcting check-in 0421cc03e0efa8f1)
FossilOrigin-Name: 5b980d72a03fa1cfd0f1bf3ed04068b9f216b75a304deb2b7bbe8ddce0e6fb96
2023-04-04 19:56:28 +00:00
drh cd0f540761 Fix an incorrect entry in the array that maps sqlite3_value values into
actual datatype numbers.  dbsqlfuzz f660c659bcec48577a43d3bab37f46baaa22f59e

FossilOrigin-Name: fa8537dc90cad962dec695418d48da2890a172801ecad0c7c804023a063d3a02
2023-04-04 18:55:31 +00:00
drh c6a54f418c Remove an assert() statement that is no longer valid due to enhancements
to query planner for improved use of indexes.
[forum:/forumpost/dc16ec63d3 |Forum post dc16ec63d3].

FossilOrigin-Name: 2b23dd249d0bc254308f5539936d33ee558f1100dec616caac7317dbe70db761
2023-04-04 18:10:23 +00:00
stephan 20537f0dc7 Expose the new SQLITE_VTAB_USES_ALL_SCHEMAS to JS.
FossilOrigin-Name: b7ef09be667dd349e195842304c03cbebb1693164e1c3587a5d7c96baa713edf
2023-04-04 17:35:38 +00:00
larrybr 0d3651bd85 Omit shell call to sqlite3_dbdata_init() when it is omitted.
FossilOrigin-Name: 0421cc03e0efa8f1a83e46e91f26837366c10eef0903d099e25eb0cc99c297df
2023-04-04 08:49:22 +00:00
drh 39b07f1afc When translating arguments of aggregate functions into references to
expression indexes, make sure to only translate them for the current
aggregate when there are nested aggregates.
[forum/forumpost/409ebc7368|Forum post 409ebc7368].

FossilOrigin-Name: 898bfa1afd8260eaaf2aa6db94e74d99ebf4e8a6dc02cf21d20cd981393609a5
2023-04-03 23:49:00 +00:00
drh f61bf3dbe8 Improved diagnostic output from PRAGMA vdbe_addoptrace.
FossilOrigin-Name: 050958c1828f7446e7ee7e458123a23b735e213b76a67c3dd8356da256cbc170
2023-04-03 20:11:06 +00:00
drh 555970bf83 When changing a COLLATE expression node into TK_AGG_COLUMN because the nodes
value is contained in an indexed expression, be sure to clear the
EP_Collate property from the expression node.  Fix for the assertion faults
reported by [forum:/forumpost/e45108732c|forum post e45108732c] and
[forum:/forumpost/44270909bb|forum post 44270909bb].

FossilOrigin-Name: cf6454ce26983b9c3ae924c44a802f3f760eaaae9547b40aee9e14e7b0c47cab
2023-04-03 17:46:14 +00:00
drh 0669d6ebbf Add the SQLITE_VTAB_USES_ALL_SCHEMAS option to sqlite3_vtab_config(). Update
the sqlite_dbpage, sqlite_dbdata, and sqlite_dbptr virtual tables to make
use of that interface.  This was formerly handled by the internal
sqlite3VtabUsesAllSchemas() routine that was called directly from sqlite_dbpage.
But since sqlite_dbdata and sqlite_dbptr are an extension, an external
interface to that functionality had to be provided.
dbsqlfuzz 1a29c245175a63393b6a78c5b8cab5199939d6a8

FossilOrigin-Name: bcd51abee0b0f82bb5dbb881025a92d55baf9df6adeaf3a305e2e0da96a81d58
2023-04-03 15:01:37 +00:00
drh 5517538a9c Make the sqlite_dbdata and sqlite3_dbptr virtual tables accessible to
the CLI.

FossilOrigin-Name: c0eff02854d469bcdb949c09531760d6f96b49c17375abd1c131d41f38b9b2f0
2023-04-03 12:48:51 +00:00
drh d8c34e3311 Stronger constraint checking in allocateSpace().
dbsqlfuzz 93d4c9ff5ef7cd29f16e767af1ee71c29ec5a4c0

FossilOrigin-Name: 9e968f4fbce061190f10f31ce9d3eb4fce6706ea6b7e5011bfa1e893d37ca68d
2023-04-03 12:33:12 +00:00
drh 6455a7066e With the -DSQLITE_ENABLE_JSON_NAN_INF compile-time option, non-standard
JSON numeric values "Inf", "Infinity", "-Inf", "-Infinity", "NaN", "QNaN",
and "SNaN" are all accepted.  SQLite should never generate these values,
but it will accept that with the appropriate compile-time option.

FossilOrigin-Name: 0a050e9013331595e13ca9f859180057b59291c70a6cedb6230eefb25956df9e
2023-04-02 20:56:29 +00:00
drh c406ecc662 More off-by-one errors in the new JSON parsing.
FossilOrigin-Name: dbc99662087b63c9ed5b398535a6091fc2c5e507907dd1fcb7ad0b6ab3f17144
2023-04-02 20:46:24 +00:00
drh 26cc3a1387 Fix an off-by-one error in the recognition of -Infinity.
FossilOrigin-Name: f7ebf3e6286ddc8cdaa9446235407785d1be2be2d9992e21ef59fcd655f68432
2023-04-02 20:27:02 +00:00
drh af33654533 Earlier detection of corruption in sqlite3BtreeDelete().
dbsqlfuzz a4c48c291d6e40157a1b749a05eaa7c7faf5a625.

FossilOrigin-Name: 978dc71c388b37740da38c310674315c7d7fe814d1daa16a146b4df71385d1e1
2023-04-02 16:43:20 +00:00
larrybr 756c6cf281 Clear executable bit on base64.c
FossilOrigin-Name: ec1ddbce7db270afc25f71546fa745935ad74bc72920fccfcb37e98649612bf2
2023-04-02 14:53:59 +00:00
drh 8d1005a593 Allow special floating-point value names in JSON: "inf", "-inf", "infinity",
"-infinity", "nan", "qnan", and "snan".  All are converted into valid JSON
values: 9e999, -9e999, or null.  Requires the SQLITE_ENABLE_JSON_NAN_INF
compile-time option to operate.

FossilOrigin-Name: fc8793e5acac7351749e360c6bace5d5a8b3de3aa600ae23e260557db650c461
2023-04-01 23:29:59 +00:00
larrybr 4d5a5a48c5 Swat grammar nit on README.md
FossilOrigin-Name: 715c00e58b774d6a29e2f60fd33a2ecb960307192ae1fdb8ed396f74fd62b444
2023-04-01 16:14:50 +00:00
drh 54b81e36ff Fix harmless compiler warnings.
FossilOrigin-Name: a4fb2864fe01cce9694242a0750623ca47fcecd68f74c4239d3eb5fbf978770a
2023-04-01 15:51:21 +00:00
drh e5ea81ae85 Improved error messages from PRAGMA integrity_check. Identify the root of
the tree when a problem is found in a b-tree, making it easier to track the
problem to a specific table or index.

FossilOrigin-Name: a1cb152e69c7c6cdd99300c91a8104716089de459d9d19e33ef38432aad70908
2023-04-01 13:14:53 +00:00
drh 2e89f1cdeb In the b-tree module use %u instead of %d to print unsigned quantities such
as page numbers and offsets.

FossilOrigin-Name: 33ac62d8eec56eb71f20ccd28a5d6e4e8051522feb2db0199abee9e18ce1f64e
2023-04-01 12:22:57 +00:00
drh 1839b7990d Omit the the count-of-view optimization if there is a HAVING clause.
dbsqlfuzz 6a107e3055bd22afab31cfddabc2d9d54fcbaf69

FossilOrigin-Name: babe2b5e59647ac9db4601e67c25190aac14eb76d5fcb9fa5b3692b955fefd61
2023-03-31 23:48:59 +00:00
drh 567b069e42 Earlier error detection for index expression usage by aggregate functions.
dbsqlfuzz 29214ace4e25c98d2ddff8fbcf97afdda23f28b9

FossilOrigin-Name: 8e841e7f025f7205959453875f2d9db36271642045593970a2b2fc20b2f847c3
2023-03-30 19:05:48 +00:00
drh 09db37c022 Omit a branch that is no longer needed following [c9c4f287652933eb].
FossilOrigin-Name: 960a488a2db92f5437bc8171cdbed44618d9381c57fd41b2f0e357b6012c80b5
2023-03-30 16:08:54 +00:00
dan 53ed383075 Fix an error in new test script test/aggfault.test.
FossilOrigin-Name: 8724fe7426da55d19dba7b30e09321ba30c73286513864cb05de32f72e50ee31
2023-03-30 12:19:38 +00:00
dan 8fe84a40c7 Fix a crash that could follow an OOM error while processing aggregate functions.
FossilOrigin-Name: 804435a2731bd3c26278c47098854b9ee7727a686587f6208e793738fbfc0555
2023-03-30 11:05:36 +00:00
dan e1afa2bd24 Avoid having OP_SeekScan jump over an OP_IdxGT or OP_IdxGE that follows the OP_SeekGE opcode. Fix for [b50528af4468237c].
FossilOrigin-Name: c9c4f287652933eb6262a3419efe7e7288f55d3db7e2ac032eeae025f665d306
2023-03-29 21:58:06 +00:00
dan 7b56e97866 Fix a problem with sqlite3_stmt_scanstatus() from within an SQLITE_TRACE_STMT callback made from within a trigger.
FossilOrigin-Name: 1fa78fafa1340de458546526b03cf8b3e9c823913c4225d7c747ad182df5c0fc
2023-03-29 18:54:01 +00:00
drh 0dc43fd4c8 Another #ifdef to omit code that is only used by STAT4.
FossilOrigin-Name: 445c75567de9bf9c8075dfc62c60dc4abeefab59c7803a5ef2cc5d69c08d7633
2023-03-29 15:16:29 +00:00
drh a9fd5f224c New #ifdefs to omit code that is unused except under STAT4.
FossilOrigin-Name: 09a9b30ba7828a487a33a3ebf8028dfaa147dff67d2724584123b90a88d9814b
2023-03-29 14:42:11 +00:00
drh df542e0500 Enhance PRAGMA integrity_check so that it can detect that a NOT NULL column
contains a NaN value and report that as an error.
dbsqlfuzz f144b642fe6f1a1c196f258ac6e60118a0cb59b2.

FossilOrigin-Name: 7638d9755dc90fd353b874d03ed418fa8aaee4440290ff69b1b552eae84e5baa
2023-03-29 11:36:24 +00:00
drh e1be2b7ad9 Fix a weird corner case in aggregate function processing that results from the
recent addition of support for index expressions on aggregate queries.
[forum:/forumpost/bad532820c|Forum post bad532820c].

FossilOrigin-Name: c34fd9fe1b76e0a5943f014f46141cbe55d41bb1e6980adf9bcb6785a03e7883
2023-03-28 16:02:28 +00:00
drh 3cbf38c783 Fix multiple problems with RETURNING on a DML statement against a view,
all inspired by [forum:/forumpost/dc3b92cfa0|forum post dc3b92cfa0].
(1) Do not allow a RETURNING clause to trick the code generator into thinking
that the view being updated has an INSTEAD OF trigger.
(2) Generate all result columns for a view in a DML statement.
(3) The automatic covering index for a view should cover all result columns
of the view.

FossilOrigin-Name: c8bedef0d61731c29ae34de1594222d15b578f9e2cddbbd5b74fb3059644fe0f
2023-03-28 11:18:04 +00:00
stephan fc6c3936aa Remove a meaningless JS test. Add a timer to the OPFS async-side worker loader in an attempt to catch a browser-specific quirk in which the worker loading silently fails, per discussion in/around [forum post a708c98dcb3ef|forum:a708c98dcb3ef].
FossilOrigin-Name: 4fc1904b8e18c7d41fa65490ced125f1df4f0c22c13de957b24615ed09b3ecb7
2023-03-27 13:57:08 +00:00
drh 3594b2b303 Do not allow constant factoring during PRAGMA integrity_check, since the
constants might be stored in registers that are later reused for other
purposes.  dbsqlfuzz dc9ab26037cf5ef797d28cd1ae0855ade584216d.  Problem
discovered by a new assert() statement added in [6f8b97f31a4c8552].

FossilOrigin-Name: 0bba27b78112b2b2271e498f41c437df985aa2faab302ee5b29d9b60003a8379
2023-03-27 13:24:02 +00:00
drh aa9192e6aa Improvements to register allocation, especially in the ANALYZE command.
New assert() statements added to help verify that memory allocation is
correct, and to help fuzzer find lingering errors.

FossilOrigin-Name: 6f8b97f31a4c8552312b4c98432ea356ae54c06d9cc929969f50c3c88360cd7b
2023-03-26 16:36:27 +00:00
drh 418f947b98 Disable factoring of constant values during ANALYZE. This is a temporary
fix for [forum:/forumpost/07de5f6216|forum post 07de5f6216].  The register
allocation logic in ANALYZE needs to be completely refactored, but that will
take longer.  This check-in will serve to resolve the issue until a better
fix can be devised.

FossilOrigin-Name: c3967d1259f1df969d303394986960bd098e174dcd337e374c9c3c39e0efa466
2023-03-26 11:54:51 +00:00
drh b900f9e6e8 When the left table of a RIGHT JOIN is used inside an aggregate function and the left table employs an index on expressions, then make sure the expressions evaluate to NULL for the cases where the left table should be NULL. Fix for
[forum:/forumpost/9b491e1deb|forum post 9b491e1deb].

FossilOrigin-Name: ffe23af73fcb324df988a00be343654ce7078b7208647c4eb779d666b8297e7c
2023-03-25 23:52:05 +00:00
drh debc3202c6 Add usage detection to the NULL value generator for the left table of a
RIGHT JOIN inside of an aggregate.

FossilOrigin-Name: 4d05a009dfb63bcb4173da4d09ccc00c308ce4dd2534a32aeb23c5d8d6a1fd4b
2023-03-25 23:40:21 +00:00
drh 7bb842642c When reading sqlite_stat4 data during query planning, be sure to expand
zeroblobs prior to running comparisons.  Fix for the issue identified
by [forum:/forumpost/5275207102|forum post 5275207102].

FossilOrigin-Name: 5c8dd8dfcaab9c364b3a126ca35880ef57f5cecbe030771e646c934c8cf43709
2023-03-25 22:37:23 +00:00
drh de0416171b When the left table of a RIGHT JOIN is used inside an aggregate function
and the left table employs an index on expressions, then make sure the
expressions evaluate to NULL for the cases where the left table should be
NULL.  Proposed fix for [forum:/forumpost/9b491e1deb|forum post 9b491e1deb].
More testing an analysis needed - there is a FIXME in this check-in.

FossilOrigin-Name: 3572b40a7dfc4acc35e72e08e79f64688f8737e57ac89e4d10e6b32bd5178c63
2023-03-25 21:01:11 +00:00
drh 28ae195658 In the byte-code, when the result of an expression needs to be in a particular
register, always use the sqlite3ExprCode() routine because it has the smarts
to know whether to use OP_Copy or OP_SCopy.  Do not try to OP_SCopy inline
because an OP_Copy might be required.  Fix for the problem identified by
[forum:/forumpost/5522082cfc|forum post 5522082cfc].

FossilOrigin-Name: c104e5c6eeb89575319d9f94f49446142b06912fa8b283c19d46aa2ccddc5bda
2023-03-25 19:44:25 +00:00
drh 52786ec185 Add the tag-20230325-1 comment that was omitted from the prior check-in.
FossilOrigin-Name: a13c01d076d23f0de500e8e6283e803dfc96f0da7509c0d97d598d6b3e7b930b
2023-03-25 18:41:42 +00:00
drh 4924e82863 The fix at [2bf5413dc2c19d5f] was incomplete in that it failed to clear
the reusable register cache that might contain registers in the STAT4
buffer region.  This additional change corrects the problem.
[forum:/forumpost/83cb4a95a0|Forum post 83cb4a95a0].  Test case in TH3.

FossilOrigin-Name: 5d554e4d0f59a4309fed40e4fb26c7be42f1d4d55ccdcaaf7b4d445aa3122955
2023-03-25 18:31:24 +00:00
drh e5895333dd Even tighter bounds on the maximum length of the filename for
sqlite3_load_extension().

FossilOrigin-Name: 787291414d2d2082a3c63e7cdd6bec4719f0c8b75ad1355f5026932ecbb28ba8
2023-03-25 12:27:36 +00:00
drh 12b954db7a Tighter constraints on the maximum length of the filename handed over
to sqlite3_load_extension(), due to
[forum:/forumpost/a43074729e|forum post a43074729e].  This is a
follow-on to [01f3877c7172d522] and
[forum:/forumpost/08a0d6d9bf|forum post 08a0d6d9bf].

FossilOrigin-Name: 9f351bdee2a09a4419bb8256a13d1f757b3e00ec26a445523f224fc56d9ae26d
2023-03-25 03:17:08 +00:00
drh 2c16ec829b New test case to further validate the fix at [221fdcec964f8317].
[forum:/forumpost/d34ad68c36|Forum post d34ad68c36].

FossilOrigin-Name: a6e218a6e1ddd74be6a313b1d336334071747efd4ecd354ed7efe303d09c849c
2023-03-25 02:07:20 +00:00
larrybr 86b67f09e7 Fix CLI non-handling of OOM reported at [forum:/forumpost/6872514e04|Forum post 6872514e04].
FossilOrigin-Name: 6f6a0fd63b13cb827d6a402de01a701eb5b3f92954032ea80e78ec864861a26c
2023-03-25 01:29:40 +00:00
drh 62e43b25dd Remove undocumented, vestigial SQL functions in the CLI that were once used
for the ".recover" command but are now no longer needed.

FossilOrigin-Name: 1ef461aa4e95d254c2c1edebbbfd92ca96d752e04f68ebe70104e8d936d36be9
2023-03-24 22:17:59 +00:00
drh 338ff1ceca Fix possible integer overflow in bounds checking for the debugging function
"shell_int32()" found in the CLI.  This change does not affect the core
SQLite.  [forum:/forumpost/be9c294ee0|Forum post be9c294ee0].

FossilOrigin-Name: 6211471138a654641a4cf4831cfa3b470e06f29a2b77e4d58177c8e065bec11e
2023-03-24 21:35:48 +00:00
drh ed8e12e97b Fix an error in the OP_SeekScan opcode added by check-in [4a43430fd23f8835].
Problem reported by [forum:/forumpost/8cc1dc0fe9|forum post 8cc1dc0fe9].

FossilOrigin-Name: 651a13fcd16f03e89eb6228c9f3250e25910b9bbe2637f627f65ff78f8ba2059
2023-03-24 21:24:52 +00:00
drh b3ad5444d0 Fix the handling of indexed expressions in an outer query that appear as
corelated values inside an aggregate function within a subquery.
[forum:/forumpost/79cf371080|Forum post 79cf371080].

FossilOrigin-Name: d8259877eaa962e6f90675790d3770ef02bd1a5d86d319cd8c1834710df844c4
2023-03-24 20:35:56 +00:00
drh d23924d1ea Fix a problem in cursor-hints for WITHOUT ROWID tables used in a RIGHT JOIN.
[forum:/forumpost/591006b1cc|Forum post 591006b1cc].

FossilOrigin-Name: 221fdcec964f8317b2c23e926cc23799615cd3b4239a8a9ff87a83588d05bc64
2023-03-24 19:17:25 +00:00
stephan 4a1c8ed725 Add a JS test which checks for the issue described in [forum post 895425b49a|forum:895425b49a].
FossilOrigin-Name: 98d30400e4721b1d48ff601698ced146052654f8c6de7c014d4d239bb2dbef43
2023-03-24 17:15:15 +00:00
drh 28fe02cd51 Fix byte-code register allocation in ANALYZE for STAT4 when there multiple
indexes with differing numbers of columns.
[forum:/forumpost/bc39e531e5|forum post bc39e531e5].

FossilOrigin-Name: 2bf5413dc2c19d5feb32e5b01aa9b990ec2f74f45f5ca0dca15215f8c9dbc9b9
2023-03-24 16:57:21 +00:00
drh c03dbc5a24 The attempt to bring STAT4 up to 100% MC/DC at [55a26c67ed4a3a93] and
at [168fa2fb22b8c1ad] are incorrect.  Back them out and replace them with
a simple NEVER() macro.  Error reported by
[forum:/forumpost/dc4854437b|forum post dc4854437b].

FossilOrigin-Name: 5992370a89f8de7a6e941130b381f85d369856dbdb7860405e6fb17dad2293df
2023-03-23 10:54:07 +00:00
stephan 2255d89f0a Internal cleanups in JS code. No functional changes.
FossilOrigin-Name: 8fbdf7d10400c4f54fc3d8bc823f97818de860deeeed35ab6ad717260cd301e0
2023-03-22 19:57:19 +00:00
drh ad99243846 The floating-point-to-text conversion with the zero-padding option now
renders NaN as "null".

FossilOrigin-Name: ad59fa17663bda54ec5d4e48ac24e04b87daa70c795d840cd8db382e2dd581b9
2023-03-22 16:55:35 +00:00
drh 5e04d26a3d For consistency, the ".mode json" output from the CLI now renders infinity
in the same format as the JSON functions.

FossilOrigin-Name: abee339d5eae8b63d9579b7ec3a25f18071e675223e0bb5294c733fb618a3a26
2023-03-22 16:37:17 +00:00
drh c408c155d7 The double-to-text conversion renders infinity as 9e999, so that JSON
output is compliant and so that values can be round-tripped.

FossilOrigin-Name: b52081d0acd07dc5bdb4951a3e8419866131965260c1e3a4c9b6e673bfe3dfea
2023-03-22 16:24:17 +00:00
drh 8854f30295 In the CLI, the magic parameter :inf and :nan bind floating point values
Infinity and NaN, respectively, as an add to testing SQLite's handling of
those quantities.

FossilOrigin-Name: c70a61d8fbfb722679398c211aa48ccd84a392a6b59ba70551fe3a9fbab9a6d5
2023-03-22 16:01:06 +00:00
drh 00cd655c8f Update the version number for the TEA tarball to 3.42.0, to match the core.
FossilOrigin-Name: 03e6918e7f749c3ca1fa66df28b203ff26680f129003e20c71ca246cf7ff34fe
2023-03-22 14:51:22 +00:00
drh 68e976bcc2 Add ALWAYS() on a branch this is always true now due to [84417bbd144b2197].
FossilOrigin-Name: badf7d0e3cfa6efaff4b132cb4ecca79a16e73197c5e275db14ccb5ff938276d
2023-03-21 14:20:10 +00:00
drh 4e45e01693 Add the fuzzcheck-asan.exe target to the MSVC makefile.
FossilOrigin-Name: 0901bc02783527090bdaa626d946ee1e92c7c6848f3c77a799e6837f4e7f8d7b
2023-03-21 12:29:50 +00:00
drh 2439fc1f6d Add the fuzzcheck-asan target to the main posix makefile.
FossilOrigin-Name: 55e94adddba77c1d6ee6ef0c0cf37f7cdccc3bdc144ceaed675d79cdb25f48a6
2023-03-21 11:56:04 +00:00
dan 29fbee66c9 Fix a valgrind error and potential buffer overread when handling a corrupt database.
FossilOrigin-Name: b1e0cd6444d1f710e58129723b285cf1321679fa920fc2841492dcb489ab8b9d
2023-03-21 11:13:53 +00:00
drh 1eca5b5a40 Minor change to btreeNext() to facilitate coverage testing.
FossilOrigin-Name: 20b3ef04d8c79e281e32676d57c7a8569fac9e782ca24337691d44d383eff7bb
2023-03-20 18:35:48 +00:00
dan bb130c9223 Fix a problem causing a cursor to retain an out-of-date cell-info cache when processing a DISTINCT query on values that are identical according to their collation sequence, but different on disk. [forum:/forumpost/e132e6cde44fb505|Forum post e123e6cde4].
FossilOrigin-Name: 1b3abc1daeac29714256b5a1d5a07a75dc986f1089054a8bee44a00583b7383a
2023-03-20 14:59:19 +00:00
drh f3ddaaa89a Back out the extra margin added to the input buffer of the CLI, as it is not
needed.

FossilOrigin-Name: ac8d1e5de5da35fa93c3f9241a7817285920431c66f4e4593d4d5c2db9f2b960
2023-03-20 10:43:52 +00:00
drh 6263cefcf5 A better fix for the sqlite3_error_offset() problem on generated columns.
FossilOrigin-Name: 770b3e67c89ca16b08a9234acb626917a26ea414b3bc0340f9601644cb7504f2
2023-03-20 01:55:39 +00:00
drh 715963f1ba Expression errors in generated columns should not generate non-negative
sqlite3_error_offset() returns.  Second of two defenses against
[33aa4c0de8a62e33].

FossilOrigin-Name: 2adb4e0dda471ffc9b2a5cbad01b6c2bc6091b7526a8c8b19944cb2d4a5111e7
2023-03-20 00:53:56 +00:00
drh 0d53e8ed54 When reporting errors in the CLI, ignore the output of sqlite3_error_offset()
if the value returned is clearly out-of-range.  One of two lines of defense
against [33aa4c0de8a62e33].

FossilOrigin-Name: 26adbb80f51b9219ab2a3593496458b74a567ba5ee46b5154f056e389cfc09f7
2023-03-20 00:48:40 +00:00
drh 2ff6569940 Increase the size of ref-count values in the pager layer to 64-bits, to avoid
any reasonable possiblity of overflowing the counters.
There is a performance and memory penality for this.
[forum:/forumpost/b741f15a35|Forum post b741f15a35].

FossilOrigin-Name: 6c5d99a813e8498c2ea91383ac8886a43eaf47023919c03900e893640c03778d
2023-03-19 21:48:55 +00:00
dan 9b532d1e1b Avoid a buffer overread in fts3 that could occur when processing a corrupt record.
FossilOrigin-Name: 02ac2297abee6af64c8df230b42b07f21cff4565d7e315860b2396a7c0c556ca
2023-03-18 16:12:27 +00:00
drh 667b5cc985 Fix json rendering so that it shows positive and negative infinity as
9.0e+999 and -9.0e+999 respectively.

FossilOrigin-Name: efce4690a52592c4f5fc7d023eebe771b0e615bb03c0fe203493f853b28e8f85
2023-03-17 20:31:24 +00:00
drh f204be8098 Add the ability to name functions using one of the join keywords like
CROSS FULL INNER LEFT NATURAL OUTER RIGHT.

FossilOrigin-Name: 0910b1925e97f7ae4dae86894c9e2f54273c85115e19d0d9bff1280ffee35eed
2023-03-17 19:18:17 +00:00
drh 2eacecfe51 Add test cases for functions named the same as join keywords.
FossilOrigin-Name: 94944b239ce674d984c88ef6029b0260a972f1b25f01614b559ca07c3ebaf8f5
2023-03-17 19:07:48 +00:00
dan 66086eb6a6 Fix a potential buffer overread in the recovery extension.
FossilOrigin-Name: 0b3b5bf9597615589a1d045aaa697c13550553ee4fe4b9008a8e51415b6fe96a
2023-03-17 14:18:39 +00:00
drh 3f22b9ee2c Ensure that an error does not delete the Table object out from under
the xConstruct method of a virtual table.
dbsqlfuzz 7cc8804a1c6d4e3d554d79096e6ea75a7c1c7d2d

FossilOrigin-Name: df4928c92b0db77d0a40d7b492b609db191252e2f87bca63d000e4fe2e206293
2023-03-17 10:43:05 +00:00
drh 14e8299aa0 Add safety margin on the CLI input buffer for tickets
[33aa4c0de8a62e33], [b97e6c5e6a91d97f], [2971fbe3f993e95a], and
[2971fbe3f993e95a].

FossilOrigin-Name: 741af08af1b93406a120580379d13e514524af627da5387ecfa6e442d004bfdd
2023-03-17 00:42:27 +00:00
drh a84ead119e Fix assert() statements that would (incorrectly) fire if an IF NOT EXISTS
trigger that already exists contained two or more RETURNING clauses.
Tickets [89d259d45b855a0d] and [d15b3a4ea901ef0d].

FossilOrigin-Name: 648899e4ded72cac6cc24bccf7ebfc709ee7309a003452b21fd6ab0ba20c34b8
2023-03-17 00:01:32 +00:00
drh b816ca9994 Correctly handle SELECT DISTINCT ... ORDER BY when all of the result set terms
are constant and there are more result set terms than ORDER BY terms.
Fix for these tickets: [c36cdb4afd504dc1], [4051a7f931d9ba24],
[d6fd512f50513ab7].

FossilOrigin-Name: 12ad822d9b827777526ca5ed5bf3e678d600294fc9b5c25482dfff2a021328a4
2023-03-16 20:54:29 +00:00
drh 7a9c235aaf Additional debug/test output from the query invariant checker showing the
row-number that is being checked.

FossilOrigin-Name: e4b6eb58e65f7fa0c92768f42bc820614169b7c2f37ded391b866ddb0d894de0
2023-03-16 12:28:48 +00:00
drh db8c6aa03a Update the tracing output for the query-invariant checker such that it shows
the SQL that is run to verify that a found query-invariant discrepency is
valid.  Changes to testing logic only.

FossilOrigin-Name: 8f45ad27403e971d88ec62e674c03f82eb19df0b43aa58d20b3400cdb6611d90
2023-03-16 11:50:44 +00:00
drh 7d5a549121 Do not use the one-pass optimization on an UPDATE if there is a subquery
in the WHERE clause, since if the subquery is hidden behind a short-circuit
operator, the subquery might not be evaluated until after one or more rows
have been updated.  Fix for the problem reported by
[forum:/forumpost/0007d1fdb1|forum post 0007d1fdb1].  This is the same
problem that was fixed by [73f0036f045bf371] only for UPDATE instead of
DELETE.

FossilOrigin-Name: 2c56b984a0bd3be5ec326a2109ea7b8f1d4ef63c8fc325caac9663cf2479eaff
2023-03-16 10:17:30 +00:00
drh 5531316dd0 Remove a NEVER() from btreeNext() that dbsqlfuzz 460aa158f9a2c41145831cc924296cde1f312b3f
found could sometimes be reached.  I will find a way to test that branch
later.

FossilOrigin-Name: 1dffeffe150fff6777cf26bc960326ca4e4190d25d05bb066b082d1e1c8eb374
2023-03-16 09:07:46 +00:00
drh ffcad5893a Disallow the one-pass optimization for DELETE if the WHERE clause contains
a subquery.  Fix for the problem reported by
[forum:/forumpost/e61252062c9d286d|forum post e61252062c9d286d].  This fix
is more restrictive than necessary.  It could be relaxed if the subquery does
not involve the table that is the subject of the DELETE.

FossilOrigin-Name: 73f0036f045bf37193b6e87ae45b578c5831614c530488257c69666178da3aa5
2023-03-15 17:58:51 +00:00
dan eb5d71ed58 Fix a broken assert() in the recovery extension.
FossilOrigin-Name: 4c4e66f293d7768cceb875a936ca0f4cd910473e20b9910698cc1e1ce221a7d4
2023-03-15 13:53:47 +00:00
drh cdfb518f6f Fix Bloom filters on an expression index.
[forum:/forumpost/2e427099d5|forum post 2e427099d5] and
[forum:/forumpost/d47a0e8e3a|forum post d47a0e8e3a].

FossilOrigin-Name: c028fb669a5ae34dbaf50fffab1ae49bc568b994435cf02e145d24da3cfb48d7
2023-03-14 20:08:39 +00:00
larrybr bf7602a7d8 Include CLI's tip for -- in all builds. Better show optionality of its non-option arguments.
FossilOrigin-Name: 9e2c771daad485d923751fdee0316b195bb49ec9f865f2e39565906edeaa5cf8
2023-03-13 16:08:44 +00:00
drh 80511f32f7 The check-in at [198b3e33dcfd74c7] caused a performance regression for some
queries, which is here fixed.  Problem reported by
[forum:/forumpost/b405033490fa56d9|forum post b405033490fa56d9].

FossilOrigin-Name: dc9f025dc43cb8008e7d8d644175d8b2d084e602a1513803c40c513d1e99fea4
2023-03-11 23:21:21 +00:00
drh 7802389398 Allow functions named using keywords "CROSS", "FULL", "INNER", "LEFT",
"NATURAL", "OUTER", and "RIGHT".

FossilOrigin-Name: eeac3d5ec93e9cf8613d764b700b6e39a3ffb39b3953ce1e6a2f84dd986f9e86
2023-03-11 12:27:12 +00:00
larrybr 20f2917881 CLI help to reflect no-more-options option
FossilOrigin-Name: 30d95a12eb8b1cfc8ed11d3ed68cd713993ba34efa2fe623c18cfe41f14df1d9
2023-03-11 00:15:41 +00:00
drh 30ebcf2ecb Fix a typo in a comment. No code changes.
FossilOrigin-Name: 76acc075402aac2d14d8279b8095008a842522a0985fdf89200da4339757a40b
2023-03-10 21:27:59 +00:00
larrybr a9fffc179f Give CLI a no-more-options option. (--)
FossilOrigin-Name: 0822788752621f6bf6af44b420b594ddd352634b3b0ed0eb835abea34b45817a
2023-03-10 20:54:44 +00:00
dan 5f708cc832 Fix a problem with the fts5 snippet() function that shows up when snippets just 1 token in length are requested.
FossilOrigin-Name: 96d5116d17cbf3de6be38952881c6ddf6a226ddfd7649197bdb2df39c42b5d8e
2023-03-10 13:36:19 +00:00
stephan 17bbbde537 Export SQLITE_FCNTL_RESET_CACHE to JS.
FossilOrigin-Name: 6195cfc86b15614b8db0e0dc5cc79b8d1acaf483f0131c8526992dc8ca075630
2023-03-10 11:57:23 +00:00
stephan 434205a2c1 Replace a lingering use of 'self' with 'globalThis' in JS code, for node compatibility.
FossilOrigin-Name: 7e3782b5aa07621db95c2dc25b25ae21da988c9876d537b78ea289a83c75b06b
2023-03-09 22:09:13 +00:00
dan bad7d5041a Reinstate some test cases accidentally removed by [cb023fe28560ce0f].
FossilOrigin-Name: 870de61f8ef8781f2f9969b012f5c1cb95b6bce1a9a4dcaf02945b7846c3aa83
2023-03-09 16:11:43 +00:00
dan 1211e1d112 Fix countofview.test so that it works with SQLITE_OMIT_PROGRESS_CALLBACK builds.
FossilOrigin-Name: 2fc7c3fcee05c2a251ceb3666f3f6e9014cfe6e2f8570b72c43f251067e6b252
2023-03-09 15:08:06 +00:00
stephan fcdfd4716e Experimental addition of sqlite3-node.mjs, for node.js, based on feedback from [forum:ac7a94d4f77db235|forum post ac7a94d4f77db235] and related off-list discussions. Build changes only - no code changes.
FossilOrigin-Name: a5db97fa17d15711ab19af70595912d342ffa326a9b7029d4deb2194ae72a9f0
2023-03-09 08:51:04 +00:00
drh 8c26e6fa16 Fix a possible NULL pointer dereference due to the sqlite3_interrupt()
enhancement at [bd8fa10e59f58886].  Reported by
[forum:/forumpost/f5a2b1db87|forum post f5a2b1db87].

FossilOrigin-Name: 84417bbd144b2197c9930a520feb94b59053957c190be79f8deaaaebca68ecf1
2023-03-08 23:05:18 +00:00
drh 07d1ef97a1 Backout the OP_MakeRecord optimization as it does not work.
FossilOrigin-Name: 25017312d0d476d9cd5a39835748ee26c2ea482e163264ce2f9843ac627276d6
2023-03-08 22:48:03 +00:00
stephan 99ace82634 Export the new SQLITE_CHANGESETAPPLY_IGNORENOOP flag to JS.
FossilOrigin-Name: ac7359b2633ead74a53b2796fe038ca285aacad45b45980db2a76a87154e66e3
2023-03-08 18:05:57 +00:00
dan 975f2062da Add the SQLITE_CHANGESETAPPLY_IGNORENOOP flag, which may be passed to sqlite3changeset_apply_v2() to have it ignore changes that would be no-ops if applied to the database (e.g. deleting a row that has already been deleted), instead of considering them conflicts.
FossilOrigin-Name: cb023fe28560ce0f8c2fd48042553fcdb9db81eba9552be75165de0d46a2645c
2023-03-08 18:03:04 +00:00
drh 7ba63831f8 Small performance improvement in the OP_MakeRecord opcode.
FossilOrigin-Name: ca89daef0fcf6cb04aa6fa90dd333d6f2474bf3f458c833d9cd5bd8e59f2a04a
2023-03-08 17:09:32 +00:00
drh 06af03d70a Change to [44135d6ea84f7ba6] that retains the historical datatype ("INT", not "NUM") for a table created as follows: "CREATE TABLE t1 AS SELECT CAST(123 AS INT) AS value;". The use of FLEXNUM only occurs on compound queries.
FossilOrigin-Name: 6d5b5896261c62a7e130b47416ee8c25793859a2afcb1646c257600537a5b71b
2023-03-08 14:28:09 +00:00
stephan 248c1abdf4 Extend wasm build to support a custom sqlite3.c to support building against sqlite3-see.c. The JS code now binds the SEE-specific functions if it detects an SEE build.
FossilOrigin-Name: dd8612c8adbaf9d06bf0d7319b9afc9bd8ca3d0fcfa1cb591a7a2fcb86480048
2023-03-08 10:05:42 +00:00
drh 217e860053 Fix a problem in the count-of-view optimization that can lead to incorrect
bytecode.  dbsqlfuzz 23d782160b71c3f8f535ccb2da313dfc8eb8c631.

FossilOrigin-Name: f45009533a79a59b302598ee2545ef787c51d0128f4e1dca60dd83589f660619
2023-03-08 00:47:53 +00:00
drh cad225d6fc Fix a bug introduced 4 days ago by [e95439119ac200cb]: do not set the
Expr.affExpr field of a generated column expression if the expression is a
RAISE() function, as affExpr has a different meaning for RAISE.
[forum:/forumpost/b312e075b5|Forum post b312e075b5].

FossilOrigin-Name: 1096b5a7cc8104db01f8820ace47020baad2f12e6711e3a7b4514ed1becc7b66
2023-03-07 23:47:38 +00:00
drh bacb8aeb35 A proposed change to [44135d6ea84f7ba6] that retains the
historical datatype ("INT", not "NUM") for a table created as follows:
"<tt>CREATE TABLE t1 AS SELECT CAST(123 AS INT) AS value;</tt>".

FossilOrigin-Name: a0e54fe2058a4ac1e794b2491e424c60dfa42c6781b1cfd83c5db65c28c11c71
2023-03-07 19:39:18 +00:00
stephan 416e9c1908 Improve how sqlite3.initWorker1API() determines whether it's running in a Worker thread. Based on feedback in [forum:ac7a94d4f77db235|forum post ac7a94d4f77db235].
FossilOrigin-Name: 2f712b836a0dafd0b2ee6ba4eaa9caa35e49375c7ad0562477e0f2d3b086ec03
2023-03-07 19:23:36 +00:00
stephan 4214cc32ba Replace use of 'self' in JS code with 'globalThis', as that works in browsers and node environments. Avoid using globalThis.location if it's not set (e.g. in node). Based on feedback in [forum:ac7a94d4f77db235|forum post ac7a94d4f77db235]. Minor JS build tweaks.
FossilOrigin-Name: dbbe8f25e58738c10b6192d41f1e3886983871f17631cbc45ce626d3f05a6e26
2023-03-07 19:12:06 +00:00
stephan 7272f6d64d In the JS sqlite3.vfs/vtab utility APIs, use a local reference to StructBinder instead of sqlite3.StructBinder, as that object is removed from the sqlite3 namespace during the final steps of API initialization. Based on feedback from [forum:d19d96183badca70|forum post d19d96183badca70].
FossilOrigin-Name: 0d89885d28b44b1858117a72a180841f4f5f44bcc574fc59a116ca3526325932
2023-03-07 12:59:20 +00:00
mistachkin c105b3eccf Fix a couple minor spacing issues in the MSVC makefile.
FossilOrigin-Name: 46b3ac6d1fdd9207cdc511d445bb4c33d11102d6e4eb43f119293d62bb7008ff
2023-03-07 02:24:43 +00:00
drh 112eff9b27 Repair an unintential fork.
FossilOrigin-Name: 8b524c849f9844ea624cfe8968225200a57cefa8b7a17ebac2153c0cb3b657d7
2023-03-06 23:39:19 +00:00
drh 5b6ba9b250 Improvements to query invariant testing such that it uses the new
SQLITE_DBCONFIG_REVERSE_SCANORDER opcode to sqlite3_db_config() to make more
accurate judgements about when a query is ambiguous, and hence when query
invariant testing is approprate.

FossilOrigin-Name: be9ab292cd14889b1c9648b47138260b33fe5be282ff2d90653b1387885a7d02
2023-03-06 23:38:44 +00:00
larrybr 51a31a3b9c Cause CLI to fail noisily when deserialize option used for non-seekable "file".
FossilOrigin-Name: 24bd7e82471925987d924188ce0f80ed4f282b10ea1022e42881a7f529814eb9
2023-03-06 21:38:38 +00:00
drh f30da226e3 Add SQLITE_DBCONFIG_REVERSE_SCANORDER for direct C-language access to the
"PRAGMA reverse_unordered_selects" setting.

FossilOrigin-Name: 83e84531b46814aea6dad1ce8283cb9f6b90ad52badb60b875ea8f66e4ac0925
2023-03-06 19:04:39 +00:00
stephan eac6e8cb6a Rename sqlite3-worker1-bundler-friendly.js to sqlite3-worker1-bundler-friendly.mjs and refactor it to work as an ES6 module, based on feedback in [forum post a255f89c2eadf4c4|forum:a255f89c2eadf4c4].
FossilOrigin-Name: af312b131457743d98b84137bd51d9ba60e0daf0bd8f5a66f05956ca35ab68fb
2023-03-05 07:44:23 +00:00
stephan 32b3444f15 Correct rendering of error messages in demo-worker1.js. Remove some stray EOL whitespace.
FossilOrigin-Name: 1d5d515ad97cf61bd679f8c1afc607815c079583fe80264b591c6ef18f56fb8b
2023-03-05 07:33:11 +00:00
drh 89b3cbed67 Fix to check-in [b9190d3da70c4171] - the agg-with-indexed-expr optimization
requested by ticket [99378177930f87bd] - that can cause an incorrect answer
if an aggregate subquery has a GROUP BY clause, and that GROUP BY contains a
term that is not in the result set, and the outer query makes use of expression
indexes.  Problem reported by
[forum:/forumpost/a68313d054|forum post a68313d054].

FossilOrigin-Name: e06973876993926fd56181281d04b8dd504c689abf883fa21a5721cc1d478ea8
2023-03-04 15:36:51 +00:00
drh c8c7243d6e Show the output value from OP_AggFinal when doing byte-code tracing.
FossilOrigin-Name: 35f10a06ba81b8a526249729828fff06214e9cf95de418a9081f19d8d69fd657
2023-03-04 12:57:07 +00:00
dan 3410e550dc Make the SQLITE_DBCONFIG_STMT_SCANSTATUS option on by default.
FossilOrigin-Name: 5a09191186bc03b374e0c0d029e1a15208c6b845bc2f5f5f9f6a8a882809d9f3
2023-03-03 21:17:12 +00:00
drh fe52615925 Fix a vdbe-coverage macro added by [f418bdd627e84e7d].
FossilOrigin-Name: 77f559d2647615379fed55ced5d69ae90515273e59811b92171bdd3089c90a22
2023-03-03 19:56:19 +00:00
drh 294872382b Follow-up to [e95439119ac200cb] to fix a case where a pointer is NULL due to
OOM.

FossilOrigin-Name: 2535bc8c256a7642a6ac00ebfd3393beb93da94394c13b886c3ddd20d114aa3c
2023-03-03 19:43:39 +00:00
drh 1b9db7f32d Enhance PRAGMA integrity_check so that it can detect when there are extra
bytes at the end of an index record, which might cause OP_IdxRowid to
malfunction.  dbsqlfuzz c1aa3986534d5feab8d21f28b3c1712df2ef358ba.  Test case
in TH3.

FossilOrigin-Name: f418bdd627e84e7d494f730d7124d8f4846ebcde031f5b2498685c9aceebb3c8
2023-03-03 18:35:00 +00:00
dan 0433789863 When it is known when preparing a statement that X cannot be NULL, transform the expression (X IS NULL) to integer value 1 instead of 'true'. This is because under some circumstances, "Y IS TRUE" may not be equivalent to "Y IS 1".
FossilOrigin-Name: cc4bb05b3653e9502b95ea6fe0bfb77feebc11285b66e1dde4c7b945928efbf1
2023-03-03 16:25:18 +00:00
drh dc81902445 Do not use an expression index on a generated column if generated column
has the wrong affinity.  dbsqlfuzz 65f5eb57f8859344d5f1f33e08c77ee12960ed83

FossilOrigin-Name: e95439119ac200cb47d0e277622f41ee7986b364487cd252b485ce5fa030d70f
2023-03-03 15:12:46 +00:00
dan 0b23765b6f Remove unnecessary call to sqlite3_dbdata_init() from shell.c.in.
FossilOrigin-Name: c4d083a3aeeee69342d41b93a1393855871b0e4e7bfdb5fcc2973138018f248b
2023-03-03 10:42:23 +00:00
drh 3f23ce664e When flattening the right operand of a LEFT JOIN
(check-in [41c27bc0ff1d3135]), ensure that the OP_IfNullRow opcode does not
NULL-out a subquery result that was computed within OP_Once.  This fixes
the problem problem reported by
[forum:/forumpost/402f05296d|forum post 402f05296d].

FossilOrigin-Name: 8fe13f7a5e5eb798189acb25a608df7a94c2f5cc83463331a048b779c7890c82
2023-03-02 13:49:50 +00:00
stephan c355f6c839 Resolve a parallel build timing issue when building sqlite3.c/h from ext/wasm. For the time being, do not add sqlite3_wasm_extra_init.c to fiddle.wasm because it can cause duplicate definitions of extensions which are already built into the shell (a better resolution for this conflict is pending). No longer add sqlite3_wasm_extra_init.c to speedtest1.wasm because it's useless there.
FossilOrigin-Name: 75fdd5b83b4c527d25649b0d08841e3dc7d4d8109c1c97b2195b303538ced73d
2023-03-02 06:58:55 +00:00
stephan c3d84b4cbd Rename some vars in the ext/wasm makefiles for consistency's sake.
FossilOrigin-Name: c23589d92cd16b67266d97f4a3d8c0991864dbea30ec074173e1a67466532d21
2023-03-02 05:51:03 +00:00
drh bf90114867 When flattening a view that is the right operand of a LEFT JOIN, using
the optimization of check-in [41c27bc0ff1d3135], always insert the
TK_IF_NULL_ROW expression nodes, even for TK_COLUMN expressions, as
the TK_COLUMN might be a column from an outer query and hence still need
to be NULLed out. This fixes the problem described by
[forum:/forumpost/26387ea7ef|forum post 26387ea7ef].

FossilOrigin-Name: 198b3e33dcfd74c7ba6abcf789ee81dfed464a50ebf15c8edeff349d36789fca
2023-03-01 20:23:46 +00:00
drh fe37348255 Follow-up to [bbaf1f2eb1e1637b]: Make sure subtypes do not cross a subquery
boundary even if the function that returned the value with a subtype is
buried down inside a larger expression.  This fixes a problem identified
by [forum:/forumpost/37dd14a538|forum post 37dd14a538].

FossilOrigin-Name: e72661eb680ea707a839cb3d5cf6c7ef03706e7b40af1b84760147e59cd61a50
2023-03-01 15:21:53 +00:00
drh 46960e4677 Do not attempt to apply the count-of-view optimization to a CTE.
dbsqlfuzz ef8623915d843b150c159166ee4548c78cc6895a

FossilOrigin-Name: abc3a383636c0346053b5d09d96585f56c64cacb5751673ea3bf339e4955d1cd
2023-03-01 13:54:07 +00:00
drh f1f4f090c7 Activate SQLITE_DBCONFIG_STMT_SCANSTATUS in fuzzcheck.
FossilOrigin-Name: 4fe1419ac3161ea8735241b04913593170c636cf3e1583756fe94edd396cd38b
2023-02-28 21:23:46 +00:00
drh 0bdee38969 Updates to speedtest1.c and the speed-check.sh test script so that they work
with the new SQLITE_DBCONFIG_STMT_SCANSTATUS control.

FossilOrigin-Name: bd02df052e1ef78b5335915a38f3c5e13c3c04ab82fd251aeb42a440d1d39257
2023-02-28 20:06:30 +00:00
dan 06382de2a3 Change the name of SQLITE_DBCONFIG_STMT_SCANSTATS to SQLITE_DBCONFIG_STMT_SCANSTATUS.
FossilOrigin-Name: a63e4a150b505fc309fac847131009ee9965eb1b798ebcb202ec8b52f9189240
2023-02-28 20:04:01 +00:00
dan 45163fc45e Add an sqlite3_db_config() option - SQLITE_DBCONFIG_STMT_SCANSTATS - for enabling and disabling the collection of sqlite3_stmt_scanstats() statistics in SQLITE_ENABLE_STMT_SCANSTATUS builds. Collection of statistics is disabled by default.
FossilOrigin-Name: 0f5579bef27b84ee855065cfe87703c51e1f9773906a9e0d4e4dafc90bd0e553
2023-02-28 19:39:59 +00:00
drh 4990fc84f1 Only use a Bloom filter on an automatic index if one or more of the key
columns in the index can take on non-TEXT values.

FossilOrigin-Name: 5916705c731604d2e6b51a307cc8d7b67f4c102062bfdfcbc716a2916e0b0d86
2023-02-28 18:06:52 +00:00
drh a353369f57 In the Bloom filter optimization, hash all strings and blobs into the same
value, because we do not know if two different strings might compare equal
even if they have different byte sequences, due to collating functions.
Formerly, the hash of a string or blob was just its length.  This could
all be improved.  Fix for the issue reported by
[forum:/forumpost/0846211821|forum post 0846211821].

FossilOrigin-Name: 090304b870419acb5b05205a07fc75830b556928149f76a843cda526f77a6fc0
2023-02-28 14:28:54 +00:00
drh f43ffb30b0 When an automatic index creates a Bloom filter, show that in the
EXPLAIN QUERY PLAN output.

FossilOrigin-Name: d7b2ac1c1a31fa4285cf6df0995db7e7705bb6a1bc94850c14c94cc4e3eb239a
2023-02-28 13:46:01 +00:00
drh 485a92cbb9 Performance optimization in the varint decoder for the cell parser.
FossilOrigin-Name: b2b91c7cb40f1efe800059614e34823411016a3ece3f988e1574aecadd4c3114
2023-02-28 12:31:40 +00:00
dan b211a4b391 Improve the error message generated by the recovery extension if it is used with a non-SQLITE_ENABLE_DBPAGE_VTAB build.
FossilOrigin-Name: c1f2a1d55c180fb51a4e203befbbe529bdd23137b26190c50b8f85820450e7fa
2023-02-28 11:02:02 +00:00
drh ca4cf12d70 Minor performance optimization in the computation of an affinity string
for an index.

FossilOrigin-Name: 07334aa17b6ded27c5bd353998d96645a94cdcf32440abb59d127a002cd98ce9
2023-02-27 18:55:37 +00:00
drh baef8442d3 Further to [46639f682975dac6], the parameters of a table valued function that
is on the left side of a RIGHT JOIN do not need to be in the ON clause.  Add
new test cases and tags to associated test cases with the code.

FossilOrigin-Name: 18ee689de3d1ae43b05ca52e0b62c49442ebf68a88814a7c679e8856250a4b0d
2023-02-27 14:48:54 +00:00
stephan ef22012b6e Minor doc tweak for the previous commit.
FossilOrigin-Name: a8832164d36ea88dc1c5c874c67212800238eab6b1477e5ca92fbf8c77f341b4
2023-02-27 07:23:54 +00:00
stephan 9885c76c4c Extend wasm build to enable inclusion of client-custom C code, initialized via the SQLITE_EXTRA_INIT mechanism, per discussion in [forum:1e1c04f3ed1bc96b|forum post 1e1c04f3ed1bc96b].
FossilOrigin-Name: 68a52cafff60f19c9c998133d04f192b1e8b23f78b8cee13807d76845ef5e13d
2023-02-27 07:12:28 +00:00
drh d7ddec765c When a table-valued function appears as the right table of a RIGHT JOIN,
the argument constraints on the table-valued function should be considered
part of the ON clause of the RIGHT JOIN.  Fix for the problem reported
by [forum:/forumpost/422e635f3beafbf6|forum post 422e635f3beafbf6].  Test
cases in TH3.  Possibly related to the enhancement at [501609eddf2a46d5].

FossilOrigin-Name: 46639f682975dac6efec4e230aca2c9b127b4fc77e0b465fad38ef4caa9b7a4c
2023-02-26 20:57:09 +00:00
drh 4edbcdd391 In the [/info/7c2d3406000dc8ac|omit-unused-subquery-columns optimization], be
sure to remove the EP_Skip and EP_Unlikely flags from the result set expressions
that get nulled-out.  dbsqlfuzz bf1d3ed6e0e0dd8766027797d43db40c776d2b15.

FossilOrigin-Name: 21aec65e5e2a01e58dd0bb8c8b9b29b8414373b53353fc7ca80a152fdd27566b
2023-02-26 11:36:35 +00:00
drh 2d7829057e Split out a new variant of cellSizePtr() that applies only to leaf pages,
for a small performance increase.

FossilOrigin-Name: 22d32eef8741ae4f2aac3869465e5a7d2e33c6bc2425dd8e77f2a746e43687e8
2023-02-25 16:03:28 +00:00
drh 20e09ba6a8 Remove an unnecessary and redundant corruption checks in defragmentPage().
FossilOrigin-Name: 2dc7342e12e9dac2d8345d045a17caa29c26eaf6f417dd052c7c645eddd6c58c
2023-02-25 15:34:09 +00:00
larrybr 4ee8e9681c Revert (mostly) to earlier CLI response to -echo and SQL at the command line.
FossilOrigin-Name: 1c6cfcf6ed5e084e5eb6874a261f3670ab5fb3e5dbcf150f37c58b2f16351ee3
2023-02-25 15:15:40 +00:00
drh 0f8aebbfcf Performance optimization on sqlite3_vsnprintf().
FossilOrigin-Name: 4430e6e96dbf3d0b464776c132838dd3e456716e0ae5972fc7e0ece13b56ea25
2023-02-25 12:50:54 +00:00
drh 96705c1d7a Add the "on" option to the ".log" command in the CLI. Allow ".log on" and
".log off" even in --safe mode.  Enable the .log command for fiddle builds.

FossilOrigin-Name: 6bba9100ae81466eeb49845c449cbfddf4f82f18b89f55e6ef575cbf66af63af
2023-02-24 21:23:53 +00:00
drh d8f13f48a8 In the CLI, do not emit warnings about the incorrect usage of sqlite3_config()
in Fiddle.

FossilOrigin-Name: 847021b402a3e624e22959a70e3e29a4f069c2005534485779cdc018722b5863
2023-02-24 21:05:12 +00:00
dan 56151f6526 Fix a problem with the sqlite3sessions_fullchangeset() and sqlite3changebatch_add() APIs on this branch.
FossilOrigin-Name: a95a4bce8892eba3ab6e76a49975a52efeedebfd9af22bb31af30301b1d92328
2023-02-24 19:58:56 +00:00
drh 407f06a823 Modify the sqlite3_config() interface so that the SQLITE_CONFIG_LOG and
SQLITE_CONFIG_PCACHE_HDRSZ opcodes can be called after sqlite3_initialize().
Enhancement request [0b75886e6d48f7c2].

FossilOrigin-Name: 7902fb806b33a27932f9ca347246f4ac43091caad0a1536009985b650fd720e2
2023-02-24 16:04:05 +00:00
drh 70e01f6adc Back out the SQLITE_CONFIG_URI option from anytime-config.
FossilOrigin-Name: 0de98b8e21e4cbf2438160651585ea23ebea952c5a4e7d88e5eabe5cc49517b2
2023-02-24 15:59:27 +00:00
drh acf8905425 Adjust testrunner.tcl to make use of the new number_of_cores TCL command
available in testfixture, and to use no more than half the available cores.

FossilOrigin-Name: e0122d3863ed03e7bb64400d6561cbc824ecb14d228949ffde443069dbb4223a
2023-02-24 13:45:51 +00:00
drh 1fa0f27e81 Add the "number_of_cores" TCL command to the testfixture.
FossilOrigin-Name: 16ee5a7b5d02cad51a35d5675fcc5bb50a62d1f243bde107fc89fed44ffd6a43
2023-02-24 13:25:49 +00:00
drh 764e53d9bf Merge enhancements from trunk into the anytime-config branch.
FossilOrigin-Name: 04b2fdf3bc986cd402e8a6c060e1b3989e60770695e2f8a4bc12c43e380a694c
2023-02-24 12:55:24 +00:00
drh 8ac60da70e Slightly faster and more precise floating-point to decimal conversion.
See [forum:/forumpost/d1387c3979c7f557|forum thread d1387c3979c7f557] for
discussion.

FossilOrigin-Name: 6dea6f4738fc7d003183e94f08bd95181f23e165a966882c0bdc64708a05a607
2023-02-24 11:54:40 +00:00
drh 4b04ced3e0 Fix an incorrect optimization that was attempted as part of
check-in [18de3a8e6b431a07].

FossilOrigin-Name: f32055e8110a2eac6c9e26d1d1e620f0668bcb475d49d309dc549cea05e1e582
2023-02-24 01:08:35 +00:00
drh ed530c0221 Omit unnecessary branches from the enhanced floating-point conversion
logic.

FossilOrigin-Name: 32b0ba0d27481d85a4b0037ec36befcbffecdec75891eba58569acb160045553
2023-02-23 22:08:22 +00:00
drh 4df23a3248 Increased precision of floating-point to decimal conversions when the
floating point value has no fractional part.
[forum:/forumpost/d1387c3979c7f557|Forum post d1387c3979c7f557]

FossilOrigin-Name: 18de3a8e6b431a075143631eafdcf0e1b1a21868b417394c365d46d2feca10b5
2023-02-23 21:18:47 +00:00
drh 648de49c03 Fix harmless compiler warnings and a code indentation error.
FossilOrigin-Name: de6c5c6bb49a0beca793cdc0462da2c17bb05f3292e08dd815be29b45b59a8fb
2023-02-23 14:43:15 +00:00
drh ad96db8df5 Allow the sqlite3_config() interface to be invoked at any time for a few
choosen options:  SQLITE_CONFIG_LOG, SQLITE_CONFIG_URI, and
SQLITE_CONFIG_PCACHE_HDRSZ.  This list will likely change before release.

FossilOrigin-Name: e1702eb48d13c7c9b7605f1e77242672222c53059edcdc4e9cea59510715822a
2023-02-23 14:22:29 +00:00
drh 3c2688d137 Provide -DHAVE_LOG2=0 and -DHAVE_LOG10=0 compile-time options for use on systems
that lack the log2() and log10() standard math library routines, to cause
SQLite to substitute its own alternatives.

FossilOrigin-Name: 7ee22f95e7a7d8650f961f20e304e56c7813e624f05655d7392ca9347748270f
2023-02-23 01:52:54 +00:00
drh af03eb3241 Enable the count-of-view optimization by default.
Enhancement request [eaed8e36ce888f1e].

FossilOrigin-Name: a4aacdd323a854d771c8cb1e2e4cfc4fb66b0020cfed23525733603605f5c63b
2023-02-22 21:47:02 +00:00
drh 95dee6d2e3 Avoid computing the values for unused result-set columns in subqueries.
Performance optimization request [baa5bb76c35a124c].

FossilOrigin-Name: 7c2d3406000dc8ac5a99cc205b036356b67e4b0b94738592ffc5680749696904
2023-02-22 21:11:34 +00:00
drh 7b56a0eacd Update the version number to 3.42.0 to begin the next development cycle.
FossilOrigin-Name: 65910216393ed287dbc7e42a2598b6980183c62a64da6572cb691ff4ee23555d
2023-02-22 20:50:51 +00:00
drh bcb3215630 Remove the push-down optimization restriction that was added by
check-in [1ad41840c5e0fa70] because it is no longer needed after
the enhancements to compound query processing in check-in [27655c9353620aa5]
This resolves the performance regression reported by
[forum:/forumpost/bcc4375032|forum post bcc4375032].

FossilOrigin-Name: aa6bd6dff751223e302575c60f335707e4bb820bff716cab3706b564e18ed7ed
2023-02-22 20:42:15 +00:00
drh 7fc1cb22d1 Update the bedrock branch to version 3.41.0
FossilOrigin-Name: dc1d85707b130aae2f07b6b1961e0acd42d7a302b3518670f3dbfa96c179386d
2023-02-21 21:22:24 +00:00
drh ca773a51de Update the wal2 branch to version 3.41.0.
FossilOrigin-Name: 7bb1f6dca2d2e0c0c70f1d2d0c7130c182fc45e48138c57a3fbe92ee1f68da96
2023-02-21 21:19:36 +00:00
stephan dbb3260627 Remove some unnecessary bind() calls in JS code.
FossilOrigin-Name: 71215599cd8ebb970500d8366dfac40420a9a6b243ea9d6ba62d5cde28fae3d3
2023-02-21 20:08:10 +00:00
stephan 2b83570b5f Unconditionally disable -sSTRICT_JS flag in wasm build because it no longer works with emsdk 3.1.31+.
FossilOrigin-Name: b5e0b69649e6341ffff4e62f9f68abd2a53519f73964559a133f3b708e007e98
2023-02-21 18:47:25 +00:00
drh 3b832a1a29 Version 3.41.0
FossilOrigin-Name: 05941c2a04037fc3ed2ffae11f5d2260706f89431f463518740f72ada350866d
2023-02-21 18:09:37 +00:00
drh b5726a76f7 Disable DQS by default in the MSVC makefile for the amalgamation tarball.
FossilOrigin-Name: 409a19145e9558686acab730b94c6d53691b6d0efcea1e9239fc7e5863586a85
2023-02-21 13:28:49 +00:00
drh 61f0a03492 Yet another comment typo fix. No changes to working code.
FossilOrigin-Name: cdbdc8013fcfa8ce1bd4f5521c8d98afc53b9c577d43e5e6d0d1e15c789b910f
2023-02-21 12:42:25 +00:00
drh 0c30eaec99 Fix a minor typo in a comment. No changes to working code.
FossilOrigin-Name: 7981671d1449b3fc6eb214deb558d91b87dc4fa6e38e24606625a146cc71664a
2023-02-21 12:34:00 +00:00
drh 6e2b2f3073 Fix formatting and improved documentation on the various sqlite3_file_control()
opcodes.  Other minor documentation changes.  No changes to working code.

FossilOrigin-Name: 13280f720a4ef5a1f7f2a2fbe72f374d3a2e12be9bc0c50f7fce50d0cd28a67f
2023-02-21 12:30:18 +00:00
drh 3eeace23f5 Update the TEA version number to 3.41.0
FossilOrigin-Name: b450a920c0c59affd105616e999205cb9d2b25a79f8a7f7f05610ddaac0f770f
2023-02-20 19:43:01 +00:00
drh ffe9bfd8e9 Fix an incorrect #ifdef in the CLI.
FossilOrigin-Name: 3c4d29cf227511bf218ef620d497dc72c708dcc819971b34c7629f3ed11803fb
2023-02-20 16:55:58 +00:00
drh e17cac189f Fix a harmless compiler warning.
FossilOrigin-Name: b2534d8de5051a3516f4b9ee5a4b99e1dcf455e9af88c60815d2c2a479fc8566
2023-02-19 19:16:51 +00:00
drh 0bdbbdabfb Merge the lastest trunk code into the bedrock branch.
FossilOrigin-Name: 73e9f8d0bf78bbbaa3435330e485126bb09f008f35c1d0f366efd5e511f8f4a9
2023-02-18 21:52:25 +00:00
drh ade7cd53ab Merge the latest trunk enhancements into the wal2 branch.
FossilOrigin-Name: a36de0c803ee170b34105fadfc1f9790cda68a921fd5034790397f9853798f43
2023-02-18 21:42:38 +00:00
drh 1603f7edee Fix stale requirement marks and fix a typo in the documentation for
sqlite3_preupdate_hook().

FossilOrigin-Name: 655991f5d9afdd3281049eb430921046e0ba90eef215020c2fb149adc0d3c4c5
2023-02-18 20:31:26 +00:00
drh 3c7e90b894 Fix a harmless UBSAN warning in debugging code of the new unhex() function.
FossilOrigin-Name: 315574d5cbe5c805ff2163052d986b054985984fb45bfb064d1a7b39a22b90eb
2023-02-18 15:50:23 +00:00
drh 16cdb4b632 Fix a harmless typo in the test case added by [29fc06465efb948f].
FossilOrigin-Name: e0a0bf56f11f32f58705098e76c276deaf90cfa87e110032bdaad10ce9674f61
2023-02-17 22:48:46 +00:00
drh ab31a5df93 Do not allow the COUNTOFVIEW optimization to run if the count() contains
a FILTER clause.  dbsqlfuzz 4f8e0de6e272bbbb3e1b41cb5aea31e0b47297e3

FossilOrigin-Name: 29fc06465efb948f98d2733bb25ffa1e0662a0189304006b3d0be9fec7dd28c5
2023-02-17 18:27:48 +00:00
drh e43d101c83 A few simple test cases for the omit-unused-subquery-column optimization.
FossilOrigin-Name: cf8f57c53425d89619ece10edc197d7d28946d3b23fcb4f526330196c76d9cc1
2023-02-16 19:41:39 +00:00
drh 7defd20cb0 Provide an optimization-disable mask for this optimization. Do not do the
optimization if the subquery is an aggregate or is distinct, but allow it to
be an ephemeral subquery.  Do not omit columns that are used in the ORDER BY
of the subquery.

FossilOrigin-Name: 6b1a1f374d1372f11f5420d99645b218867100bf070bd3a8885bf5f00c189dff
2023-02-16 18:04:49 +00:00
drh 90b7af7715 Do not perform the omit-unused-subquery-columns optimizations on a
subquery that is DISTINCT, as that can lead to incorrect results.

FossilOrigin-Name: cc148503db8ef180bce984328da7e84959afadd6a9613c2d03bc1eafeb95dfad
2023-02-16 15:54:55 +00:00
drh 6945ba787f Fix the subquery result column NULL-ifier so that it correctly handles
subquery columns past the 63rd column.

FossilOrigin-Name: 77b220a7240425fa83e142ceef78505208e6e38d797070e146b9f7d255f753c7
2023-02-15 19:53:08 +00:00
drh e3ec00ccb8 Do not compute result columns of subqueries that are never used. Make those
columns NULL instead.  This optimization potentially resolves the enhancement
request described by [/tktview/baa5bb76c35a124c|ticket baa5bb76c35a124c].

FossilOrigin-Name: 5dec3cc0225296a043d17f73126d477d90a604f82b3180628176d8f950adbce8
2023-02-15 17:53:17 +00:00
dan bdb2ec409a Update testrunner.tcl to run zipvfs test scripts on unix.
FossilOrigin-Name: e6c8e19ab0d6e7526d4596b75a45bb6becaf3c029690f7e75c016eac803c9990
2023-02-14 18:09:40 +00:00
drh d6ba4252b2 Ignore extra parentheses around a subquery on the RHS of an IN operator,
because that is what PostgreSQL does.

FossilOrigin-Name: ecdeef43b27412b0b0b09e09a62ad3a03836a3fc80f2070268090e7ca8f02712
2023-02-13 19:32:40 +00:00
drh b4dc263ec5 Do not allow WHERE clause terms to match constant string index terms, which
can happen if DQS_DDL is enabled.  Follow-up to
[44200596aa943963].  dbsqlfuzz 54c9db85ed4af7055f5fd0d50877875c82b11d46.

FossilOrigin-Name: 2d2b91cc0f6fed8cb6f738dc7019047ce0f1e86b5eb8efa997095d08a32cbcb6
2023-02-13 18:42:01 +00:00
dan 1aacb3b02f Allow vector-IN expressions like "(a, b) IN ( (?,?), (?,?) )" to use an index.
FossilOrigin-Name: 18a99d9cfbc3a0419342b3fe91a83d2217578d0f6e8ee1084237d000946b1942
2023-02-13 18:37:48 +00:00
dan 5c84aafd77 Fix compile time option SQLITE_DEFAULT_SYNCHRONOUS so that it works consistently.
FossilOrigin-Name: bf6f1ee77c4bf653f6cd2a3db5292b8c5c83f18ea9acf951107d22807546b28a
2023-02-13 18:26:58 +00:00
dan 0c69938511 Allow vector-IN expressions like "(a, b) IN ( (?,?), (?,?) )" to use an index.
FossilOrigin-Name: 1815b15ddb8785a25b7617aab19e13c2410b9377389c16a85176025b3d9400e8
2023-02-13 16:10:31 +00:00
drh 9e463e878d In the LIKE optimization, do not analyze the new virtual WHERE clause terms
until both have been added, since they are expected to be consecutive and
the analysis might add complementary terms.
This fixes a problem caused by [44200596aa943963] and discovered by dbsqlfuzz
and recorded as case 7e3b5983727d843b910b2d9ab556e4afcd777cfb.

FossilOrigin-Name: d35de3ad3fac6b30d3f266cbe4b1e9923eb31a1eff4a869205bbc3ba122eeec5
2023-02-13 12:46:22 +00:00
drh 0c55b5fd1c Change a variable from 32 to 64-bits to avoid a harmless compiler warning
in Xcode.  [forum:/forumpost/402d733c22|Forum post 402d733c22].

FossilOrigin-Name: 0216ce23cf23bc147c5de6de178a6689b7ad744bf0ee0098809938b5fe10708b
2023-02-11 21:11:39 +00:00
drh 80e936aef0 Do a better job of detecting when a WHERE clause term might be useful to
an expression index.  Fix for performance regression reported by
[forum:/forumpost/e65800d8cb|forum thread e65800d8cb].

FossilOrigin-Name: 44200596aa943963bc6ca98b5d4fd5b9235d1109d8dfc1a75eeae353b4239142
2023-02-10 21:53:33 +00:00
dan a3e6192941 Fix a problem with the fts5 trigram tokenizer and LIKE or GLOB patterns for which contain runs of 2 or fewer non-wildcard characters that are 3 or more bytes when encoded as utf-8.
FossilOrigin-Name: 00714b39b39c51519edbc0194f98c7275fecf96763a06fd95db6e1d81bb9f1f1
2023-02-10 17:17:04 +00:00
drh 74e6e30358 Ensure that the valueFromFunction() routine does not clear a prior
parser error.  dbsqlfuzz 6fa816f20cf5b62260d635d110b88f38e29d8fe1.

FossilOrigin-Name: 734766451123c98a467c3407562eaa097b3307c8a275e1c8dd93e4654fe78014
2023-02-10 14:20:18 +00:00
stephan 98094e2f4c Minor text-only updates to wasm demo/test HTML and license header.
FossilOrigin-Name: f28e2a8613571fe3c23bfbbb602311071f4cb9731653216cfe436c38b0a59736
2023-02-10 11:05:16 +00:00
stephan c9e3cbe873 Fix ext/wasm/fiddle build, which was silently broken by recent build refactoring.
FossilOrigin-Name: dcf532931136d09bef23ccad669d486cb31daec8565317c3209c280b5695d45d
2023-02-10 11:04:39 +00:00
drh 3f4795a391 Disable the double-quoted string misfeature by default in CLI builds.
DQS can be reenabled at run-time using the ".dbconfig dqs_dml" and
".dbconfig dqs_ddl" dot-commands.

FossilOrigin-Name: c995932c3ffe7f2710ebce0fa407eca3200418aa84febc15fb15f20f3758a98d
2023-02-09 15:32:24 +00:00
stephan 5b56a13977 Squelch two harmless signedness comparison warnings in shell.c.in.
FossilOrigin-Name: bdd3edec13fa1142a8074c316e9ecec93fb278d42836eeb38587376fdf22cebd
2023-02-09 12:59:44 +00:00
drh 9a8330b550 New test cases added to fuzzdata8.db.
FossilOrigin-Name: be67bafccd79ba835ed7e287e09852ce83805da0797b164d0c8e44be05d9d28a
2023-02-09 12:47:15 +00:00
drh a0c8ec9ee5 The "flexnum" affinity that was added by [44135d6ea84f7ba6] needs to also
be added to the output of the affinity() built-in function.
dbsqlfuzz d309eaa5fe492c9606a8be876c2bc7dedb29d3d8

FossilOrigin-Name: f4ec68ceefa3a607f37d6355aac89fd1ae1704da4061bad7f271db1f17c1dd3a
2023-02-09 11:51:40 +00:00
drh 601e4d4a0f Back out the 'txn' enhancement to date/time functions. The duration of a
"transaction" is confused and needs to be straightened out prior to moving
forward with this change.

FossilOrigin-Name: 4a145f07322d768a07619bed27e0390d50f3a01d07787b9296234a5ceb6f1218
2023-02-08 20:29:48 +00:00
drh 03b30b7abe Always use 64-bit integers for stats associated with STAT1 and STAT4.
FossilOrigin-Name: 6647d1cb8b53102c8114b1f3e34173d907504d78aa3140eab53f88027a36660b
2023-02-08 17:28:42 +00:00
dan 0d12a79310 Better fix the problem where optimizing an fts5 table too often causes it to become unreadable (first attempt was [35bed981]).
FossilOrigin-Name: 459d986d38fc0ccbfd66801e0f22900cfed831268cf59ac8d1cd1e556f0d1441
2023-02-08 17:28:08 +00:00
larrybr 72ce57e675 Cause gcc warning suppression in shell.c to be nice in other project(s).
FossilOrigin-Name: 6b41ba2e996ab7b9c3943ab93a19748db5cf37792f5d59d20eec301085282355
2023-02-08 14:49:52 +00:00
dan 8ae1d29b13 Update an assert() in the stat4 code that is only true for a well-formed database.
FossilOrigin-Name: 04439f3b23b22a3c187637f286764568ee922227e3e34b2e63075876b4d1f2ac
2023-02-08 14:25:00 +00:00
drh 0873d88423 Fix an incorrect assert() in STAT4 logic added just a few days ago on
[2023-02-01|/info/55a26c67ed4a3a93].

FossilOrigin-Name: 168fa2fb22b8c1ad80c7001a429d3a80b48acfbfebb5414a0caa76964ea647c3
2023-02-08 14:17:34 +00:00
drh 821fe31fd7 Add the 'txn' date/time format. Change CURRENT_TIMESTAMP and similar to use
'tnx'-style semantics instead of 'now'-style.

FossilOrigin-Name: 61cc8ed8d3541eddf23a9d542179c3f0223c5b8519b54d3be79988a376e25a89
2023-02-08 12:47:37 +00:00
drh 159cbfe1cb Test cases to show that CURRENT_TIMESTAMP and similar hold the same value
across a transaction.

FossilOrigin-Name: 7b2fa20e31b8a0314b9ccb97bb556898d552505dbc5943f248df3f86c0f4e008
2023-02-08 12:31:47 +00:00
drh ed092fc368 Revert the behavior of date/time functions with no arguments so that they
once again work like 'now', even while CURRENT_TIMESTAMP and similar work
like 'txn'.

FossilOrigin-Name: 06180caff0f5d8aba83b9f7be682ccdd6cb2ffc228b557ee6c1fd00fe5c23e92
2023-02-08 12:01:48 +00:00
drh c1dab9ded2 Change the behavior of date-time functions without any arguments (and thus
the CURRENT_TIME, CURRENT_DATE, and CURRENT_TIMESTAMP variables) so that they
work like 'txn' instead of like 'now'.  This is an incompatibility with legacy,
but brings SQLite into conformance with all other SQL systems.

FossilOrigin-Name: 1ac78be54502779236645eac35b962797f2fb98307d059d2aa19658c4fa74cb7
2023-02-08 11:34:05 +00:00
stephan 51b0373998 Merge wasi-patches branch into trunk.
FossilOrigin-Name: 9902e66a37d59c1909593bbc296091df3fa3bfa12ec7062bd84e458030f77c8d
2023-02-08 08:49:52 +00:00
drh 3393ee7ff6 Simplify the code and add test cases.
FossilOrigin-Name: d3bed4caff561e71c396cc869c5b4d9bf216ba203485e738c12ec62741f1aba5
2023-02-07 23:55:59 +00:00
drh d4af882a1b Add support for the 'txn' argument to date/time functions that works like
'now' but keeps the same time for the entire transaction.

FossilOrigin-Name: 5e4f45af96247e29910403a63ac148cb313b005f9c014b37a9a49d98f5fef9a6
2023-02-07 21:55:14 +00:00
dan c3ea539dc2 Fix a test script problem causing "make test" to fail for non-RBU builds.
FossilOrigin-Name: 2abc3f9de0d2b66342598ab9015e62568ed20c7da4744a0f0a76b1db72c7e941
2023-02-07 18:41:23 +00:00
drh 9d6725c651 Fix a typo in example documentation code for sqlite3_vtab_in_next().
FossilOrigin-Name: a95dbfae6133cc7aac950c3e7ba55a29d80b1625c39479a7d1d10b0868ee2be4
2023-02-07 16:58:58 +00:00
drh b4a5f9ccde Fix an pcache memory leak that might occur due to the munmap() enhancement
of [754f331614f6f6e6] if "PRAGMA shrink_memory" is used.

FossilOrigin-Name: f8c1f9c44bb3da7856690ac04c1de1fb8b7e3ccc32141bf414cd85bd90644a30
2023-02-07 16:23:00 +00:00
drh fbaf259e0e Merge the latest trunk enhancements into the bedrock branch.
FossilOrigin-Name: b7d9f14d78a52f87fcc7bc1d3e9f00d6ce1d648b1b244bc3989d7308f52063a0
2023-02-07 11:39:47 +00:00
dan e3d452a826 Fix a dropped error code in wal.c.
FossilOrigin-Name: 2537db6f6b9df00913b57e2ad0e4deef2040ce47b28f9a9939f42c0443d70ac9
2023-02-07 11:19:20 +00:00
dan faff28a478 Fix an assert() in sqlite3WalFindFrame() so that it matches the equivalent assert() on trunk.
FossilOrigin-Name: f5bf91d9bf3910ab3dcf44c138084dfc26fb123edb8934bfdfcdd5c4afb67f2d
2023-02-07 10:58:39 +00:00
stephan f8c73aed67 Merge trunk into wasi-patches branch.
FossilOrigin-Name: 656d36f50f630da68262469087bad1ac71b10325e233a7963103c8cbc232f61a
2023-02-06 22:25:18 +00:00
stephan 9f29998d2a Roll back part of [c54f29d8] which attempted to use symbols which that worker does not have access to.
FossilOrigin-Name: 90b12211376eb445df7221b3d3593394ff651ef93f4218492c6208bc74b8fc63
2023-02-06 21:20:59 +00:00
dan 8057fe9baa Fix RBU test scripts to avoid attempting to copy or delete a locked file.
FossilOrigin-Name: f25eb898fce44120136f8ee5557305f43541e8a75323cfc3becf103f3fa795aa
2023-02-06 19:00:54 +00:00
drh 70cfee98e5 Disable an invalid assert(). Add the "WAL2" compile-time property so that
test scripts can more easily determine that they are dealing with a
WAL2-capable build.

FossilOrigin-Name: e93113259e05fb6b81ad1b4a460bd54168ff5c372b643d4845c1b1c87b7db204
2023-02-06 19:00:53 +00:00
drh d94023257b Merge the latest trunk enhancements into the wal2 branch.
FossilOrigin-Name: 44f8f33d4ab159476800142e763dfc6beb2c2c1d857c8eebb286498dec4004fd
2023-02-06 17:48:49 +00:00
drh 41c238f03c Omit the obsolete client/server test cases.
FossilOrigin-Name: 5dde07a91dcf99b9c9a418b4e2178f66eef4cffd4799538a419674314a7530f9
2023-02-06 16:23:52 +00:00
drh 913ea77443 Fix the build after the previous change
FossilOrigin-Name: aea3b6f6515bc3b7e7205b7d17d99cb1fc68030eb929d8adbe25f065668c9c95
2023-02-06 15:49:54 +00:00
drh da3d655fd8 Remove the long obsolete "client/server" mode tests.
FossilOrigin-Name: 08e3114caec0633fc54f8febb7f4732b46a1f47508bbca0a6a2bff02591254d5
2023-02-06 15:46:34 +00:00
dan 6896da42c6 Update test scripts speed1.test and speed1p.test so they may be run by testrunner.tcl.
FossilOrigin-Name: e761d4b9c8dcacccc927d23f20fdf779a0ae5304281b5bd6aa998bade56f0a37
2023-02-06 15:46:01 +00:00
drh 634ed64312 Shorten the status line for testrunner.tcl so that it fits on an 80-character
terminal.

FossilOrigin-Name: b760a7307c453d95cf3b302c9867c84a9c899956c16c2ce1ea6cce8f025db425
2023-02-06 14:48:02 +00:00
drh adc21bd434 Fix error in the releasetest makefile target from the previous check-in.
FossilOrigin-Name: e7fe287d7904166c05d31b96ec8fd6f09318869c57437d1c661db7b79e6d8bf2
2023-02-06 14:16:15 +00:00
drh 4bce182504 Update "releasetest" makefile targets to use testrunner.tcl.
FossilOrigin-Name: 1b95676b05701952cab7087f528028c48302f611fe63799551dc45098354d27f
2023-02-06 14:11:18 +00:00
stephan 3ff812aa94 Add capability to override the JS's use of console.log/debug/warn/error() with client-provided versions via the bootstrap-time config object.
FossilOrigin-Name: c54f29d8e55419eaa9168e799dab5030e89063b13d8390a50616606422d164dc
2023-02-06 14:01:19 +00:00
dan ee18659f69 Fix a formatting problem in scanstatus2.test introduced by [81c118d9].
FossilOrigin-Name: 928ab40edbf11c02b3f03e660051d9587243002b9220adc77f972c3758c12b19
2023-02-06 13:36:34 +00:00
dan 0bd0843eb1 Add the --fuzztest option to testrunner.tcl. Also, have it print whitespace over the top of the old report line before writing the new.
FossilOrigin-Name: d83ce01fe1f21edcc93ad4c6308026156f235bab75e16e15389e27ad388fb274
2023-02-06 11:43:22 +00:00
dan 2288ecbb9c Fix problems with test scripts preventing them from running with SQLITE_DEFAULT_MEMSTATUS=0 builds. Or builds without SQLITE_ENABLE_STAT4.
FossilOrigin-Name: c0fbc89ed26b6eb361c1a05e2fb0d78d101fe2daf2947bbe5e0e58e0c2a779a2
2023-02-06 11:02:31 +00:00
dan e734207a79 Fix some test scripts that were failing with SQLITE_OMIT_VIRTUALTABLE builds.
FossilOrigin-Name: 81c118d90b281b30f40b3eeefaeb2c8350aa40080b4fe7de8f21d263986c6acb
2023-02-06 10:47:57 +00:00
drh 072f48e556 Do not use a Bloom filter if any outer loop lacks STAT1 data, since without
STAT1 data, the query planner cannot make an accurate determination of
whether or not a Bloom filter will be useful.  This fixes the problem
reported by [forum:/forumpost/56de336385|forum post 56de336385].

FossilOrigin-Name: e6ab96bd0b9efc51fd703bf7e92b079943ad9a3b92183d434adc11fb7d238afb
2023-02-05 20:29:10 +00:00
drh 9ce8643676 Add the "devtest" makefile target that runs both fuzztest and testrunner.
FossilOrigin-Name: c81398c6215d6d1bc7a15b6be9bb1f81effd430ff0a7c86158372accfa77814c
2023-02-05 17:40:03 +00:00
drh e88b209874 In fuzzcheck, only show the description of each database if the -v option
is specified.

FossilOrigin-Name: 6a58179aaffa77a5542ab620ffce6f68135e399de957b1a97113fd2f1dc0c098
2023-02-05 17:09:18 +00:00
drh 8d07c3d497 Restore a test for pBt NULL that was removed by [12a14711afa6cddb] but turned
out to be necessary, as dbsqlfuzz quickly showed us.  Test case added to TH3.

FossilOrigin-Name: 647c89dd2f90da1605e6fc01ecc8c1367c12c06b3a972ed682a69e85bf1d77e9
2023-02-05 00:47:27 +00:00
drh fdde4042eb Modify the status line output from testrunner so that each line overwrites
the previous rather than scrolling.

FossilOrigin-Name: ad4d28cd68589a9259069c77c8d375ae49cfd0bcdf3b03d3e84418569e12ebfc
2023-02-05 00:32:32 +00:00
drh 2f9525b0ff Changes to the sqlite_dbpage virtual table to tag or remove unreachable
branches.

FossilOrigin-Name: 12a14711afa6cddbf64d5307fd239fc597027d4356a431ad75836025f4bad83c
2023-02-05 00:24:42 +00:00
dan a3d4d7a0ae Avoid running rbu tests under permutation "journaltest", which does not support wal. Fix some other test script problems.
FossilOrigin-Name: 92f018698a87ed32acc31d83f7246dbb7295867bd6a801b7984a16f97a5c7786
2023-02-04 21:25:17 +00:00
drh 111057cf66 Add ALWAYS() and NEVER() macros to unreachable branchs in sqlite_dbpage.
FossilOrigin-Name: db875cd6be3b6e0032325cf42e6846eb41a7f099eb4561baf402d914446b052d
2023-02-04 20:53:31 +00:00
drh 5db5c122ef Remove an ALWAYS() that can be false in some very rare cases.
dbsqlfuzz 2274a23c85a93b870f7eb12fe686073da01abc33

FossilOrigin-Name: a4eb0b0d837291590958a5f292f41e80e0f722668a060aebfed5ea36ea176332
2023-02-04 20:07:52 +00:00
dan fb962837fd Updates to the testrunner.tcl script so that uses a separate sub-process for each test. And so that it runs the release test procedure currently handled by wapptest.tcl.
FossilOrigin-Name: 4c3c587500ac5a3758008f0acb9432bd3bc6f0f31bacd5615f46280e55842982
2023-02-04 19:01:17 +00:00
dan b713d501f6 Changes so that wapptest.tcl still works.
FossilOrigin-Name: 7624de3a93e03e9e825b1a283ccde8ad3aa41f93dd5c7f63692620a5177cbda9
2023-02-04 18:58:51 +00:00
dan 0415dd9c56 Update testrunner.tcl usage message.
FossilOrigin-Name: b7c246bc3f462098f6281ee2cd2e4e870b6d6481263d277f04956596a4764558
2023-02-04 18:50:16 +00:00
dan 4e5ff3b820 Add testrunner.tcl "njob" and "status" commands.
FossilOrigin-Name: 3869cd3d306ed68ec6e118aa10b4e835f2bab1294039888d88a30b3cfcf1d7be
2023-02-04 18:47:59 +00:00
drh ad1079e35c Better error message when trying to do an INSERT on an sqlite_dbpage virtual
table.

FossilOrigin-Name: a98e4908dabc8f45f7859a26bd1d0c04de34ec68c1367c2cdd9b0242e08b938f
2023-02-04 14:23:26 +00:00
dan bd55d3ddab Fixes for testrunner.tcl on windows.
FossilOrigin-Name: b98e19fc41cb7f89ae9e063b8af414bc01d14833a14de5d646069483ef0127b2
2023-02-03 21:18:58 +00:00
dan d40a183506 Fix a problem where optimizing an fts5 table too often might cause it to become unreadable.
FossilOrigin-Name: 35bed981416269018a0d52171d9997c43d3b097f5db1910d10d882eabe2e137f
2023-02-03 19:28:50 +00:00
dan 33d3559736 Have testrunner.tcl run various builds as part of the 'release' command.
FossilOrigin-Name: ba2fae30a81e2574f13a7351c9f843b8c2c7d2c93fcd71fb2de98f5b34f3e109
2023-02-03 18:47:00 +00:00
drh 989d0cec5d Attempt to fix harmless compiler warnings that reportedly appear in clang 15.
FossilOrigin-Name: c045d76b908a8c90d22511df7884e78d452b250db9ba70d4cb0935048a3c3ac4
2023-02-03 14:57:40 +00:00
drh 410a3bafed Improved detection of invalid command-line arguments to the showdb and
showwal debugging utility programs.

FossilOrigin-Name: 75cdaafc77b8a1efc84e71e90470994227f376e7d7de34c813e75dcadbb9f268
2023-02-03 12:03:56 +00:00
dan 60228cb3d2 Avoid using Tcl command [clock] in testrunner.tcl.
FossilOrigin-Name: b2faf9289f6fbbcd4491e1b063da731b35aeed9ff1fc97f64798e99cac847b4a
2023-02-03 11:32:51 +00:00
dan dd0d7126ea Instead of free()ing them, retain allocated page buffers on an internal per-connection reuse-list when a page-cache is reset.
FossilOrigin-Name: 754f331614f6f6e630876bf23c7da84a8408c764540983c0182271268177c00b
2023-02-02 20:27:21 +00:00
dan e18fdcd35f Do not run test script pendingrace.test as part of the inmemory_journal permutation.
FossilOrigin-Name: 69d934a1bfad86f3f60dab146db539645f5735f1bdb07dfe3a3d88828a03390b
2023-02-02 20:15:42 +00:00
dan b0c8e2f900 Add configuration data for osx and windows to testrunner_data.tcl
FossilOrigin-Name: e45d34656a110f85671d8b5d33f0d6d807fd64dfef5777fefba22ac6836f0e86
2023-02-02 19:41:43 +00:00
drh 2dd3b645b6 Improved and simplified logic for resolving the various aliases of the schema
table.

FossilOrigin-Name: 5c19491c36b9e2128430e4f153bdef48c3f7b6541d44f36044e8fc2921ecc830
2023-02-02 16:30:32 +00:00
drh 6f31eac7d8 Resolve all possible aliases and variations of the schema table names.
FossilOrigin-Name: e7a0112b235d97cb68c92b967bf2218b90258c243ec2b638aaac142392873126
2023-02-02 15:28:40 +00:00
dan d9a58ab810 Merge latest enhancments into this branch from branch wal2.
FossilOrigin-Name: 5a22010c35789c6d12e4dd45c81c10b203f4662f206cb636bd0c2781f1cd9571
2023-02-02 14:27:56 +00:00
dan 1651f418e5 Merge trunk enhancements into this branch.
FossilOrigin-Name: c9b0180203378da26dc958c183ebb59274537e9fee3b346933b1e0844ed68a4a
2023-02-02 14:21:13 +00:00
stephan 7667eedf95 Remove automatic installation of JS-global S object for the sake of client libraries which embed this library, per [forum:9d4f722c6912799d|request in the forum].
FossilOrigin-Name: 9504f68af8360ea6d61574fd4b9811af34b721c12d9653eb50adcf0f12b129fc
2023-02-02 06:17:22 +00:00
drh 71a9339672 Adjustments of assert() statement in STAT4 in order to give 100% MC/DC.
FossilOrigin-Name: 55a26c67ed4a3a937e009f60da2cd951d6f090b0bea748037db5c1680e5ff3b4
2023-02-01 23:24:34 +00:00
drh 45ae9d6ba6 New assert() statements to verify that sqlite3DeleteIndexSamples() is always
called with non-NULL parameters.

FossilOrigin-Name: 92c71fdd7167a7db055ef0aec522bbba8c7b4b37658efda10dac44699921b57f
2023-02-01 20:45:12 +00:00
dan 986c22a6d0 Update testrunner.tcl to use a separate process for each test script. And to run some extra tests too.
FossilOrigin-Name: d090948a69a9c4b86693bd3caedba3d7e5883e4b2ad1f2d4bf7ae14c105ddea7
2023-02-01 20:14:59 +00:00
drh 54725efd80 Just because a CTE is used more than once, does not mean it should be
tagged with M10d_Yes and thereby prohibited from participating in the
query flattening optimization.  See
[forum:/forumpost/1d571c02963355ed|forum thread 1d571c02963].

FossilOrigin-Name: 66f29c403d28630bfaea9124bd63ee4a047b1fe4a7e27dc5d10d67d1601b15e0
2023-02-01 15:41:07 +00:00
dan 288409d731 Fix a comment related to PENDING locks in os_unix.c. No code changes.
FossilOrigin-Name: 6b3546c871fe78a4e550e0144b48ac98325787cc8b192a9e7f5f2a2ffa57f76d
2023-02-01 14:17:25 +00:00
dan 4209742161 Fix a race condition during hot-journal rollback on unix that could theoretically cause spurious corruption errors.
FossilOrigin-Name: e365dca4e5775b4897118c8937a1063282d7a1ecc2604529256b0a6b8a3510ba
2023-01-31 20:43:21 +00:00
dan ca3aabdf53 Fix a race condition during hot-journal rollback that could theoretically cause spurious corruption errors.
FossilOrigin-Name: 20ea53ddf590a9dd19501fabd2bfdb9c10b5eb265cd2995bdb335769c936c763
2023-01-31 20:21:06 +00:00
drh f7af9ba13c Additional tweaks to the enhancement at [609fbb94b8f01d67] to further reduce
the cost estimate for constructing an automatic index on an ephemeral table,
in order to resolve the performance problem described by
[forum:/forumpost/1d571c0296|forum post 1d571c0296].

FossilOrigin-Name: bf1aae7a8c7f2c74681aa29baa35259d10ce6a1737d2607def6bf27fed592131
2023-01-30 20:44:54 +00:00
stephan 26c7cff254 Two JS file renames which got inadvertently undone while setting up [9062b31174618c0e]. Cosmetic cleanups in ext/wasm/dist.make.
FossilOrigin-Name: 0c2fde767f77d6204e95737edd573f42d72e956a3c20ea7e4daeff906657bbe5
2023-01-29 06:01:32 +00:00
stephan 2f5330036d Correct the handling of the worker1 and promiser JS files in the face of the bundler-friendly changes. Those files require separate, bundler-friendly copies.
FossilOrigin-Name: 9062b31174618c0e67e86441e14eac420c734a0cc321f7eebc7d89ff8a449c10
2023-01-29 05:09:39 +00:00
dan 05a377fd46 Do not assume that sub-queries that contain window functions are uncorrelated.
FossilOrigin-Name: f27804484df57de76be9dbd1a9e5869916617684ee407101c978df7da30b34ac
2023-01-28 21:06:15 +00:00
drh 8cc8d36e4c Show more details about the SrcItem.fg field in tree-trace output.
FossilOrigin-Name: 3d05dddd0bc826a73a8cecd6b116168082fb46b6e4868d03438c0a5a546d2f97
2023-01-28 21:01:33 +00:00
dan 20438431c8 Fix a problem causing "PRAGMA quick_check" to return spurious corruption errors for a WITHOUT ROWID for which all columns are either virtual or part of the primary key, and for which the order of the columns in the primary key definition is different from the order in the table.
FossilOrigin-Name: a7530f897127f35a212db6557edbcbbb286cc0e094754c1c1b74ce8dbf724470
2023-01-28 17:37:37 +00:00
dan 54c9250b09 Update sqllimits1.test to account for the fact that if an odd value is specified as the length in bytes of a utf-16 string, it is truncated to the largest smaller even number.
FossilOrigin-Name: 74508470c4caff04638e750cd84073f3137b8a9f72c6fe390a279da7b1c13398
2023-01-28 16:37:30 +00:00
stephan c7170350d7 Makefile doc touchups - no code/build changes.
FossilOrigin-Name: 3458a481b9ee391122dba531358d664d5219ce4a58cbce93d07df57cc64dc2c7
2023-01-28 09:51:41 +00:00
stephan 919fe5bac6 Overhaul ext/wasm/GNUmakefile to consolidate what amounts to much copy/paste/slightly-edit duplication into a single function, called once per distinctive build mode (vanilla, ESM, bundler-friendly).
FossilOrigin-Name: 168e5a93013d8650c180e19788e5f301b5d5ae8501d0ce728004fd750ca9e682
2023-01-28 09:11:08 +00:00
stephan e84454ff16 Enhance oo1.DB.exec() to simplify returning whole result sets.
FossilOrigin-Name: 7b168ee2af09f04b41a6ef4c14ccaddc0c9b0bfe9dc1e6a86d8f5317606bd78d
2023-01-28 05:09:26 +00:00
stephan 65f7942d06 Add JS bundler-friendly JS build. Minor test code cleanups.
FossilOrigin-Name: 24d3a53dea5e596230558e233cbbd9d0288b4c394cd5ea7b650fd99bff4cde2e
2023-01-28 04:20:46 +00:00
drh cfd01014d0 Fix compiler warning in base85.c.
FossilOrigin-Name: bd9613fd63193bd6b33798d83f0ef21987ba468b53d13a25a31cb9e9e5f20562
2023-01-27 23:10:10 +00:00
stephan 69141f52be Update ext/wasm/README-dist.txt for the bundler-friendly build.
FossilOrigin-Name: 6a5c4f6b19aa0704efe7f789102f757a1ed05d6542f37c7aa37a4439783944a3
2023-01-27 20:25:39 +00:00
drh f92b006a6e Add an assert() to help static analyzers.
FossilOrigin-Name: e446c8b4aa2ffe075f9c289c771d62a414661c3ffe6abae5649c50af8e32fab6
2023-01-27 20:15:48 +00:00
dan 844e7183ae Do not try to run rbu tests with builds that do not support rbu.
FossilOrigin-Name: c74ad902e3bbfc28d6b30a37f3791176d64cdd8fd5950f1d1749541e244a0494
2023-01-27 19:59:46 +00:00
stephan 6a3e2d3776 Add a feature idea note to DB.exec(), derived from a forum discussion.
FossilOrigin-Name: 792f43209c4b6e85386b6b3906814b7f2ad310d50ba563b3fd1cb37e55adf8db
2023-01-27 19:56:40 +00:00
stephan cf73cb2fbc Cherrypick [3773934e91c20ca243] into trunk.
FossilOrigin-Name: 50cf4300a6a92302010fa4a7662db82bf55180a4930771f0d3c07e558bc87e7d
2023-01-27 17:19:30 +00:00
stephan aca8ce1535 Extract emcc version in JS build and use it to conditionally set build flags. Initially a workaround for [https://github.com/emscripten-core/emscripten/issues/18610|Emscripten ticket #18610] but may have other uses.
FossilOrigin-Name: 3773934e91c20ca2433cc414aaba75de9a4a4318fd1126fb781dba510ddefd94
2023-01-27 17:14:55 +00:00
stephan 9dc093d095 Work around a JS null pointer deref which could be triggered from the dev console, but not (it seems) from client-side code.
FossilOrigin-Name: eabb551b8b3d33fc3a327ecf7225436a3a3f616901e22c868fd76a5e3adc7b3f
2023-01-27 07:53:49 +00:00
stephan 87ce1ff7f7 Merge trunk into wasi-patches branch.
FossilOrigin-Name: 2ce89f5efcdb8b4c58eb2d30833a76d79ae0134c31d5ab8564be9e1cf5a1f4f0
2023-01-27 05:37:24 +00:00
stephan ca86a5351e Cherrypick [fa784101775b7|emscripten ticket #18609 workaround] into trunk.
FossilOrigin-Name: 9a26fae545b9c97129893b83ff97e62b1c477eccd1379af1dce4a3cc4fa9f932
2023-01-27 05:17:10 +00:00
stephan c64a3fa6c6 Work around upstream emscripten 3.1.31 bug [https://github.com/emscripten-core/emscripten/issues/18609].
FossilOrigin-Name: fa784101775b795077a23c211b5b16f51ad5a13967c284511f310dfcbfa9f77a
2023-01-27 05:14:34 +00:00
stephan a0013fbe87 More work on creating a separate sqlite3.js build which is hopefully friendly to JS bundlers.
FossilOrigin-Name: b7b896fb448a7f46eb88eadadb1359255aec637a384cabcdd526276a02f4f0b4
2023-01-27 03:18:16 +00:00
stephan 67bfea4ea6 Resolve a nested if-block bug in ext/wasm/c-pp.c which caused output after a nested block to be unduly elided. Remove a kludge, added in the previous check-in, which worked around that bug.
FossilOrigin-Name: 7a026a4b24d57c1b0970923b972dd42c3f1bb5b282f908079075468b2e1bf601
2023-01-27 02:21:16 +00:00
stephan 0cd38cd2b9 Beginnings of a bundler-friendly build of sqlite3.mjs. Not yet ready for downstream testing.
FossilOrigin-Name: 4271bf5f41df091696f1dcfc4ffe7a60d24066fc75c896941e0b56de95fe5f89
2023-01-27 01:33:12 +00:00
stephan 0945197a39 End-of-line whitespace cleanups and doc typo fixes. No code changes.
FossilOrigin-Name: bdfd72a083fadd724030c4c89adae71426e1ddd402c6bc5abf40801ecf3253cf
2023-01-26 20:08:59 +00:00
dan 3db12cd7c4 Have some RBU tests run as part of veryquick.test/testrunner.tcl.
FossilOrigin-Name: f51406e3bf92f2b46f13d08fb7c7fe7683feba68b5a8fa18f6f6b8845662deac
2023-01-26 18:16:51 +00:00
drh 3c12ebd4a3 Improved fix to allow sqlite3_vtab_in_first() to reliably return SQLITE_ERROR.
FossilOrigin-Name: b25eec1c13f6a8c7ba993e5a55e26798f3b67a8b9571998459671cb570cbdfb4
2023-01-26 02:18:53 +00:00
drh e3d71c078b Fix a harmless compiler warning in FTS3.
FossilOrigin-Name: 68d02d3c2ab1b4afff2d3a71516ceee404fad0d2b717369ad732832c4f9b19a8
2023-01-25 21:28:30 +00:00
drh e9b20a9c4b Defer calling the destructor for the user data on a module until after the
module has disconnected.
[forum:/forumpost/b68391eb71fdff73|Forum post b68391eb71fdff73].

FossilOrigin-Name: 1ab122289fe6ac4cb3b0c47d17ac22627fcdfed80c941a57e6e6cc4d9dbd9273
2023-01-25 19:05:04 +00:00
drh b604f1456e Enhance the sqlite3_vtab_in_first() and sqlite3_vtab_in_next() interfaces so
that they reliably return SQLITE_ERROR (and not SQLITE_MISUSE) if they are
invoked on a parameter that did not have multi-value IN processing enabled
via a prior call to sqlite3_vtab_in().  See
[forum:/forumpost/a823d4a3d5f73def|forum thread a823d4a3d5f73def].

FossilOrigin-Name: 144326dc171025dc8b5a77bebd8de3c19d5244ab807f5aa41f95313a25b880bc
2023-01-25 16:56:24 +00:00
dan a0fe1e7d13 Fix a problem with fts3 auxiliary functions and one or more NEAR expressions ORed together.
FossilOrigin-Name: de4690a10ad4631e7452ccbb05b177a821d9dda387a854d216a6c54c7a189ead
2023-01-25 15:45:45 +00:00
dan 0fa2bbf9e1 Fix another problem with fts3/4 auxiliary functions and NEAR expressions that consist entirely of deferred tokens.
FossilOrigin-Name: a8c91c132f6157b7e3649f57a799984b1d7f8a18fd434515c875617d4195db29
2023-01-25 13:42:55 +00:00
dan f36fa4e902 Add scalar SQL function unhex().
FossilOrigin-Name: 890e9629a7480138c9c1d3acc2d1e7b3c05e0d156e5c5fba428bc1aeb790fbfb
2023-01-24 20:17:43 +00:00
dan a50db438eb Update unhex() to allow a second argument, specifying a set of characters that are permitted to appear between pairs of hexadecimal digits.
FossilOrigin-Name: 66c8562690b19f17972589611810e1dccad3a48777acb05208289c1f77076f71
2023-01-24 17:19:47 +00:00
dan e35d1e1e6a Fix a problem with using fts3 auxiliary functions with expressions like "E AND ...", where E is a NEAR expression that consists entirely of deferred tokens.
FossilOrigin-Name: 39bfae4c4698a13e07c4a0725f2790955e03b601fe64e17a000c691def1bdcb8
2023-01-24 11:24:28 +00:00
drh f648389a2e Suppress a harmless compiler warning.
[forum:forumpost/e3f72e9291189925|Forum post e3f72e9291189925].  The code
was legal and correct. The revised code is actually less clear in its intent.
But at least now there will (hopefully) be no warning.

FossilOrigin-Name: 48bb7c88787bf5de1d70cf3cc81ada38c6c02e476dbdff12c8676c6d5ee19aed
2023-01-23 21:41:41 +00:00
drh c3f616597c Due to a coding error, check-in [8efd61e8518594e3] did not actually use
just read transactions if the operation is read-only and the
SQLITE_MAX_ATTACHED macro is set to 31 or more.  This was due to a misuse
of the writeMask field of Parse, pointed out by
[forum:/forumpost/aa173c18d5|forum post aa173c18d5].

FossilOrigin-Name: 8760566893b64325874c7ec0aff8014026ce525e7fff1489027d2ce698495e76
2023-01-23 20:45:47 +00:00
drh a794af53b4 Add the missing CARRAY_BLOB macro to the carray.h extension header.
FossilOrigin-Name: 10bf639cd169147d4608918543f9a9b15417c10a231590b961c91500a1856f6e
2023-01-23 16:16:00 +00:00
dan e3c11d55db Add experimental user function unhex().
FossilOrigin-Name: dbe424b5db33ce2c7562dfb44daf2969cf3074234cc891eb9b8d0d907faf6a78
2023-01-23 14:11:34 +00:00
larrybr f75e485ea7 Give CLI .version a place in .help output.
FossilOrigin-Name: 5f2dfdcc345453ee0a05311f6826d90b7c1d7b95fdaf77a0a8383923a8fc7213
2023-01-22 21:54:17 +00:00
stephan 0bcded30e5 Minor API doc cleanups and JS code simplification.
FossilOrigin-Name: f608a3a45609693c1c0237f12c394275ec9a6225fa986e62345f21af763293a0
2023-01-21 16:53:01 +00:00
stephan cdcb84ef00 Merge trunk into wasi-patches branch.
FossilOrigin-Name: 6fc20d75d49310aedbc3351a4a5f1aa9ef5b4100501c7bfbe556aca2be2e44d7
2023-01-21 12:18:28 +00:00
drh 3fdb05e883 Fix a memory leak in the TCL test harness associated with carray.
FossilOrigin-Name: 5a316f9fd94b73fc54ef94a1dab2f13bdd16c4fb13d2620b7c907d104d5b4d7a
2023-01-21 00:59:20 +00:00
drh c1b54fa225 Fix harmless compiler warnings.
FossilOrigin-Name: ded60f5b3431283d70eb8c41761c0fa9727021e6309d0df0ae3fb0a5274c1299
2023-01-21 00:19:29 +00:00
drh a96cd076fe Extend the carray extension to allow the use of BLOB values to be bound
as an array of "struct iovec" objects.

FossilOrigin-Name: e117a03ca6560e958884f277db827c51ae337a970c17143a0a5f9b150f52f87a
2023-01-20 21:00:03 +00:00
drh 42d652c047 Two branches associated with memdb are now always taken (I believe). Tag
them with ALWAYS() to verify this.

FossilOrigin-Name: 20b9b5aa4fa6136059cf1123f00d86cb7bd8fb0f0f86f971e9714f22725e60b5
2023-01-20 19:19:24 +00:00
dan d993b15aa3 Ensure that the database encoding cannot be changed while there are statements running. And that the connection is left in a valid state after an obscure OOM within sqlite3_deserialize().
FossilOrigin-Name: a02da71f3a80dd8e817e89cdaa775c95e38c90d2471f8fec516bed086539e2c0
2023-01-20 17:50:24 +00:00
dan ab5ebc4082 Add tests for changing the database encoding via RESET_DATABASE/VACUUM. And test that it is not possible to trick another connection with this.
FossilOrigin-Name: b869054acb8f171cbc16808eb87be85e6e38d5c5670522ff4e49aef92250d5b2
2023-01-20 15:13:30 +00:00
drh 3601db8f86 The json_group_array(), json_group_object(), and fts5_source_id() functions
should all be deterministic and innocuous.

FossilOrigin-Name: edcb83fa0d3c832d044b267d608fed31a2ac841b81c7c9406b2136ac43d61a1a
2023-01-20 13:34:14 +00:00
dan b8f9913baf Fix numbering issue in and remove duplicate test from windowB.test.
FossilOrigin-Name: fa10e561f5dcdb23af862c2e486e877d379f12eae077ae5fd3da6028f1c20b49
2023-01-19 18:16:09 +00:00
stephan 8a45f68b1c Emit sqlite3-api.(m)js during the JS build process, which are the JS APIs without the Emscripten/wasm-loading parts. They are hypothetically useful for arbitrary build environments/toolchains but have notable caveats related to the wasm imports, as elaborated on in the makefile.
FossilOrigin-Name: 966b55c513a14c6ab3ec128cfe6d157ecd9bec32e3fce2f559f130249cec50ab
2023-01-18 17:39:04 +00:00
dan ae2e7fcc4c Fix another issue with very large compressed LSM databases.
FossilOrigin-Name: d71154265a294a1ece89d257f55f6855db7c30aec55ea0dc4eeb61bce1e8fad3
2023-01-17 19:34:01 +00:00
drh f06067f0b2 Change an now unreachable testcase() into an assert().
FossilOrigin-Name: 517b2c4c4ee9afc50e5457a6877e5af34358a6267d8dad669d25410c124866ac
2023-01-17 19:00:22 +00:00
drh d37aabe53f Fix harmless compiler warnings in the CLI.
FossilOrigin-Name: 7cf282d307a23697b38f5fd4b46f13f63a9e6f7cc74ccd66297d88c00aa8f454
2023-01-17 18:40:54 +00:00
dan 19ab86f3ff Update virtual tables json_each and json_tree so that adding "ORDER BY rowid" to a query does not require an external sort.
FossilOrigin-Name: ce18f0ed684824e67ed6c09acab8e735fef2c52b2ed32270dee1a2a67802c59b
2023-01-17 15:46:27 +00:00
drh c37af308fa Add test cases to confirm that the schema parsing quirk in which an
ON CONFLICT clause is accepted and ignored on table CHECK constraints but
raises an error on column CHECK constraints.  We want to continue supporting
this harmless quirk to avoid breaking legacy applications and databases that
accidentally use it.

FossilOrigin-Name: 92b6a9cd0fb027fe675b3913aa07c75445bba0cfac9530d08d7e48f7869c04cc
2023-01-17 13:33:51 +00:00
larrybr e8f114baa6 Cause .clone to not trip over sequence table as reported at [forum:/forumpost/71ff9e6c4c|forum post 71ff9e6c4c].
FossilOrigin-Name: b44d04f7b051d807a81152a6e4f15a765f7b9ed1f01b48b40dc5420c11e0c251
2023-01-16 21:49:37 +00:00
drh 5b17533994 In the CLI, create our own private version of strncpy() to work around
false-positive compiler warnings from Alpine Linux.

FossilOrigin-Name: 83f21285fe86430a66ce6841606e3ad7c27da52ac75a034c6a00c7a9fdb9791d
2023-01-16 18:13:00 +00:00
drh 57003792df Omit the long-disused FTS1 and FTS2 implements from the active source tree.
The code will persist forever in the source repository, but there is no point
in carrying it around in the latest tarballs where it is never used.

FossilOrigin-Name: 2bb50d5aedef0fd216d94058f477a58d88aa3a68bbadc94fa67998b7c391a8ff
2023-01-14 19:53:42 +00:00
larrybr 3f7b9944b8 Add OOM check, per tip at [forum:/forumpost/933479b2d5|forum post 933479b2d5]
FossilOrigin-Name: eda84dcffee6016fb3f8588d96c7ffb6275edd626b11f6fe12e81be90226c7d8
2023-01-14 19:27:40 +00:00
larrybr 58abb646c5 Doc-only fix, per [forum:/forumpost/0cfaf6876b|forum post 0cfaf6876b]
FossilOrigin-Name: eac135fd6a5dd220a0f7c1b61f987bbd801faabdb76846c4417fef0dd03fcb87
2023-01-14 19:09:26 +00:00
dan 2bb3a4d6d6 Fix another lsm compression-mode bug.
FossilOrigin-Name: f9fafc941370f805f6656ac13543333b36ee52f1ad44c891252b99a74bd2ddf4
2023-01-13 20:49:42 +00:00
drh 475e76d43e In expression nodes of type REGISTER with sub-type COLUMN, ensure that the
iColumn field is set correctly, as otherwise the "IS NULL" operator might
be incorrectly optimized.  Fix for the problem described by 
[forum:/forumpost/d010a26798915b53|forum post d010a26798915b53].

FossilOrigin-Name: 0819a1869a39d54a405259ea323365506a182962a02affdef16a03446005da64
2023-01-13 19:29:46 +00:00
drh bd8e3d7d0a Add missing "const" on variables in recent RBU fixes.
FossilOrigin-Name: 9b8dcd79050f9bfa745271f51377b38cfed4d8ce41a36b773a30799d69f7b943
2023-01-13 18:20:40 +00:00
drh b28b5f348b Enhance query planning so that it check for sqlite3_interrupt() calls and
periodically invokes the progress handler callback (if any) during long
query analyses.

FossilOrigin-Name: bd8fa10e59f58886e10c84ea9934fa1b5f491aac2eff302757a76d2d36d59b69
2023-01-13 15:54:25 +00:00
drh b928141edc Update documentation for sqlite3_progress_handler().
FossilOrigin-Name: 41b4c407b7f6968f4493fb9a0b220c55a6f6a353a8e6cb054efd3eb01bdd11dc
2023-01-13 15:31:24 +00:00
drh 7ef26cbffa Remove an incorrect legacy assert().
FossilOrigin-Name: 3006b3b32fb1efedde3ade418c19c35b6e9a4af1e6a7cb5d5c72a080ec1f6f3c
2023-01-13 11:31:49 +00:00
drh 94effc8a05 Reduce the frequency of calls to sqlite3ProgressCheck().
FossilOrigin-Name: 795aca4a269ea4ede547844037c218ac714c40e8bee003e4d9b786befc9cf97a
2023-01-12 20:36:43 +00:00
drh 2fc9dc9330 Fix sqlite3_prepare() so that it only invokes the progress handler on every
N-th call to sqlite3ProgressCheck(), where N is the progress handler step
count.  Also fix faulty asserts exposed by the ability to interrupt in the
middle of sqlite3_prepare().

FossilOrigin-Name: 05461651599bb490ac6cfd893645dabab9cccedc6adcce15aee2487b2ea6027a
2023-01-12 19:51:49 +00:00
dan e4172b690b Do not attempt to run new test file rbupass.test if the "demo" VFS is not available.
FossilOrigin-Name: c5d958eb942a8d6ea1e6e1d2d57637f9359d518965a28f298c7fbacdb9080f59
2023-01-12 19:28:34 +00:00
dan 67da9239ba Avoid crashing when using an RBU VFS with a version 1 parent VFS.
FossilOrigin-Name: d149772d18c47bf986decb7e08d148b1d417bbcc8522fd6240ead836ec34074b
2023-01-12 19:11:41 +00:00
dan 64f03fa54f When deleting any old OAL file before starting an RBU update or vacuum, use the same VFS as will be used for the target database, even if this is not the system default.
FossilOrigin-Name: 5a86c6cb1f16a15bdbc42544b8ed1912d9e87f04c514b8481a77442fbbd8accf
2023-01-12 17:13:44 +00:00
dan 9271d22909 Fix a startup race condition that could occur if a wal file grows from 0 bytes while a BEGIN CONCURRENT transaction is running.
FossilOrigin-Name: 2c14f3832e0e21c4e68b556d5592860fcd79b6ca21db6c47f52a240b24559a9a
2023-01-12 15:44:48 +00:00
drh f84cbd1676 Attempt to provide a mechanism to do early termination of long-running
statement preparation by invoking the progress handler at strategic points
during sqlite3_parpare().  This experiment shows that sqlite3_prepare() might
leave the resulting prepared statement uninitialized following an interrupt.

FossilOrigin-Name: 79636f2d80aee70832913a78933da2a7e30cc037810b93903ebbc1925ea93fef
2023-01-12 13:25:48 +00:00
stephan a7c498599f Merge trunk into wasi-patches branch and add missing yes/no result to the configure script's output for the --with-wasi-sdk=PATH test.
FossilOrigin-Name: adc0ede0a43241b85563408d0de8e640a75ec4f7bdfe5e48acc5cdd6b182b88c
2023-01-11 22:45:20 +00:00
drh 8518eaccd7 Fix duplicate semicolon in btreeInt.h.
[forum:/forumpost/8db1711ca2|Forum post 8db1711ca2]

FossilOrigin-Name: 7526c46632578a2b602622b9debc406b52af4a42cc880970c4307d13853d59d3
2023-01-11 20:52:15 +00:00
drh d29bcd9d08 Fix a false-positive in the out-of-range jump detection logic that was
added as part of RIGHT JOIN.

FossilOrigin-Name: ab5bcb91cda45576ae9f3f272ec92eb3be3c26436a440ebb89f51f49c42e0fd2
2023-01-11 17:59:18 +00:00
drh d2467a89fd If OP_Rewind has P2 of zero, that is an assertion that the table is never
empty.  This fixes a false-positive in the out-of-subroutine jump detection
logic added in version 3.39.0, and which was causing the assertion on the
previous check-in.

FossilOrigin-Name: 33fd9997ebb88f0d78522c036e75aef08015d31d28b1cbee08ae7c4cd5ecc6aa
2023-01-11 17:50:24 +00:00
drh d2b11786e8 Add an assert() to the byte-code engine that goes off if the OP_Halt opcode
is invoked with SQLITE_INTERNAL.  This causes the RIGHT JOIN error
"Opcode jumps to ... which is outside the subroutine ..." to fail immediately,
causing it to come more readily to tester's attention.  There is at least
one testcase in test/fuzzdata8.db that asserts due to this change.

FossilOrigin-Name: b8f994414285264f4f7c472dfad646a061fc3580b754eac0f20080c24ecc256d
2023-01-11 16:44:57 +00:00
dan b609a79f4a Fix problems with compressed LSM databases larger than 2GiB.
FossilOrigin-Name: f884224578e549c7a15cd710e351c675631bd8939bd1d353b4337dcf4144c868
2023-01-11 16:25:55 +00:00
drh 5dd74bf384 Improved progress-handler and interrupt detection during PRAGMA integrity_check.
FossilOrigin-Name: 6db42780a9e530bcc94490cc6080536309666dc13523272d1799d6661137e908
2023-01-11 16:17:31 +00:00
dan 7c6f915af8 Fix another 64-bit offset problem in lsm.
FossilOrigin-Name: 1f3d0bdc20aa74a595f7bb7b2c152259a9d3a8ffbe7cc229c57ad2142df5376d
2023-01-11 16:02:20 +00:00
drh 3b7a19b033 Add a new sqlite3_is_interrupted() interface that can be used by long-running
app-defined functions and similar to see if they need to exit early due to
an sqlite3_interrupt() call.

FossilOrigin-Name: d030f341369b7f32789cbcf3d0ad9a2ac5cad99a56dac7dfe68b7f06dc339b17
2023-01-11 00:27:06 +00:00
drh 706631de32 When computing the datatypes for columns in a view, use the same datatype name
as the underlying table if such is available and is consistent with
the computed affinity of the column.
[forum:/forumpost/7fb1fe9dcf310ef5|Forum thread 7fb1fe9dcf310ef5].

FossilOrigin-Name: 497a98363fd1ed079544620ec5d0883f987ed03013131542741c93d7568e8568
2023-01-10 19:57:26 +00:00
dan 8340f54bdd Fix handling of unix paths that contain ".." components such that "/" is considered its own parent directory.
FossilOrigin-Name: b829dd1b976454619ca04c69aec5ebb5a2286772f4267b82ab2ea5fdb81ccd41
2023-01-10 15:07:18 +00:00
drh b0779fb969 Improvements to the SQLITE_DIRECTONLY documentation.
FossilOrigin-Name: b277ba40a8b0acea15bd73036d1c86fb5187f047ec8500ebc88c738ea3dbd118
2023-01-10 14:33:26 +00:00
dan 2e77734b98 Fix handling of unix paths that contain ".." components such that "/" is considered its own parent directory.
FossilOrigin-Name: 3c6cadb396de3981bd950eddd532daa8134bd4bf22c578620e323835c96a8500
2023-01-10 14:31:56 +00:00
dan 7d7b88dd2f Fix some problems with large, compressed, lsm databases.
FossilOrigin-Name: 956e985ff9bf8f21a048e49f7190bcc52ec4a7d35ecd685b0a293a01c9777a96
2023-01-09 21:10:42 +00:00
larrybr bf395ec58e Clarify help for .quit.
FossilOrigin-Name: 8004a2b7439748f1034df897af7b6c58b48a46923c6fdddbe7d78c89b8d7b438
2023-01-09 18:42:28 +00:00
drh 6791891f92 Fix JSON functions so that they work correctly under PRAGMA trusted_schema.
[forum:/forumpost/c88a671ad083d153|Forum thread c88a671ad083d153].

FossilOrigin-Name: 51a5d83c425d2e31508b73074d0076156817afb19003f847d16bf4a69ae5077b
2023-01-09 12:01:30 +00:00
larrybr 4ff47d3e37 Doc-only update, sqlite3_preupdate_hook() return
FossilOrigin-Name: 2da51d7e1b9f16ef03efbb096ce2a84e8c23b883380f48b2d374bdc521865aeb
2023-01-07 22:28:00 +00:00
dan 4b11e6efd6 Account for read-only connections on databases with non-default block or page sizes.
FossilOrigin-Name: 3a2075b089ecdb4b55243235dfabf466ea7999c2118aa8f58300925fb75c884e
2023-01-07 13:40:32 +00:00
drh 50da6ac8c1 Changes a testcase() into an assert() due to the
[e58bba93717cd6ff] change.

FossilOrigin-Name: a6251d72894f9c2e21fc6e91b1d2452a204952f5e1a94fd93835a47c7dfb9be3
2023-01-05 14:41:18 +00:00
dan e3b1c3868b Fix a problem with applying integer affinity to a MEM_IntReal value. Forum post [forum:/forumpost/d270125fae|forum post d270125fae].
FossilOrigin-Name: e58bba93717cd6ff950c6f9e077b4327b59b1956dd5f6668be3de9509584b8fe
2023-01-05 13:35:23 +00:00
drh b8a655beeb In the zipvfile extension (which is not part of the SQLite amalgamation, but
which is included in the CLI by default) ensure that fopen() is not invoked
using a NULL filename.  Fix for the issue reported by
[forum:/forumpost/d1c96a9032e564f8|forum post d1c96a9032e564f8].

FossilOrigin-Name: d32757ddf5d311b6f62545c861fd606244e03f02cb2e317c1131ee6b80c2299c
2023-01-05 01:26:58 +00:00
drh e0237c00e1 Adjustements to the tool/warnings.sh script to account for compiler
differences.

FossilOrigin-Name: 863c03bee117e81bcea8c3ae78bf125afc7750aa5bcb89e42ee66d9c8b6ddc29
2023-01-04 18:32:14 +00:00
dan 52c08ee982 Ensure that LIMIT clauses may be passed through to virtual table implementations even if the WHERE clause uses operators that may only be optimized by virtual, not built-in, tables (!=, functions, MATCH etc.).
FossilOrigin-Name: f38caab23bcef1df02618376de22d208a3333d023628cde310345505933329f1
2023-01-04 17:46:29 +00:00
larrybr fd318937e7 Employ deliberate_fall_through macro to quiet some compilers.
FossilOrigin-Name: 869635fb81e0868dee80ce77653a2c1d2af41b3ffbf0a3f7b788ac99dc808887
2023-01-04 16:54:55 +00:00
drh d0fe0fc531 Enhance PRAGMA integrity_check so that it verifies that the string values
stored in indexes are byte-for-byte identical to the values in the table, and
not just equivalent according to the collating sequence.
dbsqlfuzz 686e2e205e0c0594d3fb524bea0c25e621d1a870.

FossilOrigin-Name: 9302e4bfdce5905576b8f0af7d6b3a864e1dcd58ed89bb303010a1c4f826e915
2023-01-04 15:18:52 +00:00
drh cef97068cd Remove the unused "sqlite3StackAllocZero()" macro.
FossilOrigin-Name: f0ca57207e9f3ab401264498f65c249b479d9e0da7158e001b002584ac67dd0a
2023-01-04 11:58:36 +00:00
stephan 373d34d610 Elaborate on the open-in-read-only fallback behavior of the SQLITE_OPEN_READWRITE flag, per user request. Unrelated trailing EOL whitespace cleanups.
FossilOrigin-Name: 1003144fc192e1531e1bc968d7e1d0ccc7ad31e501180f90e1479565a4abfb96
2023-01-04 11:57:27 +00:00
stephan 5d2a25b254 Remove the JS-side SQLITE_WASM_DEALLOC sanity check which triggers the problem mentioned in [688c5c13d156] and [ae0196d86ee8], for reasons covered in the code comments, per discussion in [forum:e5b20e1feb|forum post e5b20e1feb].
FossilOrigin-Name: 65ff3200c6009a1649fc108d7ce36f5c014185ba46bbf98471ec86eaeb572656
2023-01-04 03:14:06 +00:00
drh a9e1e4edaf Small performance increase in the symbol hash table.
FossilOrigin-Name: a19597b4fd2530a89363992b9179f9f3f0621a6e3861a91324f29311aafe09b9
2023-01-03 19:27:00 +00:00
drh fbde3f50bf Avoid an unnecessary call to strlen() in the sqlite3VdbeMemStringify() routine,
for a performance increase and size reduction.

FossilOrigin-Name: 284382d37850f469dc4bf3ab8efd2f20971554e897f1ba3e94d3f2f0c35d97d0
2023-01-03 18:51:18 +00:00
drh 1eb88d6e5a Small performance improvement in the btreeOverwriteCell() routine.
FossilOrigin-Name: df5ea47130d275e9fa1bd6aa8d55f1d5e136cba30ba13b7ede22224aa7f78919
2023-01-03 15:11:01 +00:00
stephan e364a9da72 Add some docs explaining a particular piece of [ae0196d86ee8]. No code changes.
FossilOrigin-Name: 7f96803c1cbd0633367173ab0a67a8b301b9e0746e460c50d506b536c15db0a0
2023-01-02 20:52:46 +00:00
stephan b1cc895c92 Another reformulation of SQLITE_WASM_DEALLOC to attempt to work around a Safari-specific quirk reported in [forum:5489305f601b8c3f|forum post 5489305f601b8c3f].
FossilOrigin-Name: ae0196d86ee8ca424b5ef5a43c32988f4ab5131ea146669bc1467e31a2384901
2023-01-02 20:07:30 +00:00
stephan a347927d60 An alternative solution to mapping SQLITE_WASM_DEALLOC to the proper function pointer in JS, to account for a Safari-specific quirk reported in [forum:e5b20e1feb|forum post e5b20e1feb].
FossilOrigin-Name: 688c5c13d156f987b895df1d5a5b770616b0d9caec4726e03ba122eb8539e7b2
2023-01-01 12:22:16 +00:00
peter.d.reid 206db97ddd Remove redundant assignment in kvvfs's decoding.
FossilOrigin-Name: 2ffbf0c73c5a0f08717093cfaac99c4aedee506ec7b5a9a62c92168a2bcadc14
2022-12-31 05:26:35 +00:00
peter.d.reid 576fe9035b Avoid an infinite loop when an unexpected character is being decoded by kvvfs
FossilOrigin-Name: 6909195f77b50650982a6afd9d72b3a6ac9aff1f86c653fa18d0d9bf64e9bd8f
2022-12-31 05:19:45 +00:00
drh 12e1eb344f A call to sqlite3_declare_vtab() should not cause DML/DDL authorization
failures.

FossilOrigin-Name: eed1e030722deb24674e7c2d165a2a359576c6bb5769d3bdd5fa645bc0f2ecc7
2022-12-29 18:54:15 +00:00
stephan 010ffff8d7 Add sqlite3changeset_new/old_js(), which work like sqlite3_preupdate_new/old_js() but on changesets.
FossilOrigin-Name: e8afad630b085a9208491e0516a6a30c9cda77a20b1aa2cba49b2f44eb9fa2f8
2022-12-27 22:46:49 +00:00
stephan 294968e030 Expose sqlite3_preupdate_hook() and friends to the JS API.
FossilOrigin-Name: cc02783a1210a083683320fae1ec1519e45b8e3003a9e32809d808513a2ce06b
2022-12-27 14:34:32 +00:00
drh 4e013284a0 Fix a missing 0 at the end of an integer literal in check-in [8da0f0c38a458c57]
FossilOrigin-Name: e3776796f55574f357eb429e20399389092a68e9ca00aa104ed33eb559ae0de5
2022-12-27 14:03:04 +00:00
dan 1dff401c42 Fix a broken assert() in fts5.
FossilOrigin-Name: ff5a9b054f4d5ad3d33787aee93b4181822c6dfc513626236d9867bb431f64da
2022-12-27 13:52:25 +00:00
stephan de75c9c5a0 Add links to module-symbols.html for newly-added APIs.
FossilOrigin-Name: 335ef03eb9694d1fd123f113f235cd34e97834fadc5f7c80489b6513be45d070
2022-12-27 12:25:28 +00:00
stephan 69d8b3636e Add two more tests to [f99f8e3ecfe20].
FossilOrigin-Name: 9cffccd335b11feb6a5beadb26e1a68af7b6b1b3c953063dcf6ef74edf3c59b1
2022-12-27 12:17:20 +00:00
stephan 55a21fbcef Expose sqlite3_commit/rollback/update_hook() to JS API.
FossilOrigin-Name: f99f8e3ecfe205337996ee61c0b9f139d3e8788b14f32e26560888b3a16564de
2022-12-27 12:13:01 +00:00
stephan c8f245ab5c Add an optional argument to oo1.DB.transaction() to specify an explicit BEGIN qualifier.
FossilOrigin-Name: 507335c12b1dbe21d180cf6f0a0deb4cc737417acb44c8f1d8fac98b86f62b01
2022-12-27 11:40:05 +00:00
stephan 84261bac96 Expose the auto-extension API to JS and reorganize some nearby code.
FossilOrigin-Name: 52b229d11d82bfb81c8b63e252c51c57a34dc50498dd685451588c185873c628
2022-12-26 17:15:05 +00:00
drh ff78b2bee8 Fix an infinite loop in the MEMSYS5 auxiliary memory allocator that occurs
for memory allocations between 1GiB and 2GiB in size.  Error introduced
by check-in [949133231f8f751a].  The problem only affects builds that
include the SQLITE_ENABLE_MEMSYS5 compile-time option.

FossilOrigin-Name: 8da0f0c38a458c57f979d59b49cf4804ef81fc2eccabde1f166bab24dd1dabea
2022-12-26 15:14:24 +00:00
stephan 64fa85bb5e Document sqlite3.capi.sqlite3_prepare_v3() as accepting an ArrayBuffer and ensure that it can.
FossilOrigin-Name: ae3ae92ec45d3d5de92e70876502f8108fc3fcd87848e86c2b83f8842f1ff139
2022-12-26 15:08:48 +00:00
stephan d9cfd0f339 Correct some internal-use JS docs and update the 'string:flexible' type conversion to accept an ArrayBuffer (as it was recently documented to).
FossilOrigin-Name: eff5d3bec29043cc1182bbb5229040dac5ff50264d025e354736bb63b4bc97a0
2022-12-26 14:55:00 +00:00
stephan 67e4f6fcdb Only add an on-db-close cleanup entry for collations if adding the collation succeeds and xCompare is-a JS function.
FossilOrigin-Name: 18a5480f1e0dca55703b43fa17685a4cc577cab8841ce47c807af02348ad85ee
2022-12-26 14:25:21 +00:00
stephan 171aa209fa Extend [0e69b2c379e618] to support uninstalling stale JS-to-WASM function pointers added on behalf of UDFs.
FossilOrigin-Name: 7a46e629dcbf97ae037c5abb39306af7ad55f1910c1e962373e09d88d8bd5a33
2022-12-26 13:45:10 +00:00
stephan 7197f0a8f2 Add JS infrastructure which enables sqlite3.capi.sqlite3_close_v2() to clean up stale JS-to-WASM collation function conversions installed on behalf of a given db handle. The same for UDF mappings is TODO.
FossilOrigin-Name: 0e69b2c379e61893c7db8a9c9d270650f2bd63b6cea30811d41136392a2e4f04
2022-12-26 13:00:58 +00:00
stephan 20170adf14 Reimplement sqlite3.capi.sqlite3_close_v2() and sqlite3session_delete() as a hand-written bindings so that they can attempt to clean up certain (potentially) FuncPtrAdapter-installed functions before closing. Correct the create-function family of JS-to-function-pointer automated conversions to include the UDF's arity as part of the mapping's key so that (un)binding a UDF to different functions for different arities works (and add tests confirming it). Correct a broken doc link in module-symbols.html.
FossilOrigin-Name: 60b262ef0f57b162c2566b12e70685a92afb00b441332ea7a6540fcb188cc7af
2022-12-26 11:13:09 +00:00
larrybr 3a8fbc0749 Grammar fixup in comment re SQLITE_TRACE_PROFILE trace event.
FossilOrigin-Name: b6dc80cbf63ed521ef8f878fba24b0110d61813763ca7bfbcfb0a145656b300a
2022-12-26 01:44:04 +00:00
stephan b87efce2c0 module-symbols.html: add some missing links to C-side API docs for recently-added JS counterparts.
FossilOrigin-Name: 7d4f4e96f33f37b0774cb8df22ed1ef33062534653a4fadf606704b855e28c49
2022-12-25 23:54:16 +00:00
stephan 7103801df1 Update JS worker #1 API and related tests for recent API changes.
FossilOrigin-Name: d797e183e96e04520636865204c02307b751fdc2949a04587de9259a1733e37b
2022-12-25 23:45:59 +00:00
stephan ec35e92d1d Rename the new wasm.irSizeof() to sizeofIR() because that seems clearer.
FossilOrigin-Name: 49a83ca3ba46d3ae8021fbdd72b48ce9181eea6845fa7bb7aa83b16fad1728be
2022-12-25 22:55:45 +00:00
stephan 6431f860ae Minor internal API doc correction and extend a test for oo1.DB.selectValues().
FossilOrigin-Name: b7295b1ca700945ae4e89be69f954dce9960e8975664e620898392b2cc78d452
2022-12-25 22:49:55 +00:00
stephan 9f1adb8c12 Add sqlite3.oo1.DB.selectValues(). Correct a logic error which could cause DB.selectValue(), DB.selectArray(), and DB.selectObject() to fail to finalize a statement if a call to bind() failed. Add more session API tests.
FossilOrigin-Name: 6adc8a10146190037d55d3328d2f78aa5233559f88d4aa70fbbf9e10145b9b6c
2022-12-25 22:44:13 +00:00
stephan 3caf13f1cb Merge wasm-session-api branch into trunk, adding the session API to the JS/WASM components.
FossilOrigin-Name: dfb8b651fa4faef2d3307a05512cdc479398484c3a59715827179c363861a777
2022-12-25 20:25:44 +00:00
stephan 9d61db1944 Add basic session API JS tests.
FossilOrigin-Name: 64e032602cf420851c8029603c76f5512000d1c9a40fa7a545528d69d6d1d4cc
2022-12-25 20:22:20 +00:00
stephan 7d59d90a5b Add sqlite3.wasm.irSizeof() and extend certain allocation functions to make use of it.
FossilOrigin-Name: 1cbc7b1875e8611b9db7a747b4c9499501450deaf90c929d212511837d6f72b6
2022-12-25 20:05:11 +00:00
stephan 7a5544b958 Add tests confirming that JS sqlite3_exec()'s SQL argument participates in the 'string:flexible' type conversion.
FossilOrigin-Name: 926d0c61a391c601adc2804d3fdaa8b667ae2abd565939cddfa12d5151b098f8
2022-12-25 17:12:24 +00:00
stephan 73bf9d5fed Replace the "manual" implementation of sqlite3.capi.sqlite3_exec() with a briefer "automated" one via the [7f9ace1b11a67] feature addition. Minor code-adjacent internal cleanups.
FossilOrigin-Name: 4888957baf18c6763f959fbba998a74156ff656368779107f502b926e9e9d949
2022-12-25 17:09:34 +00:00
stephan 031ee6b9da Remove duplicated JS binding of sqlite3changeset_apply_v2().
FossilOrigin-Name: 2e8336b6e1cba89dbcc11d6316e39c929bf8b018a18b92efc232abd47e0a5cc6
2022-12-25 16:20:05 +00:00
stephan 8b5a5ef030 Add the address of the associated db handle to the sqlite3_trace_v2() output originating from sqlite3.oo1.DB's trace flag.
FossilOrigin-Name: 0a36568c5a3db56a8ad23fd8abc672c5a7f1abed55984d902842ffd3ebb816e8
2022-12-25 15:28:19 +00:00
stephan 3494ec15e5 Update the session-related JS bindings to account for today's internal API changes.
FossilOrigin-Name: be63944d4114f53f2dab65bc6c1b85f4766a4ea14ee7b2690acde239a2a0bf54
2022-12-25 15:14:10 +00:00
stephan 04071524ae Merge trunk into wasm-session-api branch.
FossilOrigin-Name: 7f8f1acd82be7dc2eb2147d96299b1b443e86774dfe0b0a8d32669a0500fc9c6
2022-12-25 14:13:52 +00:00
stephan 7015aa9f49 Add missing sqlite3_context_db_handle() JS binding. Reimplement sqlite3_set_authorizer() and sqlite3_set_auxdata() JS bindings to take advantage of [7f9ace1b11a67]. Teach FuncPtrAdapter to emit a console.warn() message if it is invoked after the library is bootstrapped, the goal being to inform users that it's an internal API and should not be invoked from client-side code.
FossilOrigin-Name: 8e3d4f6294037396e388ec21abb18bf0201a6bec6ff004730cc5d11b705a6d2b
2022-12-25 14:04:06 +00:00
stephan 485229e147 Enhance sqlite3.wasm.xWrap.FuncPtrAdapter to be able to handle sqlite3_create_function() and friends and reimplement those bindings to use this feature (this will also simplify certain session API bindings). Interal API changes only with no client-side breakage.
FossilOrigin-Name: 7f9ace1b11a6703031790af9cf08ab25df25850a86e6ca2a7aeaefd8aa395e6d
2022-12-25 12:51:53 +00:00
stephan 75c04ba89c Simplify the signature for JS functions, as opposed to function pointers, passed to sqlite3_exec(), eliminating the superfluous initial two arguments. Update related tests to demonstrate both function-passing approaches.
FossilOrigin-Name: e7cc70cdda426863f82ebe1305f4c3053824c5a605b1516b0b7f205f1203178b
2022-12-25 10:22:27 +00:00
stephan 60a1a0f7df Incorporate wasi-sdk RANLIB configure script patch from the VMware OCTO team.
FossilOrigin-Name: 478fe96ee7f3ffd5732231b75e6f04e898a59368b3c91f79e6af9496dfcef6d3
2022-12-24 15:34:16 +00:00
stephan 6e893aee95 Merge trunk into wasi-patches branch.
FossilOrigin-Name: 52f40ab12e437c59af2b91c7ac105ab7784db57fc8d9ab7a1356f17092681f43
2022-12-24 15:31:56 +00:00
stephan 4099b3cab3 Replace JS-side use of SQLITE_TRANSIENT with the new SQLITE_WASM_DEALLOC, reducing the amount allocation/copying required by sqlite3_bind_blob/text() and sqlite3_result_blob/text(). Remove the 'experimental' log message from the virtual table tests.
FossilOrigin-Name: ffe2999a91a7dec129a38afb675fe9e539d7c347886bfea85cba55f6367d54d1
2022-12-24 15:28:45 +00:00
stephan cede6384fd Extend oo1.Stmt.bind() to accept ArrayBuffer instances to bind as blobs.
FossilOrigin-Name: f76bd30137fbff981625ffcb28cddd5e8651803dfc3f2d8d7801ead33496311d
2022-12-24 14:16:02 +00:00
stephan 0db1c90137 Add a test for the (failure) case of client-level code calling the oo1.Stmt constructor directly.
FossilOrigin-Name: 6a37874db04f3b4842994ad17fc74cb6222f8ea0fa1315a23aff1ffa69bcd12a
2022-12-24 13:46:27 +00:00
stephan 4b4ae86445 oo1.DB.exec() rowMode="$columnName": a minor optimization and a preemtive fix for a hypothetical corner-case bug.
FossilOrigin-Name: 0b2df22bd92914708ad0851d0401ad2cf3edb1968b88b2c07fe40792a731c5ba
2022-12-24 11:16:49 +00:00
stephan 0cdc01d01a If sqlite3.oo1.DB.exec()'s callback returns a literal false, stop step()ing over results as if the end of the result set had been reached. Unrelated minor code-adjacent cleanups.
FossilOrigin-Name: 33a58c8ece3b37a8edc3434af36643e2d489a53a672d778c5b77e66666045c30
2022-12-24 01:59:42 +00:00
stephan 027afccdcd Reimplement JS's sqlite3_bind_text/blob() with hand-written bindings to permit more flexible inputs. Add automated JS-to-C function conversion to sqlite3_busy_handler(). sqlite3.wasm.xWrap()'s '*' argument conversion no longer treats JS strings as C-strings: those conversions require explicit opt-in via the 'string' converter (or equivalent).
FossilOrigin-Name: 96ba44946b3e88b6aa305c4363cbbfeab0d9120b3d8c4d2587d68b9293ea7cc6
2022-12-23 23:46:33 +00:00
stephan bb769f0b4a Merge trunk into wasm-session-api branch.
FossilOrigin-Name: 6cdb036d8e8c5ddb0c6578aeefe318e74d7a90228e57b9f9047057dae3252963
2022-12-23 21:23:26 +00:00
stephan ab9c2d571e Internal JS cleanups. Correct part of [ac136925a645] to account for the eTextRep flag being able to hold flags other than the encoding.
FossilOrigin-Name: 1dfc03ab1e0269807beef27bf884ab9ead7553d4a5f6ed213f812d7fa052045f
2022-12-23 21:10:49 +00:00
stephan 77eac0507c Merge trunk into wasm-session-api branch.
FossilOrigin-Name: f1decc831fe0dc8523956e74474d9663852b0e5b56240dd8504952726e713a97
2022-12-23 19:33:22 +00:00
stephan 3705f38ab0 Consolidate/unify how the JS bindings of the create_function/collation family of functions react to a non-UTF8 encoding: they now treat a falsy value as SQLITE_UTF8 and fail with SQLITE_FORMAT for an invalid encoding.
FossilOrigin-Name: deffe6fb211410fa1a1fbca824a52b4e09b54d4b4f4a4e12d71c9e4b7e8606fb
2022-12-23 19:16:45 +00:00
larrybr 19d14f9717 Add base64() and base85() to shell sources for the non-configured makefiles, too.
FossilOrigin-Name: ac136925a6453d3e53c7a380911dfeac5706d49f936294289f6ea0b74e26e18a
2022-12-23 19:11:57 +00:00
larrybr 0953c5354a Add base64() and base85() text/blob conversions to the CLI.
FossilOrigin-Name: 4bc98a2d9520efa9b80142163cbfab72a5f2fe9854cd6ba8291dcefdb872e657
2022-12-23 19:04:59 +00:00
stephan ace682e149 Explicitly omit threading and extension loading from the ext/wasm/c-pp binary build to avoid a link error on some systems.
FossilOrigin-Name: 5d9a14715c25efcd81cadafabf03aad7213251bd1b3dc181939c2dba8d942fb6
2022-12-23 18:38:14 +00:00
stephan 7298c2e099 Add SQLITE_ENABLE_MATH_FUNCTIONS to the list of feature flags in sqlite3-wasm.c.
FossilOrigin-Name: 58503cd148c9613abfaf7c1386c34806150bd521966864ccbb821ea7dede8e5a
2022-12-23 18:25:48 +00:00
stephan a9e1d96cd8 Cherry-pick [c4dab53b8ea3401abd] for sqlite3.wasm.xWrap() optimizations.
FossilOrigin-Name: 9b97412d3aa791870016ab3c6f565b6a6afa1764f98e969833aec093b9b29919
2022-12-23 18:19:28 +00:00
stephan b5915699d0 Internal cleanups and minor speed optimizations in the sqlite3.wasm.xWrap() infrastructure.
FossilOrigin-Name: c4dab53b8ea3401abd57671b8f3cb39fa4431b864d4c4e14ae24592f8d4cba0a
2022-12-23 18:14:36 +00:00
drh 3547e4997f Fix lots of harmless, nuisance compiler warnings, mostly unused parameter
warnings in extensions.

FossilOrigin-Name: c14bbe1606c1450b709970f922b94a641dfc8f9bd09126501d7dc4db99ea4772
2022-12-23 14:49:24 +00:00
stephan 0f29f17bf6 Add sqlite3.capi JS bindings for the sqlite3session_...(), sqlite3changeset_...() and sqlite3changegroup_...() APIs, noting that they are completely untested. Aside from missing tests, these bindings reveal a slight string-argument-type shortcoming in the callback function pointer "reverse binding" which should ideally be resolved before publishing them.
FossilOrigin-Name: 0a39172ee134816f5ce17a403b960e9c22bb56efd5bcf77ecde465efe0d88b1d
2022-12-23 14:11:54 +00:00
drh de5f3af2b9 Additional fixes for yet more completely harmless compiler warnings.
FossilOrigin-Name: 7d3772f0bd0e2602fe919573b49001da4e2b9f3874cb0183dea675204afa7abd
2022-12-23 11:46:57 +00:00
stephan cc1cc9d7b7 Initial pieces for binding the session API to JS. Far from complete. See [forum:210e36a1e3 | forum post 210e36a1e3] for the discussion.
FossilOrigin-Name: cd8c100808da1043fcf63555f48f30c90272c48c6627321ceb0a0995b34733d1
2022-12-23 11:46:26 +00:00
stephan dc02d5658b Squelch a new (and, in this case, harmless) compiler warning.
FossilOrigin-Name: a02e19dd6ce00492f3d187e3c3c9bde4d9d1ee9a23616e62ea3590eec95652bd
2022-12-23 11:32:06 +00:00
drh f26bad66ba Asserts added to ensure that the iCompare variable in the bytecode engine
is correctly initialized before it is used.

FossilOrigin-Name: 7b5900a111b9410f7d60c937e5a56304f2f66b94cd0881e94abcc5eedde52514
2022-12-22 21:32:58 +00:00
drh cce70d52d0 Avoid having to reinitialize Vdbe.pResultRow upon each call to sqlite3_step()
for a small size reduction and performance increase.

FossilOrigin-Name: 6a00d67f5955ab86eea982c27b3a03b680fdf644ec63f49586ade6342a4d64a6
2022-12-22 19:12:48 +00:00
drh edc2713fdc Rename the Vdbe.pResultSet field to pResultRow in order to better distinguish
it from other variables with similar names.

FossilOrigin-Name: 1fd6211ef7bd26ed625177bfedfd5153ace547de6a71365ecfa076578d043f1a
2022-12-22 18:44:39 +00:00
drh c2d853e562 Small performance optimization associated with shared cache in the
byte-code engine.

FossilOrigin-Name: 3181331c1c0259d5cd274dcb33faba930dae51b1f0fe51e8a0318d9c564b94f9
2022-12-22 18:35:21 +00:00
drh 57366d8cec Small performance optimization on the OP_Insert opcode.
FossilOrigin-Name: 781fdcb9ce85aa2844ef8c00cf908f1a87eeff80dadaf73a71f88b4279260e57
2022-12-22 17:36:02 +00:00
stephan 913dcaf0ca sessionfuzz.c: use lt/gt brackets instead of double-quotes when #including zlib.h, as reported in [forum:91a104bd65 | forum post 91a104bd65].
FossilOrigin-Name: 7d7780c350f3239f9c9feb91924a01a4eba31b88060946c766719c9d50c16bcd
2022-12-22 10:53:08 +00:00
drh d50f1f4f5b Merge the latest trunk enhancements into the begin-concurrent-pnu-wal2
branch.

FossilOrigin-Name: bcab19d21be42121f1ea3961aedbbe463c60d85d0a3dd8b9593040d217bac3a7
2022-12-21 20:43:53 +00:00
drh 77508c4768 Merge the latest trunk enhancements into the wal2 branch.
FossilOrigin-Name: 21ebd4cfcf843925cf8eaeb2f7e196713dba116cf618aea9f9f40e31f6aa6653
2022-12-21 20:28:58 +00:00
drh 79b9bc4452 More efficient implementation of large precisions on the "%c" format for
the built-in printf().  This is an effort to avoid a reported
timeout on a (ridiculous) query generated by OSSFuzz.

FossilOrigin-Name: 371f9b88387a44a5f820279d79733d1deb7eafc72f320ec47a11679bbdbb49ef
2022-12-21 19:11:56 +00:00
drh fe83892bc1 Ensure that the expression of a virtual column really is an expression and
not just a reference to another column, as a real expression is necessary for
the indexed expression coverage optimization to work properly.
[forum:/forumpost/07b36e3899a9ae21|Forum thread 07b36e3899a9ae21].

FossilOrigin-Name: 40549bacb3923e439627b0103bedd7da30258b69a46960040f7176e060f51f2f
2022-12-21 14:13:49 +00:00
drh 4f0c36b327 New WHERETRACE entries for adding and removing objects from the
Parse.pIdxEpr list as part of indexed expression coverage.  Debug and
testing changes only - no affect on production builds.

FossilOrigin-Name: 4dc438a951bdbe27fbd5ee7cfb907e30ec0efbf8ce02109f18997cc221e0ad3b
2022-12-21 12:18:06 +00:00
drh 87bdc152a1 Add about 150 new fuzzer cases to test/fuzzdata8.db.
FossilOrigin-Name: a8d8d9cd65e64e0a492a80c00050f54ab75de82651230049f65c9d2b78a164f2
2022-12-20 15:22:57 +00:00
drh 2656b1b0a0 Add ALWAYS() to always-true branches in the locking logic of the memdb VFS.
FossilOrigin-Name: 01fa760212a40cec5b43da99b917ab6389561c96d0567a9f67f516bdfa8f5f3d
2022-12-20 15:02:44 +00:00
drh 1379b9f51a Improvement to the dbsqlfuzz fix of [f113eebdbe68246f].
FossilOrigin-Name: 79d62956f206cb2987891d4ba5985cece21fc8c8c1158f46c2951f65c0dcf4e8
2022-12-20 14:02:35 +00:00
drh 98164c3452 Do not use indexed expressions to replace the expression being indexed on a
RIGHT JOIN as that leads to problems, as shown by
dbsqlfuzz a644e70d7683a7ca59c71861a153c1dccf8850b9.

FossilOrigin-Name: f113eebdbe68246fd38d640b0cdf0fd73ab5d2abad9226274dfa2e41fe286bf0
2022-12-20 01:48:43 +00:00
dan 73e460c932 Fix a faulty assert() in fts5.
FossilOrigin-Name: 569018170b928cadfa0745db631981084e595d90a8c3d1e236c42adae161018f
2022-12-19 20:14:22 +00:00
drh b7fe3cb890 Remove an unused variable.
FossilOrigin-Name: 5c4d94147d290d4a68ac0d5ae8f9e6bbfe7522ad58b2298e387f8c699048e111
2022-12-19 19:04:32 +00:00
stephan c6c1d7bfbd Minor sed script tweak in the wasm GNUmakefile for BSD portability, as reported in [forum:4a97813fcbd4f63e|forum post 4a97813fcbd4f63e].
FossilOrigin-Name: 49efcf6d9f894c28a3ffb350a7c5f0ffd12c7cbf61f5c27f76eb7f777b8a24c5
2022-12-19 14:46:17 +00:00
stephan 3ce8badcba Update test-opfs-vfs.js to account for recent API changes. Reported in [forum:4a97813fcbd4f63e|forum post 4a97813fcbd4f63e].
FossilOrigin-Name: 2a636746360c109f1d39fc352b9f1c83a739735520252a264c774e848597475d
2022-12-19 14:43:17 +00:00
dan 1532f9d155 Fix an assert() in fts5. Simplify memdb xLock/xUnlock some.
FossilOrigin-Name: 7fe158aa8071acadd959b2b4a4b66e8a7f7eecec207ba681abf516b0457c2921
2022-12-19 14:06:36 +00:00
drh a4569bd5d0 Fix a minor problem with the printAggInfo() routine (used on during
debugging).

FossilOrigin-Name: c8329967f32efedfc8244631a5682f7f7d21c1b3fcd7d5cc67716555716d4926
2022-12-19 12:25:41 +00:00
stephan ba1e6f8279 JS namespace updates in ext/wasm/api/README.md.
FossilOrigin-Name: ca59e736b1839c30a820dd5795b4534e1565387ef701423451ea4c503a9701ea
2022-12-18 12:00:10 +00:00
drh b7959e3419 Fix an unnecessarily restrictive assert() in the aggregate logic.
dbsqlfuzz 699bee2aa26c5dc84afabf6894685c316d936485

FossilOrigin-Name: 8cc0af81ad03ac65f230d78cf97e1448c2d93d556467ea3ef700d5abd4a3890c
2022-12-17 22:10:05 +00:00
larrybr 88fbe16154 Remove slew of inadvertant tabs in favor of spaces.
FossilOrigin-Name: 0aa6da69e12e483b5561bb9a054ec27a99f51772340c950783e342e6dd39beb9
2022-12-17 19:56:28 +00:00
drh 1c70429569 Add NEVER/ALWAYS to branches that have become unreachable due to recent
enhancements.

FossilOrigin-Name: b4281db4e4762773684d27b809ef911da08b81cb953b3666e535b990d4d5f701
2022-12-17 12:49:04 +00:00
stephan a4d04c59ba Update wasmfs.make to get WASMFS building again, but changes made to OPFS-over-WASMFS since we last tested it have made it incompatible with how we used it. It can now only be used from worker threads, eliminating the one benefit it had over the sqlite3_vfs OPFS implementation. Remove/amend references to WASMFS in the docs and remove all WASMFS-specific test app links from index.html.
FossilOrigin-Name: 32dc98c8001387b1970623192ec6cb89445000fa49bda5f4bd7c525bbe20bb5b
2022-12-17 11:14:35 +00:00
drh a9d18d83fc Disallow some of the query invariant tests on SQL statements that contain
a GROUP BY since output terms that are no covered by the GROUP BY are
indeterminant.

FossilOrigin-Name: 28ba1fec4519e7ca9466f1a2f787c905f2cf880d38a004639f99f948e46a51e7
2022-12-16 18:32:07 +00:00
stephan e9f740e484 Expose a JS-friendly subset of sqlite3_config() to JS, with the notable caveats that (1) setting up the JS bindings requires starting the library, making sqlite3_config() illegal to call and (2) calling sqlite3_shutdown() in order to make it legal to call sqlite3_config() may undo certain JS-side library initialization. Move sqlite3_(de)serialize() into the int64-mode-only bindings because of their int64 args.
FossilOrigin-Name: 62e0c931ac952219f68e22d64e20836781bf330b4581e4662266172a97c9289b
2022-12-16 13:04:21 +00:00
drh d980442188 In the fuzzer invariant checker, do not add new WHERE clause terms that
make use of an ambiguous column name.

FossilOrigin-Name: d5b46541c30bcbeb7e57b5b5951856d564e81f7f9638d66d205157797964418c
2022-12-16 12:07:48 +00:00
stephan 7c1c5b6232 Add a test confirming that exceptions are not passed through C-space if a JS-side sqlite3_set_authorizer() callback throws.
FossilOrigin-Name: 5a77c2c7aaa556007274e1b85790934665c2c12661ca11d896eb7d09cd49ce72
2022-12-16 11:33:42 +00:00
stephan d83ab0cf85 Add sqlite3_set_authorizer() support and related tests to JS.
FossilOrigin-Name: 551b848894c249cb3c9d237643d2ed53ffcb3b003d0cf3f797a535df1731ce39
2022-12-16 11:13:32 +00:00
drh 00d6b2755f Create a new affinity called FLEXNUM that works like NUMERIC except that it
never tries to convert integer to real or real to integer.  The affinity is
only used internally - it is not possible to create a table column with this
affinity.  This affinity is used on subqueries and views that are built off
of a compound SELECT and where the datatype is controlled by a CAST expression.
dbsqlfuzz c9ee6f9a0a8b8fefb02cf69de2a8b67ca39525c8

FossilOrigin-Name: 44135d6ea84f7ba6b36549954b38a8bc048d5ffea5a9779e35950afa4eb2dfb2
2022-12-15 20:03:08 +00:00
dan 118b53bd21 Fix an incompatibility between the Tcl interface and the "memdb" vfs by allowing memdb to accept filenames that begin with '\' characters.
FossilOrigin-Name: bd537f2057a4800bd30e7dd57405c3e57df649471104c80bd32573a89568029e
2022-12-15 18:56:12 +00:00
stephan a51e044a5b Added a couple missing 'not part of the public API' tags on functions in sqlite3-wasm.c. No code changes.
FossilOrigin-Name: eb27feecea508f3491a09699f2339951facd2345d479cfd3020183dc2af703b2
2022-12-15 18:50:30 +00:00
drh f000c9b638 Omit the columnTypeImpl() function from the build when SQLITE_OMIT_DECLTYPE
is defined, because it is no longer used in that case.

FossilOrigin-Name: 751e344f4cd2045caf97920cc9f4571caf0de1ba83b94ded902a03b36c10a389
2022-12-15 15:37:52 +00:00
drh 569aef04b0 Fix harmless compiler warnings.
FossilOrigin-Name: ddb260cc52efce6472b9423cbc6b788d8c38746f1842c9f1c184c08c651b3387
2022-12-15 13:45:49 +00:00
dan 8b58fbf5ee Update a test case in scanstatus2.test to account for recent planner enhancements.
FossilOrigin-Name: 53e07dc6b6521bb7757d0bc2cf12a501bb929afafdb16737058496afca8e6742
2022-12-15 11:39:12 +00:00
dan 7acee13eb2 Fix an unsafe use of sqlite3_mprintf() in sqlite3_overload_function() identified by forum post: <https://sqlite.org/forum/forumpost/95b338860d>.
FossilOrigin-Name: 9fa2b94c2e0fd43c1a9c15a79fe1325afa1699f0685dcd039024a80185cc5658
2022-12-15 11:31:11 +00:00
stephan 73b471964b Internal refactoring of how sqlite3.wasm.xWrap() handles JS-to-C function pointer conversions, to enable similar conversions to be added more easily.
FossilOrigin-Name: 10cfe3fae6f680d3ecc3b0afbbf628ce91e34e3757b19dd27c231f0daf44232a
2022-12-15 02:28:55 +00:00
stephan d60061616f Remove an unnecessary/obsolete Emscripten-specific export.
FossilOrigin-Name: fa278022afd6dd6e499d26f74a8359f3e9973e1680772059ce331b64e77ec582
2022-12-15 02:26:13 +00:00
drh da69802e92 Fix minor problems in the new sqlite3ExprDataType() function.
FossilOrigin-Name: c1d5261b222bbf94c20e558089f3d2eae6a88b6d739225ee4f7d0338e0e59994
2022-12-14 23:53:29 +00:00
drh 70ac36e877 Adjust the query planner to be more aggressive about using co-routines rather
than materializing subqueries and views.  Implementation of enhancement
request [73ebbb3a3d1a7bd3].

FossilOrigin-Name: 0b58d1b354a0f91bafeafee102acedd4654dc420afdf4eb4aaac9bf5bcb63c1f
2022-12-14 18:34:05 +00:00
stephan 1a1d8bbdc3 Rename the new peek/pokeF32() and peek/pokeF64() to peek/poke32f() and peek/poke64f() for consistency with related APIs and because they're easier on both the eyes and the fingers.
FossilOrigin-Name: cc86f03ed698a071cd02ef3dfaba69092226bff20c917d662f808061627ad285
2022-12-14 18:06:01 +00:00
drh 604314d94e Merge all the latest trunk enhancements into the coroutines-exp2 enhancement
branch.

FossilOrigin-Name: ea278438b72e85d217e72c836cbefd68bd8b336baf33507b2d8d12ef436424cd
2022-12-14 17:52:15 +00:00
drh daa46142ec Rework the code that computes the datatypes and affinities for subqueries
and views.  If the subquery/view is constructed from a compound SELECT where
the affinity varies across arms of the compound, set the affinity for the
corresponding column in the constructed view to be BLOB so that it is allowed
to take on any data type.

FossilOrigin-Name: 27655c9353620aa58105e87d1e171d1f0a637deedde41c081824078385cd49ac
2022-12-14 17:46:35 +00:00
stephan e5e61d40e9 Slight reformulation of some test code to test multi-arg wasm.peek() call form.
FossilOrigin-Name: 349bb28f614bc210b3283ed1c1144c45fbc7d9e2ffc3dc6193ccb9932d347efb
2022-12-14 16:40:17 +00:00
drh ed07d0ea76 If the SELECT that implements a subquery or a VIEW or a CREATE TABLE AS is
a compound with different result datatypes on two or more arms of the compound,
then the overall column type becomes BLOB (ANY).

FossilOrigin-Name: 6ebb178c6b688ebd632e91a5ec4c748567833037c679ab3d72fb770deb230fe1
2022-12-14 14:41:35 +00:00
stephan 8a5c032b07 Use the new peek/poke() variants throughout tester1.js.
FossilOrigin-Name: ef7b783552f7783a728837fcfc81e493b135a7aac59747397e7e3d28f4309f95
2022-12-14 14:40:36 +00:00
stephan feb9123a8c Add convenience variants of sqlite3.wasm.peek/poke() for each numeric type to help reduce errors related to typos in the final argument (type-name strings). If wasm.xWrap.FuncPtrAdapter is called as a function, instead of a constructor, it now behaves as if it were called as a constructor (previously it threw an exception).
FossilOrigin-Name: 14e3fc01b929fa3f9a2fdbd93deb4a8aad58c111d46369c772def0437152fa75
2022-12-14 14:28:54 +00:00
drh a8b5c8739c Back out the part of the change in [88a05141c28e5ff1] that adds affinity to
the materialization of a view, as the affinity can be undefined for a compound
query.  This passes all TCL tests, but shows failures in the TH3 tests derived
from [forum:/forumpost/6f842bc5b2dadcb2|forum post 6f842bc5b2dadcb2], presumably
because the WHERE clause of the query uses constraints of the form
"source_crs_code='8675'" instead of "source_crs_code=8675".  Perhaps further
changes on this branch should reimplement affinity on joins in cases where
the affinity is unambiguous.

FossilOrigin-Name: fe5a77bcc4de8f49cc4fe6bd2e2e1f31da8d3bc84120daaa99eb853b06291d15
2022-12-14 09:06:45 +00:00
stephan bca56f384f Add a test to demonstrate/verify which object acts as "this" in a oo1.DB.exec() callback.
FossilOrigin-Name: 8fc4b3f10f61e965853a22bf7a64ad2511ef656d66658a7910bac35b0f9805ec
2022-12-14 08:12:01 +00:00
stephan b943df9323 A micro-optimization in sqlite3.oo1.DB.exec(). Changed the rowMode option of that method to only accept $X instead of $X, @X, and :X, as the extra options are entirely superfluous and may lead to confusion.
FossilOrigin-Name: 82a6c7fdf59729c182545b15c084b136e34513f340e87a7b6e5aa199117357b0
2022-12-14 08:01:34 +00:00
dan 5d75eb3fbf Fix a comment in ext/session/test_session.c.
FossilOrigin-Name: bd814af14b6d67c4f8cf6482a0657f9437811a2820196ea6ccf22514291fe360
2022-12-13 19:52:23 +00:00
dan e1dcb44307 Prevent the sessions rebaser from generating changesets containing UPDATE records for which non-PK fields are present in the old.* but not the new.* record. Also update the changeset iterator to work around such changesets.
FossilOrigin-Name: f9cd23dffba06b1982c0a5e5362dba53eba768120a2daa985b4f649d3fea1427
2022-12-13 19:40:56 +00:00
drh c799217cc8 Remove an unused variable from sqlite3SubqueryColumnTypes().
FossilOrigin-Name: c6e3ff396947e6954e77195419b35de8b39bd540d3f4bff456bab25b512e1792
2022-12-13 16:59:09 +00:00
drh 177b8c9c28 Use standard datatype names for the column datatypes in views and subqueries.
FossilOrigin-Name: baf2c2413346feb72ecb0bac800be9b945ebb4e01b42212132434d8ddef9cad0
2022-12-13 16:55:23 +00:00
drh 9e66087a49 Refactor the sqlite3SelectAddColumnTypeAndCollation() routine. Improved
comments.  Now called sqlite3SubqueryColumnTypes().

FossilOrigin-Name: 4dfb1b450b87e2c6207c83c102b785781d1dbdeadbeeab6a83f96d58bb96cfbf
2022-12-13 15:54:43 +00:00
dan 83084267e0 Add extra test cases for applying sessions module changesets to databases that have been modified using "ALTER TABLE .. ADD COLUMN".
FossilOrigin-Name: a7b404f21f657f395eddb32e218eae14e09df08fa03f68ca8bba79ea322ce8ba
2022-12-13 14:59:28 +00:00
drh 6042e30bda Slightly faster implementation of sqlite3ExprAffinity().
FossilOrigin-Name: 01cf3278c9c00dddcac67aa4b22ca26a52c31932cba35daa634a56b4c264bdeb
2022-12-13 13:33:28 +00:00
stephan 3c9749ab74 Document the role of the sqlite3.client JS API property.
FossilOrigin-Name: 5f11f1c7d5f8b627acd2ace277521d39fe2ffb0ef1dfe532e65fb56ea3dfb4d8
2022-12-13 10:54:12 +00:00
stephan 30f50a2d34 Extend the sqlite3.wasm function pointer argument converter to be able to handle the "two-layered context" of sqlite3_create_collation() and friends and make use of FuncPtrAdapter to perform JS-to-WASM function conversion for them.
FossilOrigin-Name: 0a60b7215e433f8c50027c70731b11e58d74c90ec5903e66ae42f9c98e40b044
2022-12-13 08:25:28 +00:00
drh 393021938d Merge recent trunk changes into the coroutine-exp2 branch.
FossilOrigin-Name: c43f433bcab29db0f1f8afd3948f5a4149e1f277c853c66f99c31f226d82bc0f
2022-12-13 00:51:34 +00:00
drh 7ca4312ff2 Further fix for ticket [57c47526c34f01e8]. If a subquery has a result set
column of the form "CAST(expr AS NUMERIC)" do not give that column NUMERIC
affinity.  NUMERIC affinity always goes to an integer if able, but a CAST
to numeric is a no-op if the input is a number.  So the two are not equivalent.

FossilOrigin-Name: f0325359d5795237b79f90f21b42d7d942c7e918137cb0231d404365d3041e81
2022-12-12 21:22:23 +00:00
drh 89e160a96a If a subquery has a result column of the form "CAST(... AS NUMERIC)" then
give that column no affinity rather than NUMERIC affinity.  This is because
casting to numeric preserves real values that could be integers but numeric
affinity does not.  By using no affinity on the column, we make the behavior
consistent if the subquery is implemented as a co-routine or is materialized.

FossilOrigin-Name: ece07d091c2ef3367a914187e0b6512c1f2390b8c34844536ad50e88c7e8c2f2
2022-12-12 18:58:53 +00:00
stephan 33ce0b3ff3 Revert part of [9386d6f63468] because the new automatic function pointer binding cannot properly track per-context function mappings when the context is more complex than a single context-type pointer. e.g. it is fine for sqlite3_trace_v2() but it breaks down with sqlite3_create_collation() because that one needs to use the collation name as part of the context key and we cannot sensibly do so with the current code.
FossilOrigin-Name: 6cd21b79075367227b57bccf829cc7d4ccc7d7fbcfaed226b4c8e942ddae4eb6
2022-12-12 18:42:39 +00:00
dan d8e07c780c Add new logging code SQLITE_NOTICE_RBU and use it when logging for the spurious error that RBU injects into SQLite as part of applying an update.
FossilOrigin-Name: cd881d35150be7f28cc1ca1eca0e950b5a039bef61190fcae4f944ef0e91f234
2022-12-12 17:33:36 +00:00
stephan 124fc52d96 Move JS-to-C binding signatures from sqlite3-api-prologue.js to sqlite3-api-glue.js to allow for use of the new/experimental sqlite3.wasm.xWrap() feature which automatically binds JS functions to WASM/C as needed, which simplifies creation of bindings which take C function pointers. Reimplement sqlite3_exec(), sqlite3_create_collation(), sqlite3_progress_handler() to use this new feature.
FossilOrigin-Name: 9386d6f634680b4e0fa5487c34c63acb29f0b7a6ae738b8f6164ad084a229b62
2022-12-12 14:31:38 +00:00
stephan 5dbfc0dfdd ext/wasm/module-symbols.html: for C APIs which have extended JS-side semantics, link to the JS-side API docs instead of the C docs.
FossilOrigin-Name: d557015208f504c6d5d20ebf1e451b3f07b19590d76371b16a9f4b54e9645282
2022-12-12 11:38:22 +00:00
stephan 13e5512b28 ext/wasm/module-symbols.html: add a hyperlink to the API docs for each exported sqlite3_...() function.
FossilOrigin-Name: 01d3a9bba3120cfec3f752048281f18ce7ab48fa7584750c097233a9ce095a20
2022-12-12 11:22:05 +00:00
stephan b2eb8a5352 Rework the internals of how argument/result types are converted to/from wasm in prep for attempting to support automated conversions of function pointer argument types.
FossilOrigin-Name: 58052d8285cbc2aa8c4f8a9f5c2d33ac12014f6a70afbc93bc4ce5b9fd1f9ee5
2022-12-12 08:16:47 +00:00
stephan 81aca2574a Add sqlite3.mjs to the new 'quick' wasm build for the sake of the snapshot build.
FossilOrigin-Name: 5a3f2224c37c7e28ce627bd98a9907a16635bffeea36e8ead707586aad37ccfe
2022-12-12 07:52:22 +00:00
stephan fda3c30b7d Expose sqlite3_txn_state() to wasm.
FossilOrigin-Name: f6f3397a4667b15a96bdf4189c54789d622fd90351405e963d7f56ac93a9344c
2022-12-12 07:31:37 +00:00
stephan 9bda4c53c1 Expose a number of infrequently-used sqlite3_...() functions to wasm whose absences were noticed while documenting.
FossilOrigin-Name: 3144e7c0b8633d99daa07d75dabfe6e115ad401110f6f3c85f768e973299f3d9
2022-12-12 07:27:22 +00:00
stephan 56bc31588e Minor build tweaks in ext/wasm to create a faster-running build for the common dev-mode cases. No code changes.
FossilOrigin-Name: 6a61b14a6cf8279a5b8b77f934c2b97965f5e3fe4cddb03e78c2c0a840dbe6f6
2022-12-11 07:53:03 +00:00
dan 1d0de81f84 Ensure that the types and collation sequences of a view are taken from the leftmost component of compound SELECTs, even when column names are explicitly provided for the view.
FossilOrigin-Name: 6f0bf892695acaa2f6fd1fccc767340caf5edfdb8b0816bda33d52cc21d5da45
2022-12-10 19:22:39 +00:00
dan 6fce561905 Ensure that the types and collation sequences of a view are taken from the leftmost component of compound SELECTs, even when column names are explicitly provided for the view. Possible fix for [679ed6a2].
FossilOrigin-Name: b0e49aa81a09c183b8baf8346b8a9b593c2b13073958617b6594500958d5bd46
2022-12-10 17:54:55 +00:00
stephan e8bbf12ad3 Remove -flto from wasm build flags, as it increases build time and doesn't seem to have a measurable benefit.
FossilOrigin-Name: 35a88747eb0bb9eebb60550f91d0fe0c7a5ae11ec0a1315ae056fd79e9436a3f
2022-12-10 17:45:40 +00:00
stephan 130cd85026 Correct wasm heap corruption introduced in test code added in [e144fd5c88fa4] which led to unpredictable failures.
FossilOrigin-Name: 6c1e07e3857b38e394853891a9028e88319bad4bc7b7b15078e25f430c3b1222
2022-12-10 17:20:29 +00:00
stephan 8c0041f5bf Remove two incorrect calls to structType.dipose() which prematurely freed objects in use by the virtual table test/demo code.
FossilOrigin-Name: 060eb2848975a24ff6683a8a9c4d7546ae36147323b0edae01fb42f52d9bb2d6
2022-12-10 15:41:47 +00:00
stephan 9cb6ff2792 Expose sqlite3_column_value() to WASM and add sqlite3_column_js().
FossilOrigin-Name: 7783aa4af1331190fd1f42a71bb724041e2e82b51745f9740926e4ead83a97ed
2022-12-10 15:13:29 +00:00
drh 22166f83b4 Rerun autoconf using version 2.69 to minimize configure script differences
with trunk.

FossilOrigin-Name: b63342839427917ae48e13b959457baa5c2e0e1be8a444b0e1a4e0446ace9178
2022-12-10 12:06:31 +00:00
stephan 5c99d91e53 Refactor the internal JS routines for converting UDF results and errors to JS into public APIs.
FossilOrigin-Name: 35d1d63c7d60119b64341c561294890812837d5432d1d7bed3ed88d6212fbfa0
2022-12-10 10:24:46 +00:00
stephan 3bcb5ce472 Imply configure flag --disable-shared when --with-wasi-sdk is active because libtool is running gcc for linking the shared lib, which cannot work in a wasi build.
FossilOrigin-Name: 9253ba43c0c7657ecf4c3d32df4084577e7188e6151b883a5fa92a43aa4c3e30
2022-12-10 07:02:46 +00:00
stephan 146961889e Add --with-wasi-sdk=DIR flag to configure.ac. This mode compiles but fails to link the DLL because libtool is apparently hard-coded to gcc for the DLL.
FossilOrigin-Name: 39c9f441519d3b3c365c52293a73cdf4af86de639e0009170c27725aad6653f6
2022-12-10 06:44:48 +00:00
stephan f0962c9f49 Run configure.ac through autoupdate, as required by autoconf for rebuilding configure. These (many) changes were made entirely by the autotools, except that some whitespace-only changes were reverted by hand to reduce noise, and are a preliminary step to adding a new flag to configure.ac.
FossilOrigin-Name: 2be2d5d428acd538e44d7b9345b64470131bef40f67ba3190f5cd42f9949f355
2022-12-10 05:52:02 +00:00
stephan 9be7a363cc Merge trunk into wasi-patches branch.
FossilOrigin-Name: 13dade955d46020ab408734123261e9fadb8bccbaf67003b111112af4789d8c0
2022-12-10 05:00:16 +00:00
drh 93f41e22d4 Remove the SQLITE_PREPARE_SAFEOPT flag. The name is obsolete and it is at the
wrong level.  Instead use the SF_UpdateFrom flags on the Select object.

FossilOrigin-Name: 78723a9a7e72b42d28fc5645661da17f20cedcf864819b861800ad9340007be1
2022-12-09 18:26:15 +00:00
drh e974e73f69 Do not allow OOM faults in EQP messages associated with automatic indexes
on co-routines to go unreported.

FossilOrigin-Name: d125d5afdf1b0a1c64fc64f180898099af07b8290ea9da49419df75d8b455f71
2022-12-09 17:33:21 +00:00
stephan 8ccef8f27f Export sqlite3_result_subtype() and sqlite3_value_dup/free/subtype() to WASM.
FossilOrigin-Name: 4600a7bbdc4cbe14591d48ea19fe5f7de3a0c10a14cdd97fd51e263a13163d10
2022-12-09 15:26:58 +00:00
stephan 706780fbd9 Expose sqlite3_value_frombind/nochange/numeric_type() to WASM. Add a flag to sqlite3_value_to_js() to cause it to return undefined, instead of throwing, if no conversion can be found.
FossilOrigin-Name: de8fc4bf34f80f320012a0e506ed8e3e24806daf67845d5dabb00b916108f6ef
2022-12-09 15:12:07 +00:00
stephan 99f30f1cd4 Refactor the sqlite3_value-to-JS conversion from an internal detail to sqlite3.capi.sqlite3_value_to_js() for use with routines like sqlite3_module::xFilter().
FossilOrigin-Name: f6dbf280f99809a80c99337e4c22a86dea7a35ae41ae9a69144c4502385a0a1f
2022-12-09 14:46:24 +00:00
drh edfbde52fd Fix typo in the SQLITE_DBCONFIG_LOOKASIDE documentation.
FossilOrigin-Name: c6e7582aea4ebcc4563afb4367fded1e8a74f6ef522a569460023c340ca24b30
2022-12-09 13:49:44 +00:00
stephan 3ec44736b5 Remove some unused sqlite3_status() codes from the JS API. Add custom JS wrappers for sqlite3_create_collation/_v2() which accept JS functions (plus tests). Expand the argument options for sqlite3_wasm_db_error() to enable it to translate exception objects to C-level errors.
FossilOrigin-Name: 073a2f1eb006230ae0995a5ea6c789407bcaa819ec15b5064c66d8973ed4671a
2022-12-09 12:12:49 +00:00
stephan 81a3683174 Rename the oft-used, verbose sqlite3.wasm.get/setMemValue() and get/setPtrValue() to peek/poke() and peek/pokePtr(). The old names are retained as aliases just in case any client code actually uses them, but they are now deprecated.
FossilOrigin-Name: ad0a8139b0b025f8e9d2eca0c303557ef10fdfab8c8b65afb08c510a804073d5
2022-12-09 09:23:27 +00:00
stephan 5a83f52f48 Micro-optimization in the oft-activated JS-to-WASM arguments conversion step.
FossilOrigin-Name: ee47e9b83ca668b37dc1d8e519048a635693cf33d9967a2d81ff0824b7eea4ba
2022-12-09 08:44:22 +00:00
stephan a3451dd990 Correct a test bug which broke the previous checkin's wasm tests in higher optimization levels. Test bug - it should not have worked in -O0 mode.
FossilOrigin-Name: f07141b7500c36169c2c96e045acf37ec97a0d75c0a5f4d0ebed4dc1d2094ccf
2022-12-09 07:17:54 +00:00
stephan 464c80d4c2 Export sqlite3_status() and sqlite3_stmt_status() to wasm. Expand the arg/return semantics of wasm.get/setPtrValue() and get/setMemValue() to simplify handling of multiple pointers.
FossilOrigin-Name: e144fd5c88fa4151429a2fef3daf389588402e158267f0afa0dfd142527d76b9
2022-12-09 05:47:42 +00:00
stephan 75f54dba50 JS API doc updates.
FossilOrigin-Name: 4f80fd3b8d4c85894664093d8310d6f5299faac4eb879edc608b3ffcd8558e9a
2022-12-09 02:23:15 +00:00
stephan 735dd03377 Expose sqlite3_table_column_metadata() to wasm.
FossilOrigin-Name: c31eb509e5cb1025de058132ee9a45d70c84ee47a6abe18811a65ce339f062a0
2022-12-09 02:12:43 +00:00
stephan 6ca03e14e9 Expose sqlite3_db_status() and sqlite3_db_config() to wasm, noting that the latter requires several internal wrappers to account for the various varidic forms (C varargs cannot be bound to wasm).
FossilOrigin-Name: d5753668915c1db204fa80153614653243081ffaddea22f26ad59bb1836948b9
2022-12-09 01:49:17 +00:00
stephan d27c3e4e7e Reorganization and renaming in the new VFS/vtab JS pieces.
FossilOrigin-Name: 1c2dda177a11fcc5b66e5554507c23ba4b9948a710b3bccfb26963b9851d40a4
2022-12-09 00:50:39 +00:00
dan 2797ac028c Support ".scanstats est" to enable including planner estimates in query profiles.
FossilOrigin-Name: 0fe71287c953bd813a34ba383f5debd4d1fc8bf3c74e1e27adacec0d6e207ded
2022-12-08 21:05:33 +00:00
stephan 1a9a29815a Further docs and minor cleanups in the JS virtual table helper.
FossilOrigin-Name: c31e7488ac1a6b957782b72bd026b1f0590637b631e44a1fdf1dedeb5c587819
2022-12-08 18:18:37 +00:00
stephan 500fa7d518 Ease-of-use/legibility improvements in the virtual table JS helpers.
FossilOrigin-Name: 54c7ad7e08bdb87579398ade366605bfa2e2538a94aabcc6e4cda8e173649760
2022-12-08 17:07:27 +00:00
drh a60d61bf14 Improved comments on the new co-routine logic. Fix a C++-ism in the code.
Update test cases to accommodate the more aggressive use of co-routines.

FossilOrigin-Name: 9243e850ae656d16adc8f0e5c4dcf3dcf476312cee192c39c38685fc437ccbbd
2022-12-08 16:47:16 +00:00
stephan 4bfacc4b40 Initial support for virtual tables implemented in JavaScript.
FossilOrigin-Name: a1454744c770a30a32a6d7b7fc59ef7be48cf67348b238540592850d7c2c7757
2022-12-08 15:00:53 +00:00
drh 05cb9d881e Do not use a co-routine on a subquery that is participating in a self-join.
FossilOrigin-Name: ea4306a03cd8dd706100dac3f11653068a25c50c5d06c34ad08534aec0a580bc
2022-12-08 13:56:06 +00:00
stephan 56f7813141 Describe the SQLITE_DBCONFIG_RESET_DATABASE flag's policy regarding virtual tables, per /chat discussion.
FossilOrigin-Name: 53dcb81b029e4422bdc0b5cf694183854b997195427d437b6154ed4dad7ad0ba
2022-12-08 12:51:11 +00:00
stephan 16bf97b507 More work on the JS vtable tests.
FossilOrigin-Name: 51e3c3b569dce2097063d39dc484f44b2f98cad0a902ef66765ca4cdc3e06f47
2022-12-08 10:06:07 +00:00
stephan d061f14466 Got JS non-eponymous vtable working thanks to a hint from Dan.
FossilOrigin-Name: 750719b4981df62fa2ff3665e6f559ec760609ad2493495628295ad9ae6024fc
2022-12-08 09:06:20 +00:00
stephan ab8b22a03d Remove some dead JS code and tweak some docs.
FossilOrigin-Name: 0ee495452c014680697aa9035c245024df127a52d1820ab0e02580a015d96ecb
2022-12-08 04:19:38 +00:00
drh 619ff3f505 Merge the latest fixes and enhancements from trunk into the
coroutines-exp2 branch.

FossilOrigin-Name: 1c5f41986f5af181bf389675361c9f176e9195e847319f07ebd5c87992ded38b
2022-12-07 21:19:37 +00:00
dan 7f4b066eb2 Reduce the overhead of SQLITE_ENABLE_STMT_SCANSTATUS some.
FossilOrigin-Name: 212927e97e7be7d237de08359dce0dfb9211ac406b32009a6e15afd79c006475
2022-12-07 20:09:54 +00:00
drh 209dbab997 Fix harmless compiler warning in the dynamic continuation prompt of the CLI.
FossilOrigin-Name: 68947b0a1147365a29e335d5e4dc55fe5d9afe3562a0709b111067d6017c5a42
2022-12-07 19:51:48 +00:00
larrybr 7474de702d Omit CLI use of pragma_table_xinfo when it is not defined in the build.
FossilOrigin-Name: 976c23520fa64b39aa54047f2fd12445e65940206768ce51c0d690d0d0d570ab
2022-12-07 19:29:13 +00:00
dan f2cc3387f7 Have sqlite3_stmt_scanstatus() report cycle, loop and row counts separately for creating an automatic index and using that automatic index.
FossilOrigin-Name: 3bc9df82ea5b0fb085c56a326a65e19c9baf98d48d8fa6344c0d7004747594ba
2022-12-07 17:29:17 +00:00
drh 7c66faa634 Streamline and improve testing of the locking in the memdb VFS.
Follow-on to [15f0be8a640e7bfa].

FossilOrigin-Name: d71a08375aeb525c10037c373b8eeb7e29f7dfaf7c4bfc02f4d99616c5940405
2022-12-07 16:58:04 +00:00
stephan 1eb1b59b89 Work on an alternate (slightly simpler) approach to binding JS vtabs. Non-eponymous vtabs are not working, for reasons as yet unknown.
FossilOrigin-Name: 6a0fefb93bcccd950df211cf5c2f49660c7b92115dd01b2b508a4ab9e3ab3d23
2022-12-07 07:22:34 +00:00
stephan 30da58c5d6 Add addOnDispose() method to Jaccwabyt and code-adjacent minor internal cleanups.
FossilOrigin-Name: 6a2723fe3f28dd94328d901e64e1e9ee9a1b2e9eeaed6c54038a5b83c914db78
2022-12-07 03:42:39 +00:00
drh b54f71e205 Fix a (harmless) off-by-one error in code generation that comes up when
doing a DISTINCT query against a virtual table with an OR term in the
WHERE clause and where the ORDER BY clause has 64 or more references to
the result set.  [forum:/forumpost/dfe8084751|Forum post dfe8084751].

FossilOrigin-Name: 04af7ef77043702f93cbff23548610759786893bd3d4d6fc08181e1e249c6663
2022-12-07 00:14:25 +00:00
larrybr 7726629ab2 Enhance CLI .sha3sum to warn of text fields that are not to-blob-to-text reversible.
FossilOrigin-Name: bbde0f36d03cdbbc749427fe7d2dafd5c5031c9e655ebd772857b521f53eb18f
2022-12-06 19:32:07 +00:00
larrybr 1751f875bc Merge from trunk prepratory to branch end.
FossilOrigin-Name: 1d01f8483af7b6a4e93a49fd8bbb6a2b9300e969b574efeb96d56081a33055c4
2022-12-06 19:25:07 +00:00
larrybr 3c01af6ffc Avoid several -Wall warnings in textfixture build.
FossilOrigin-Name: 03ae8680e430c6d2c39ca26fa2cb6eed46e3c3e52b53f7a8bf548560446c6d3d
2022-12-06 19:20:49 +00:00
larrybr 09d953706c For CLI .sha3sum, emit warning to stderr for any invalidly encoded text fields.
FossilOrigin-Name: 8e833ecc81367658e81acd1d3a0242ab954a62d6f719af56f1d185656d0b73e6
2022-12-06 18:48:37 +00:00
dan 2adb309ead Have sqlite3_stmt_scanstatus_v2() return an NCYCLE value for all loops, not just virtual tables ones. The value returned is the sum of the NCYCLE counts for the various opcodes that move or read data from the table or index cursor associated with the loop.
FossilOrigin-Name: 9499b2f51e8174c6b8a67840c92ba23b7dd1dc8dc2b91fca0c5dc07b71662149
2022-12-06 18:48:06 +00:00
larrybr 29226fc023 Clear a few more -Wall warnings and simplify dynaprompt feature keep/omit macros.
FossilOrigin-Name: 540e895d877fab1ea138786e56923a202018f68c78199a89adfc296c75735b30
2022-12-06 17:59:05 +00:00
drh ad9ff1d5f2 Simplified experimental changes to promote the use of co-routines. Less
cruft than the coroutines-exp1 branch, but still does not work.  Checked in
as a work-in-progress.

FossilOrigin-Name: e2318a30bf6ad2aeb4c4cb29661bc24ff78eb51bf07decc4b8da1ecac0015ef0
2022-12-06 15:24:05 +00:00
drh d84f185f97 Fix compiler warnings in the new dynamic continuation prompt logic of
the CLI.

FossilOrigin-Name: 0d80500d358fa1c9b5867c2c8250d278ba813bf2ad81bb0bc3f820a71489b374
2022-12-06 15:11:13 +00:00
drh b1026aeda9 In the unix backend, when implementing the defenses against small file
descriptors, delete a file just created if it was opened with
O_EXCL|O_CREAT so that it can be created again the next time through the
loop.  Fix for the problem described by
[forum:/forumpost/699af709ab3a8ccf|forum post 699af709ab3a8ccf].

FossilOrigin-Name: c0cfe0582add87981826d124a0763482f51fae4b105b5a970dd56919f1d04d60
2022-12-06 13:12:33 +00:00
dan 109f173b6f Support an SQLITE_SCANSTAT_NCYCLE statistic for "CO-ROUTINE" elements.
FossilOrigin-Name: d3f6a207fd1b3f53688fa1ff35eb71e1c5bcc4115c0c35bb187d22ccf7100b71
2022-12-06 11:41:05 +00:00
stephan 241cde98b8 JS vtables: add infrastructure related to accessing and modifying sqlite3_index_info.
FossilOrigin-Name: 0d77c348039926c24e0fb50a7dc7e4b62895cd201c021f8e29832917e4b8b09f
2022-12-06 11:21:46 +00:00
stephan d254db53e5 Remove deprecated symbol sqlite3.opfs.OpfsDb, which was renamed to sqlite3.oo1.OpfsDb on 2022-11-29.
FossilOrigin-Name: 0ce51bed65d5e430364f74bf959fb76c42ac5eec0769490231d8c8110a1f388c
2022-12-06 09:49:04 +00:00
stephan 75435f8b2d Rename wasm.cstringToJs() to wasm.cstrToJs() for consistency with other wasm.cstr... APIs.
FossilOrigin-Name: cbf483ea0ba3e6dc08ad7ed654380f818544b4c3cedfdb8aa848a83298268ceb
2022-12-06 08:46:39 +00:00
stephan 671386c637 Add wasm.cArgvToJs() to support sqlite3_module::xConnect().
FossilOrigin-Name: c3ebdccf94d5e63c229bf91056c08052d78732e663334070ef3b0ef6fb4bfb8f
2022-12-06 08:39:17 +00:00
stephan b849832a79 Minor internal JS code/docs cleanups.
FossilOrigin-Name: 21331bdd36a91b07a687ffadce392dcf2ccd0fd824b35d9dd027d4289a40fc96
2022-12-06 08:21:23 +00:00
stephan f2bbef3951 Merge trunk into wasm-vtab branch.
FossilOrigin-Name: d106edb956bbe22bddc2c6e5bcdf4fb8f797794c725c2e359e3fa4b31d881843
2022-12-06 06:21:02 +00:00
stephan 2a665edbd4 Minor test tweaks.
FossilOrigin-Name: f902f3b2c79d6c699ead1efeb1426e1e0f4ac709afdff88be1de62f34f3d5ccc
2022-12-06 06:16:11 +00:00
stephan 6b271abc98 Add a demonstration sqlite3_vtab/module implemented in JS, based on ext/misc/templatevtab.c. Add oo1.selectArrays() and selectObjects().
FossilOrigin-Name: 60482c97e02bc4cafefef281be0cf0bc8c5c53232162829c137f3f7a80cdc534
2022-12-06 06:09:03 +00:00
larrybr f9d40801c3 Add optional feature: A CLI continuation prompt which reflects open lexemes and parens, similarly to PG shell.
FossilOrigin-Name: f41f18b1c3c950565ee3c237aebb51cfc3241813c6425813a8217e07aa0153a6
2022-12-06 05:31:20 +00:00
larrybr 9ab1598e52 Add optional feature: A CLI continuation prompt which reflects open lexemes and parens, similarly to PG shell.
FossilOrigin-Name: dac2ddc287db7a68d0cd49b785060f62290868fbb1aa2ee09e54d3b1acfbf55f
2022-12-06 05:09:51 +00:00
dan d2db31c971 Enhance the sqlite3_stmt_scanstatus() API and add sqlite3_stmt_scanstatus_v2(). For creation of enhanced query performance reports.
FossilOrigin-Name: 4893b4e3eafc7c9c22b24717f90a585862203f987cf108b079ce6e946093e675
2022-12-05 19:16:23 +00:00
dan 07c8e08889 Update comments in sqlite.h.in to account for sqlite3_stmt_scanstatus_v2().
FossilOrigin-Name: 009462f2344b1f468cf9440343a47fec68d783a2bfb4fa6168bb227ec910b918
2022-12-05 18:52:12 +00:00
dan 77e171e8fa Merge latest trunk changes.
FossilOrigin-Name: 1a72777b1279f74f212fb2f675a4594a238e5d28f048879d7f5ad5287673c3c4
2022-12-05 18:26:37 +00:00
dan a3d0c158a0 Add loops and rows counters to "USE TEMP B-TREE FOR ORDER BY" records. Also fix the sqliteHwtime() function so that it returns a 64-bit value.
FossilOrigin-Name: 41a0e05e8c0fca3b803fe4bd017a157c172b2ca518356a2a4d4ed4f12d01a1e3
2022-12-05 18:19:56 +00:00
stephan 2582d418d3 Remove two features of jaccwabyt which were fundamentally flawed, along with approx. 250 lines of unit tests which heavily relied on them. Thankfully, none of the sqlite3.js-level code used those bits.
FossilOrigin-Name: a190abc307847174f36421eaa3f47ef349c6f84a2bb35857fa64f64bbe722708
2022-12-05 15:05:46 +00:00
stephan cfb66014bc Jaccwabyt (JS) doc updates.
FossilOrigin-Name: a329a809b5da135a9c251e4d5f637d45d01d0248110ac05f2ad8f01d9df38c64
2022-12-05 14:32:35 +00:00
dan 49d402684b Add test case that should have been part of previous commit.
FossilOrigin-Name: dc7dd2d3e50e7cc474b22f1b5b219da32bcd7aa1ba56864d1dbcf0d3a6fa06f2
2022-12-05 14:20:54 +00:00
stephan e1d25177c2 Expose sqlite3_get/set_auxdata() to wasm. Minor test app CSS tweaks.
FossilOrigin-Name: 44659ad32a9fe6363badfc5dbb0bd51d6fb2ee1c8aa47b71e9cf3dbd631fde9e
2022-12-05 14:13:55 +00:00
dan 27ef91b551 Fix a problem in the memdb vfs xLock() function allowing clients to upgrade to EXCLUSIVE locks when other connections are holding SHARED.
FossilOrigin-Name: 15f0be8a640e7bfa4130edd4650a745337bd96083b119a1553f9abf9ff066806
2022-12-05 14:12:14 +00:00
stephan e0818715b7 Export collation-related APIs and strncmp()/strnicmp() to wasm.
FossilOrigin-Name: c3c56d9b944fd0d806d8dad9f0c7be3d7a5441765310908872cc525d82ab6a33
2022-12-05 13:33:42 +00:00
stephan 9a49a97487 Export sqlite3_vtab_collation() to wasm. Rename 'flexible-string' JS argument adapter to 'string:flexible' for consistency.
FossilOrigin-Name: 15f8042fddaeabab43dd187c463d3ccc56758cbf19bf2ca4837d9087a4850c1a
2022-12-05 13:12:48 +00:00
stephan 8c696375b5 Remove SQLITE_EXPERIMENTAL tag from sqlite3_vtab_collation() and fix a related doc typo.
FossilOrigin-Name: 35d670b3593a46e13ded541ef477fa35dac6bcb9c31e6cf4b90bd7fa550a8ee2
2022-12-05 13:07:06 +00:00
stephan cf8f0d2046 Rename 'static-string' argument adapter to 'string:static'. Replace JS unit tests which were lost via editing a generated copy of tester1.js instead of the original tester1.c-pp.js input file.
FossilOrigin-Name: 9d81d51d5a255b42f8416da850c992a9e4c8eebc940e0702a9262cfcaa6d7b2f
2022-12-05 11:54:13 +00:00
stephan 0adef09374 Export sqlite3_bind/value/result_pointer() to wasm. Add 'static-string' argument converter to support the lifetime requirements of bind/result_pointer()'s string argument. Correct an endless loop in wasm.cstrlen() when passed a non-C-string argument.
FossilOrigin-Name: a94552434a657376d5ce1831de05c1b15fb153020848cd825fb0df413c3baa70
2022-12-05 11:30:39 +00:00
drh fe8ecfd696 Remove obsolete "experimental" comments on the virtual-table interface
documentation in sqlite.h.in.

FossilOrigin-Name: 28fc32abdcb1c6af19f0926a3a7317e15930a0ba89645d5ffbf6ce99d8463273
2022-12-05 10:50:06 +00:00
stephan 08fc64ea04 More work on the JS side of the virtual table APIs.
FossilOrigin-Name: cb9881ec001b0e2faf047e57acfd1722d2b546255a54e0f850f568edfe2df1cd
2022-12-05 07:51:25 +00:00
stephan 864c3c029b Remove some dead code. Improve some error checks and comments.
FossilOrigin-Name: 6712fbe46a97867cea309f78a274edbb6bd166a505b41e18a580306da0e063db
2022-12-05 05:45:00 +00:00
stephan e177447972 Initial infrastructure for adding virtual table/table-valued function support to WASM.
FossilOrigin-Name: c202d7a0398b9aabc2babba5c4c91a313f32bbf37549d419775642bb4aa3936a
2022-12-05 05:30:03 +00:00
drh 95ade3bdb9 Improved query planner cost estimates. Fix for ticket [e8b674241947eb3b].
FossilOrigin-Name: 1b779afa3ed2f35a110e460fc6ed13cba744db85b9924149ab028b100d1e1e12
2022-12-05 02:52:37 +00:00
drh b542298376 Use the smaller estimated row size for searching IPK tables, but use the
original larger row size estimate for scanning, since the leaves can have large
rows.

FossilOrigin-Name: df3818997b822743ac407dde45c5fd75845ca40f461e31350d86963dffec6cd6
2022-12-05 02:42:30 +00:00
larrybr 8af6d712da Fix safe mode authorizer callback to reject disallowed UDFs. Reported at [forum:/forumpost/07beac8056151b2f|Forum post 07beac8056151b2f].
FossilOrigin-Name: cefc032473ac5ad244c0b6402c541b2f76c0c65a041bda03bfbe7c0e2c11fac2
2022-12-04 23:20:38 +00:00
stephan a81c5b3a88 Export sqlite3_result_zeroblob/zeroblob64() to wasm.
FossilOrigin-Name: a60e56627fc0ef8831429941d429ee02c6ee51ce5a2c1af581dc5bc5a00d911e
2022-12-04 08:16:33 +00:00
dan ad23a47acd Enhance SQLITE_SCANSTAT_NCYCLE so that it reports on virtual tables.
FossilOrigin-Name: 622d8eb3724bee617b55d6fb71f1a2d683db6858065adced6bf3ce9525bcd6b5
2022-12-03 21:24:26 +00:00
drh 1d8f4e6ce8 Increase the nominal row size for IPK index lookups slightly, for better
balance.

FossilOrigin-Name: 1a61c500add4a2bfe80c0c691d559cfca166dc5f8262651a58da7ec16a51d430
2022-12-03 19:04:09 +00:00
drh 878b44c128 Add TOTAL rows to the output generated by tool/vdbe_profile.tcl.
FossilOrigin-Name: 9800586393c9d3b82459ef657620d245a7985ef5fa389b8a9ea633d6a29c7299
2022-12-03 18:21:30 +00:00
dan f6f01f15dd Add CYCLES scanstat measurement to "USE TEMP B-TREE FOR ORDER BY" lines.
FossilOrigin-Name: 365011ae8b7e3fcaa2c4ea9601231a3ef2223e60d7a53ec33013109dca22ad58
2022-12-03 18:16:25 +00:00
drh fcf4243c36 Add a test case to show that ticket [e8b674241947eb3b] has been fixed.
FossilOrigin-Name: e20de6d450c2b4d5bde737f625de16ff53262c22ce7aa6917b64f1665170d33f
2022-12-03 17:23:29 +00:00
drh e4fa4794be Further improvements to the estimated cost of sorting. Take into account
the number of columns to be sorted.

FossilOrigin-Name: f3290cf83b7c02d17d85d8942954f052b486c370cd5ec732969da9061dc1d19a
2022-12-03 17:09:15 +00:00
larrybr 30cbcc0004 Cause CLI .sha3sum to warn of text fields that do not survive CAST(CAST(t as BLOB) AS TEXT) due to invalid UTF encoding.
FossilOrigin-Name: 123f2a0785790addf9c60a0fd09077dda9cb84d33a2594901a2b22bb555be491
2022-12-03 16:09:32 +00:00
stephan 9baafdabf7 Correct a memory leak in tester1.js.
FossilOrigin-Name: e42b052dd754e577b8429e2430821c5f88096170ffa58b4e5e3cb7ee19b6c37d
2022-12-03 15:41:29 +00:00
stephan cbcc08ddc4 sqlite3.wasm.allocFromTypedArray() now optionally accepts an ArrayBuffer as its argument.
FossilOrigin-Name: 75a1a796f86d289c7275666fab19013934775dcccaed44a1a61d1749a6cb99c9
2022-12-03 14:58:45 +00:00
stephan 09c27a59db Rename wasm.xWrap.resultAdapter() X:free entries to X:dealloc for consistency with wasm.dealloc(). Add an undocumented feature to replace wasm.alloc/dealloc/realloc() with the C-standard allocators (after an allocator misuse led down a several-hour rabbit hole trying to discover a mis-free() violation). Related test updates.
FossilOrigin-Name: d9807656f8a7c2a893d3f68ee5592f44826b8e999ae66f7d9000674b5c1b0207
2022-12-03 13:10:58 +00:00
stephan 54ac04c831 Doc typo fix for SQLITE_MAX_ALLOCATION_SIZE in malloc.c. No code changes.
FossilOrigin-Name: ed1ed21221b048ac5a5275cdfc4d9b2a406acdc7d4b648c3b61bcc822d88d955
2022-12-03 13:05:33 +00:00
stephan fed255d794 Move kvvfs xRead()/xWrite() buffer from function-local stack memory to heap memory to accommodate changes in emsdk 3.1.27.
FossilOrigin-Name: f1da32410ca7b808b3bef5f5a59766e7281e9e6ea343c8b979599bf1fc1060f5
2022-12-03 11:51:29 +00:00
stephan 85ef4e4344 JavaScript: add sqlite3.wasm.realloc(), sqlite3.capi.SQLITE_MAX_ALLOCATION_SIZE, and related tests.
FossilOrigin-Name: eeb84ba5de1152ef0f42105b8b285fdee9f5ad58281e60a4e0c8b1d6de1dead8
2022-12-03 11:16:55 +00:00
stephan 2564ca7d15 wasm build: rename the path to the wasm docs checkout, for clarity and consistency.
FossilOrigin-Name: b820db32365b2ca8e2397fd6ea85883e4555ffd82948e248a0f98415b7328349
2022-12-03 03:06:16 +00:00
stephan 1369d6be02 wasm snapshot and doc-related build automation tweaks.
FossilOrigin-Name: 13eb1abd06f55fb88fc7f7be6149fd94b12057d9c38cc6b97bec3940e7e01f04
2022-12-03 02:42:21 +00:00
stephan a37fed0f62 Rework the oo1.DB's distinct-per-VFS post-open() step to accept either a batch of SQL or a callback function. Increase OPFS's busy timeout to 10s.
FossilOrigin-Name: 9feefe253ac487cb52be6bdf91bdd305963266716baa08f2bf9505954ee76321
2022-12-03 01:59:03 +00:00
drh 1edd0a089c Tuning the query planner by adjusting the weights that predict the relative
performance of sorting and index lookup.

FossilOrigin-Name: 9f2806da4d88beceac2e81e05421f00481dd3dd100b096cd2ae6c828adb42ca7
2022-12-03 00:52:21 +00:00
dan 231ff4b027 Enhance the sqlite3_stmt_scanstatus() API and add sqlite3_stmt_scanstatus_v2(). For creation of easier to read query performance reports.
FossilOrigin-Name: 55800833645739efeddcacef464c623931cb6aeb43f4219b4e4faf473c25c8bb
2022-12-02 20:32:22 +00:00
stephan bb4e4a4840 Minor internal tweaks to the OPFS VFS. Resolve a missing result code which lead to a null deref in xFileSize().
FossilOrigin-Name: 57dd593ef0efa17dfb3a9f4eac36d5b8b879e271de817d8cd94a8c8b56d31870
2022-12-02 18:56:37 +00:00
stephan 95bc4d67bb OPFS VFS: translate createSyncAccessHandle() exceptions which appear to be locking violations to SQLITE_BUSY. This seems to improve concurrency considerably even with a reduced retry count of 5 (was 6).
FossilOrigin-Name: 0d36021d107d3afca190ad61c3380536ad0cc2d493d345d48f9f9c1191741128
2022-12-02 18:06:26 +00:00
drh d2603adf46 For the sqlite3_bind and sqlite3_result interfaces for UTF16 strings, round
the number of bytes down to the next even number, to avoid creating a UTF16
string that is an odd number of bytes.
[forum:/forumpost/411199488d065f83|Forum post 411199488d065f83].

FossilOrigin-Name: b57e3c3db00a6bc6db20c82530479f9eba7e37b731f0da6fe81682e84c7ac916
2022-12-02 17:52:52 +00:00
drh 8b5d7fda59 Use sqlite3_result_int64() instead of sqlite3_result_int() when returning
potentially large values from the DBSTAT virtual table, to avoid integer
overflows in the result.
[forum:/forumpost/ada2ab044f|Forum post ada2ab044f].

FossilOrigin-Name: 5652154a8c93cf3b1ff6c2e55e94abbe995b0bb625f733461df20e006c2f13f8
2022-12-02 15:31:47 +00:00
stephan 80ea1f406a Minor touchups to the JS test index page and test server push rules.
FossilOrigin-Name: 0881f3e92364b3dc81443220451e853c0763efaf97653a6348bc39bd9bdb23ad
2022-12-02 11:35:21 +00:00
stephan 5ad4e89659 wasm builds: explicitly set a default stack size because emsdk 3.1.27 reduced it from 4MB to only 64kb, leading to memory corruption when kvvfs is used (it requires at least twice that for I/O).
FossilOrigin-Name: 758112460fb624198a144ed7d8d2324497fa72a2eee80010bd22b621e064b389
2022-12-02 10:43:14 +00:00
stephan d09414e623 wasm: after building snapshot zip file, emit instructions for pushing it to the test server.
FossilOrigin-Name: 9615c77919fee60d708d72729c741b373a89900aabc318e0ceb4154638cdf339
2022-12-02 09:23:38 +00:00
stephan 1078ee0959 Correct the problem which triggered the rollback in [7eec635562f6]: an incorrect default db page size (not a multiple of 512 bytes).
FossilOrigin-Name: e06e490c240aa56b616e6f0380b5d08abf06f35f9e683e5150c5ac464eae9e55
2022-12-02 08:51:22 +00:00
stephan 20f0128acc Roll back the SQLITE_DEFAULT_PAGE_SIZE part of [c260895faacb34] because kvvfs does not work at all with a page size of 8kb.
FossilOrigin-Name: 7eec635562f65592121d7ff0a31a7b4ff6cdfe38e657f326302ef851b3b37c3f
2022-12-02 08:38:04 +00:00
stephan f3409db2d1 sqlite3-wasm.c: code legibility and coding style tweaks. Increase SQLITE_DEFAULT_PAGE_SIZE from 4k to 8k, as that improves OPFS speedtest1 performance by roughly 12%.
FossilOrigin-Name: c260895faacb3458c557778630756d02a8520c0f1864bddcf86cdd27ef4a42bd
2022-12-02 08:29:03 +00:00
stephan cc70dfe679 Expand JS tests for db export/import and document reason it cannot currently work with kvvfs. Fix a minor JS build dependencies bug. Update page title with PASS/FAIL prefix for tester1.js to improve overview when launching multiple test tabs. Add ability of tester1 should-run-test predicates to report why a given test is disabled.
FossilOrigin-Name: 75f610d3a4cf3d972220f9abc27cdf5990451e3835ceb9cf66973934004dfc5c
2022-12-02 07:14:56 +00:00
stephan 31892c2820 Remove extraneous/unused sqlite3.oo1.version object. Add httpd makefile target.
FossilOrigin-Name: 8e4d30ac033a6d9019a7eeedfe788dc0120f565cef2ae8f09d2bf32eb94d8a33
2022-12-02 03:37:49 +00:00
stephan 919dbc846a sqlite3_js_create_file() now accepts an ArrayBuffer data source. Add test for OPFS-based export/re-import. The (sqlite3*) argument converter now optionally accepts sqlite3.oo1.DB instances.
FossilOrigin-Name: 14a84b67fb17e16a5691ea4bf7f374123ac73a361a5d3d0efca53788d2001e3a
2022-12-01 15:22:03 +00:00
stephan 9ec1a7a7a0 Reformulate and simplify some JS tests related to the previous checkin.
FossilOrigin-Name: 9ea2d3dcf798393a7fd231e199c0e2c6302949fe2a7f2573178fb0e50c78a2f4
2022-12-01 04:45:51 +00:00
stephan 2b2199570d Expand "sqlite3_vfs*" JS-to-WASM function argument conversions to accept VFS names (JS strings) and capi.sqlite3_vfs instances. Implement sqlite3_js_vfs_create_file() to facilitate creation of file-upload features which store the file in VFS-specific storage (where possible, e.g. "unix" and "opfs" VFSes). Correct an argument type check in the SQLite3Error and WasmAllocError constructors.
FossilOrigin-Name: e1009b16d351b23676ad7bffab0c91b373a92132eb855c9af61991b50cd237ed
2022-12-01 03:55:28 +00:00
drh 85ec20ac66 Add a testcase() macro to verify that the case of a NOT NULL error message
hitting the string length limit.

FossilOrigin-Name: 91f50964c10fb12d889bda7d597d8edf475d97d2d8b534b4400e0fed1d753c6a
2022-11-30 21:18:23 +00:00
drh 75fba2fd28 Fix an over-zealous assert() reported by Yong Heng.
FossilOrigin-Name: 6ee61f8cede4998f0c838d6506b058c6b09f34b3d7f30ed296100785c93f8d00
2022-11-30 20:22:31 +00:00
drh 2a757658a0 Attempt to rationalize the bits associated with ".wheretrace". Provide
a decoder key in sqliteInt.h for what each bit is intended to do.

FossilOrigin-Name: 8ec361695a107a94f2cf6a7fe509656a99d85d49bd7c74133c69903f059a2675
2022-11-30 19:11:31 +00:00
stephan ec192e0659 Rename some JS files from X.js to X.c-pp.js to keep the maintainer, and downstream build customizers, aware that those files contain constructs specific to the c-pp preprocessor and will not run as-is in JS.
FossilOrigin-Name: 2eade7c7b17a186735c72974c11a34798a08364861d0f307e897ba765c0a93c7
2022-11-30 18:21:01 +00:00
drh a42281a312 Always use nanosleep() (instead of usleep() or sleep) if the
_POSIX_C_SOURCE macro says it should be available.

FossilOrigin-Name: 6620c57b9d3eae7226a412318b43393196df069b5b90aae0cf1743fdd2d102dd
2022-11-30 13:44:31 +00:00
stephan 8ae9545577 Install sqlite3_malloc/sqlite3_free() as the JS-side WASM allocator (as opposed to replacing C-level's malloc()/free() with them). All tests work and this eliminates the potential for allocator discrepancies when using the (de)serialize APIs.
FossilOrigin-Name: 95c78f6b46e0d8efa4313061f47677479f48610b7a7261dc8d0fb1859aca2ad9
2022-11-30 11:50:16 +00:00
stephan 412237fba5 Merge trunk into wasi-patches branch.
FossilOrigin-Name: 64cf34a85505814b4c93e82855c25559c8096bb3ce38cdc5e0fb9218c6bcce32
2022-11-30 10:05:02 +00:00
stephan c5141c9efa Simplify how the OPFS VFS async proxy copies initial state sent to it from the synchronous side of the connection. Make the lock-wait time a multiple of the wait-loop timeout interval.
FossilOrigin-Name: eddafafffa634a42ceeed70aa3fc58be130527612157a4bf4ff9e65c7f6dc26c
2022-11-30 08:37:17 +00:00
stephan e7f5118bd3 OPFS VFS: remove an invalid TODO and fix a property name typo which caused xCheckReservedLock() to always report false.
FossilOrigin-Name: 3b037caa2fa07b6c44c485574e9e5dc71f4a8e82bc902c1321bb0b918b139c74
2022-11-30 07:48:35 +00:00
stephan 36a0163e40 Rename some OPFS JS files. Prevent JS bindings of sqlite3_uri_...() from performing JS-to-C-string argument conversion on their first argument, as doing so is specifically illegal.
FossilOrigin-Name: 79832808de2cbdba140ed9e0558f1502b51d131ab4315265315922cda7b748cb
2022-11-30 07:17:29 +00:00
stephan ad4f782815 Refactor a significant chunk of the OPFS sqlite3_vfs init code into sqlite3.VfsHelper, and internal-use-only API encapsulating code relevant to creating new VFSes in JS. Intended to assist in pending experimentation with an alternative OPFS VFS.
FossilOrigin-Name: e25d7b080a807e35b32cb885ea75b384130e5c6e936dfef783c5b45d9bfe77d8
2022-11-30 05:27:36 +00:00
stephan d0945f4638 Doc and logging text tweaks in the OPFS async proxy and test app.
FossilOrigin-Name: 7ce8608e221924d2c7067687eb6eef0f3cab181d5b4132e55a67d8514b6ce94b
2022-11-30 03:08:50 +00:00
stephan 72ee096775 sqlite3.oo1.OpfsDb: default to journal_mode=persist, as current tests show it to be marginally faster than truncate/delete in Chrome v109. Also increase default busy_timeout from 2 seconds to 3, admittedly on a whim.
FossilOrigin-Name: d0c8fa30a31c691bc1be5e98d806eeb1e23a8fc6cd54d87e5c1b720aa936e707
2022-11-29 18:28:40 +00:00
stephan 6e86d5569d OPFS speedtest1: hide a currently-broken/to-fix WASMFS-build link.
FossilOrigin-Name: cde95d382f8debcbca27c4aaf08470ffda35ab10d723a887786669367590ad3c
2022-11-29 18:00:01 +00:00
drh 7741f3457a Change the handling of hwtime.h to make it easier to compile performance
measurement builds that make use of hwtime.h.  This should not affect
productions builds.

FossilOrigin-Name: f64a224244743ab121371abd516fccbfc93c110e0952211764bd1b217e792c1b
2022-11-29 17:52:04 +00:00
dan 3c3896c9d6 Fix legacy build system main.mk to include basexx.c in testfixture builds.
FossilOrigin-Name: 8c9200b7e156206c4270ff60631c57d5898e2769f7f68294ce652bfec4cfa7c2
2022-11-29 11:28:45 +00:00
stephan a302e68476 Rename one instance of opfs.OpfsDb to oo1.OpfsDb, as per [0c5c51f4fb04].
FossilOrigin-Name: 75b04c9b302ec66749d8f072a49e61de9640b868bec99eadeb5b4bdef354c336
2022-11-29 06:56:09 +00:00
stephan ceedef888f Minor internal cleanups and docs in the OPFS sqlite3_vfs.
FossilOrigin-Name: 61799b05ff232c2ac349169c27bfe7f8d9277366093b0c9dd2739828993b3066
2022-11-29 06:09:32 +00:00
stephan 04184761de Internal restructuring of the OPFS sqlite3_vfs in order to facilitate certain experimentation and improve error reporting/hints if it cannot be activated. Deprecate the name sqlite3.opfs.OpfsDb, preferring sqlite3.oo1.OpfsDb for consistency with JsStorageDb and any future DB subclasses.
FossilOrigin-Name: 0c5c51f4fb04a4b90c50ec9704cfea9a3fb7d7d0ee55c1b0d4476129188217a6
2022-11-29 05:25:08 +00:00
stephan e6b0154138 Add an explicit warning about the current API-instability of the sqlite3.opfs namespace, which may need to be eliminated based on re-thinking of how the OPFS sqlite3_vfs is registered. Comment changes only - no code.
FossilOrigin-Name: 0cb2fd14179397051a25d066256a553fc198656d5668c7010c016f2b8f495bf4
2022-11-29 02:23:12 +00:00
drh d3fc2e63aa Check-in [8d5b76593d82b3a5] contained an error that was causing some obscure
error codes to be lost.  Fixed here.

FossilOrigin-Name: 46cdd3637d6a206ad2bcf8653cc6f2c7a886a16cc7685c45967938609941a755
2022-11-28 21:17:30 +00:00
larrybr 0b0070f1bd Get ext/misc/basexx.c into the testfixture.exe build for MSC.
FossilOrigin-Name: bb2c5d088e4784f6763eb0ea2a8542ad2129529f181fa92963c7231d7a68f25c
2022-11-28 21:09:49 +00:00
drh 5b046dadfc Fix harmless compiler warnings.
FossilOrigin-Name: 8d5b76593d82b3a57bc904ced33c24cd49369868b5dd2fe7f2c7b114f5aee2f6
2022-11-28 20:08:15 +00:00
drh 57ba01de4a Implementation of enhancement request [695a1a53dea6b240]: The query planner
has improved awareness of when indexes on expressions are covering and adjusts
their costs accordingly.

FossilOrigin-Name: e3474d79b27298e96e7686e5a6f9a8e736b8a6568a0a08fb1abe5bcca038ca63
2022-11-28 19:42:48 +00:00
dan 1b3d13e65e Add the SQLITE_FCNTL_RESET_CACHE verb. Use it to ensure that the page cache is purged before and after a the recovery extension is run.
FossilOrigin-Name: 6db0bc4bc0d272b610bef2aeeae43f539ed6e7cc0a9cc767d5af85ecb0019d5f
2022-11-28 18:41:41 +00:00
drh cbb9585805 Test cases added derived from the enhancment ticket.
FossilOrigin-Name: c022c0152ad61a4f56e36f4062609073e2273fbf6f826c20652159be229c2d46
2022-11-28 18:17:42 +00:00
drh 8ffa77f3a9 Merge trunk changes into covering-index-expr branch.
FossilOrigin-Name: 89d775ada39aed4dc532374ace02156d07dc06e8ae54a194608af0c5a582d20f
2022-11-28 16:11:10 +00:00
drh 84606c9737 Rework the covering index checking routine, whereIsCoveringIndex(), so that
it can return a "maybe" result for aggregate queries where we are not exactly
sure.  The index is scored as if it is covering, but the main table is
still opened.

FossilOrigin-Name: b8eec4214363192e6f3e12b3faa5810d8269a5fdaecab3ec09b02e5002cf798a
2022-11-28 15:23:53 +00:00
larrybr e862b5fe1d Add (optional) base64 and base85 UDF extensions.
FossilOrigin-Name: b44ab10c49bc2895483a9d40813be3798710ee713cc4bf04e449dce55a68452a
2022-11-28 14:51:50 +00:00
larrybr d8d68a5b8e Sync w/trunk, zap surplus space.
FossilOrigin-Name: b8345630a2a322234bda49ee4b996f6ba20e2b080621e229a2ec5e820892a663
2022-11-28 14:11:48 +00:00
dan abbfa7aa17 Ensure the RETURNING clause is honoured when a row of a temp table is updated by an ON CONFLICT clause.
FossilOrigin-Name: a2449bcc2c71d0f4c3289621fbf1cb97f0f407c9f7b5bf18245b7854a07c6cfa
2022-11-28 13:47:27 +00:00
larrybr c1ca183abf Conform CLI .trace arg handling to its help.
FossilOrigin-Name: 31546ea320e8daa020f8a9f491718ffc3dde0e32954018afed88e8494a6aff6c
2022-11-28 02:28:44 +00:00
stephan ca9309cc56 Document an OPFS API change in Chrome v108 which does not break our code but does change several formerly async methods to synchronous. No code changes.
FossilOrigin-Name: e9dd87e28a96eb6560943b9a2cb3cb0ff14698c96fe63865944dcfed73b67bd3
2022-11-27 00:57:56 +00:00
drh 9d34152bcf Better reporting of when the WHERE clause analysis thinks that an index
is covering.

FossilOrigin-Name: 17ebcf316b91042c823eea7bb6f1309325023cb5c70538cdb2ce932caee2ef05
2022-11-26 20:52:38 +00:00
drh 217e77d240 Show which AggInfo.aCol an Expr node represents in the tree trace debugging
logic.  Does not affect production builds.

FossilOrigin-Name: b8076f4dc55810855f668dfd4221bdb555cc78c61df200cf0cb87eb137e22850
2022-11-26 20:48:56 +00:00
drh 8beb7f56f5 Merge trunk fixes into the covering-indexed-expr branch.
FossilOrigin-Name: 9ac67ff968e874cf955e46e3993e3215c766feec3d5bdd38d77884eedd86b59e
2022-11-26 20:13:54 +00:00
drh 743c9ec4a5 Fix an incorrect assertion-fault in the "TREETRACE" debugging logic on
select.c.  The problem does not affect production builds.

FossilOrigin-Name: f11bb42292d94e7b7687bd2134f72afe1353182238cb4837fc0a6d78a021dd7e
2022-11-26 20:12:02 +00:00
stephan 4245667e35 OPFS VFS: add the opfs-unlock-asap=1 URI flag which tells the VFS to release implicit locks ASAP instead of during VFS idle time. This improves concurrency notably in the test app but brings a significant performance penalty in speedtest1 (roughly 4x slowdown). This is not the final word in OPFS concurrency, but gets us a step further.
FossilOrigin-Name: 9542f9ce9e023b489e2d93661f719fb0751c1e28f72fded9d3c2156d5777e7b1
2022-11-26 15:24:58 +00:00
drh 6e71186904 Experimental changes to help the query planner detect when an expression
index is coverting.  Works somewhat, but there are tests that fail.

FossilOrigin-Name: 968c189bcf29a9b25305251a58d09b7d52ab9dd08f5057dc3ab1f7ad1a5316a0
2022-11-26 14:19:47 +00:00
drh 902ea83925 Relax restriction (8) on the push-down optimization so that it only applies
if one or more columns uses a collating sequence other than BINARY.
See [forum:/forumpost/3824ced748baa808|forum post 3824ced748baa808] and
check-in [346a3b12b861ce7b].

FossilOrigin-Name: adbca3448e2099f0d6149a073978f230ed9a92a2f384779879ef89e672231bcf
2022-11-25 17:05:55 +00:00
drh 49d400b091 Fix #ifdefs so that restrictions (8) and (9) of the push-down optimization
are still enforced even if compiled with SQLITE_OMIT_WINDOWFUNC.  This
fixes a bug introduced by check-in [346a3b12b861ce7b].

FossilOrigin-Name: 09e1e42e0ff26f9a71cbd128169f060a66425828d637bf8f781490ca38d99103
2022-11-25 16:32:59 +00:00
drh ae5cfda656 Immprove the query planner such that it is able to make use of indexed
expressions within an aggregate query with GROUP BY.  This implements
enhancement request [99378177930f87bd].

FossilOrigin-Name: b9190d3da70c41717eb188474fd225ee43d0b46646e1b03de5967bd332553870
2022-11-25 16:10:48 +00:00
drh 23f61a4ba8 Add restriction (9) to the push-down optimization: If the subquery is
a compound then all arms of the compound must have the same affinity.
dbsqlfuzz 3a548de406a50e896c1bf7142692d35d339d697f.

FossilOrigin-Name: 1ad41840c5e0fa702ba2c0df77a3ea126bd695b910b5d1271fa3129c38c58f5f
2022-11-25 15:52:00 +00:00
drh bedbb1bfab Remove a redundant assert() statement.
FossilOrigin-Name: e3e1b453dc47884ddc9e51c4302fa2f4d40bca3d69ac7c13d8e2ae1adc81ac56
2022-11-25 13:26:51 +00:00
drh f5d0656f16 Improve the robustness of the updateAccumulator() routine against OOM.
FossilOrigin-Name: d2844f577b69fdc16a0a2568c0958fc3d8aff33e9a0ef80e0f58e92f01097432
2022-11-25 13:15:48 +00:00
drh 7960da0346 Improved comments. Add assert()s to verify that the AggInfo structure
is unchanged after registers have been assigned.

FossilOrigin-Name: 5200b84195ee1ccaa387f7032eae3d463724c48cb53ba0251bbc79e927dd9752
2022-11-25 13:08:20 +00:00
drh 8683c09281 Correctly deal with IF-NULL-ROW operators when dealing with
indexed expressions in aggregates.

FossilOrigin-Name: 939cb47025354e2df047de7654c0b06f791957cfe4e904abe8892207cea90215
2022-11-24 23:35:27 +00:00
larrybr 472de149ae Speed up base85() conversions and sync w/trunk.
FossilOrigin-Name: 17b823500a2ed135c1f40aa7f4d87ba5b2eab35c0abd9e0856041cf0f510cbee
2022-11-24 20:11:34 +00:00
drh e644f7c24b Take care not to try to add comments to a TK_AGG_COLUMN opcode that does
not have an associated Table object because it is a reference to an indexed
expression.

FossilOrigin-Name: 5fc23863e4a6388884ce061cd69546757b4c39d731bbb58a8121b92fd5144f7b
2022-11-24 19:02:40 +00:00
drh 594357f2e1 Change a NEVER() into an assert().
FossilOrigin-Name: a0fd44f4e5a0ec83465203a0009f307ca50223833575895a3ebf8a289515714f
2022-11-24 18:45:56 +00:00
dan eb84c81c53 Fix a test case in fts3expr4.test to account for different locales.
FossilOrigin-Name: a2b6883ac2ef878f525ee847b170beb9f9ab9d1fa67557101be2cdae1e7f7a57
2022-11-24 17:58:55 +00:00
stephan df5d06d03e More work on the OPFS concurrency testing app.
FossilOrigin-Name: c0458caca3508d5d252f9b5198bda4f51a5c1874540f014b17e409f2daab1706
2022-11-24 17:53:09 +00:00
stephan 6b86b33793 JS documentation cleanups. No code changes.
FossilOrigin-Name: 18e89a436daa18a8c972caf06b298da43c97a6ea3e2e5229dccb6920c27bfdb9
2022-11-24 16:03:49 +00:00
dan 647b0dd12d Update multiplex3.test to account for the fact that the multiplexor xDelete method may return an error even if it manages to delete the first chunk of a file.
FossilOrigin-Name: 1a7f3254735054ed8ca32d5ec7c8cde9195a64702638bdc50392007e396fead2
2022-11-24 15:32:00 +00:00
drh c25f5ea6e8 Add NEVER() and ALWAYS() macros on branches that are believed to be
unreachable.

FossilOrigin-Name: 3a901e88c87fc76c7fe42e47a976a5706830f0dbd6027605663e4d55f4f33590
2022-11-24 15:04:23 +00:00
drh b669bb5e2b New test cases. Fix the logic so that it works for GROUP BY aggregates
that do not require sorting.

FossilOrigin-Name: ef6ebe7922f56c1584a005deedc85ca1070b4fe5082ada8bbf8d06df54f1c9ef
2022-11-24 13:19:25 +00:00
larrybr 9b9017def4 Speed up base64 conversions, and add test with more data for the baseNN conversion to grind.
FossilOrigin-Name: 6c84ae4ba83713c751fddff8be41686bbcb525ac8135e1520436c62d0bc23d2c
2022-11-24 02:59:33 +00:00
stephan f46091d73f Add sqlite3.oo1.DB.prototype.checkRc() and tests for both that method and its class-level counterpart.
FossilOrigin-Name: f7eaa6ba2147bfd6dbdc2444d0f919d846aa7f9b68cccab17ef585ffdacf3d60
2022-11-24 02:35:03 +00:00
drh 4b1555a9fc Merge the latest trunk changes into the agg-with-indexed-expr branch to
simplify diffs.

FossilOrigin-Name: 38c3d3f1ed0fd2bb62aa8a7e5a27f2b247123e094e2fdb0a2475d788c3dfbc04
2022-11-24 01:41:44 +00:00
drh 9b80cb5f39 Add explanatory comment to the new optimization. And add a test case.
FossilOrigin-Name: e6c20f61de7d048eee65c8e74a3eb36760ab9747ebd1ab50e49642b777c10306
2022-11-24 01:40:20 +00:00
stephan f1ce4f40c7 Minor JS doc updates.
FossilOrigin-Name: 27efd63ad7fb3bf8d0d07f2c9f48652c8cacc4e697c229c8085120a8e6b50a39
2022-11-23 21:09:51 +00:00
stephan 875db41afc Add optional zSchema argument to sqlite3_js_db_export().
FossilOrigin-Name: 9c23644b1e5bf44bfb431a35fd1674c11ccb99e9eb0989f10175b0cb2a858eaa
2022-11-23 21:03:22 +00:00
stephan 056a71562f OPFS concurrency test: add a URL flag to enable/disable unlock-asap mode.
FossilOrigin-Name: 1c1bf22eadae2a5a7d4358e7cdd22641c2efb9296f42e7376749293b3a58b114
2022-11-23 20:49:08 +00:00
stephan e79cb67c35 Add an experimental OPFS VFS-specific URI flag, opfs-unlock-asap, which tells the VFS to release implicit locks ASAP. This permits higher concurrency but hurts performance considerably. This may or may not be obsoleted by other concurrency-related experimentation.
FossilOrigin-Name: d23c917013485ec2793125221f3936b05c39d6eca941629fb819b6b4aa714520
2022-11-23 19:03:22 +00:00
drh 2e30d95fb6 Aggregates with GROUP BY now make use of expressions on indexes. This code
works and gets the correct answer for the test case in the ticket.  Lots more
testing and documentation is needed, however.

FossilOrigin-Name: 8dcf9f2031c16f296d187fe876d4204c71fc96fec120984ff11b6d8b03d58a5f
2022-11-23 18:51:04 +00:00
drh 590f013a87 This attempt at modifying AggInfo to make use of indexed expressions does not
work.  It gets an incorrect answer for the test case shown in the ticket.

FossilOrigin-Name: 84c06023f4a1606664fdb9811312603b31f7c94a43d0e443ba7dde7fdba029e3
2022-11-23 17:56:00 +00:00
stephan ad1285c5c0 Initial infrastructure for adding a mode to the OPFS VFS which causes implicit locks to be released ASAP, which increases concurrency at the cost of performance.
FossilOrigin-Name: c5b7a9715a13b696ab3ee965aa0a310f59b65f07cecd72faa2e3504bfd8eb632
2022-11-23 16:39:07 +00:00
dan c32e16643d Update Makefile.in to include new target "sqlite3r.c". For generating "sqlite3r.c" and "sqlite3r.h", versions of the amalgamation that include the recover extension. To build the shell tool against these files, add -DSQLITE_HAVE_SQLITE3R.
FossilOrigin-Name: 5f135575b923cb59947667071c6af9ff14063c933cedf37d6c2a0a1b86c85032
2022-11-23 16:08:49 +00:00
stephan 91a81316b9 Remove a bit of over-cleverness which breaks loading of sqlite3.js in some main-thread cases.
FossilOrigin-Name: 220cc4c6399b772b4ece03305a41b437ef0654d586a8a1c3dc5e7606fd36d655
2022-11-23 15:52:52 +00:00
stephan 3eaaec945c OPFS concurrency tester: ensure that the work interval timer does not overlap with the work time.
FossilOrigin-Name: 8f2076553bc486ea6a17934695ecc02217461af2082d891697b62aab89bd1b43
2022-11-23 15:33:49 +00:00
drh c6138e970e Further foundation prep work prior to starting to flesh-out the
optimizeAggregateUseOfIndexedExpr() routine.

FossilOrigin-Name: 23145fe999ff74d787a3999baedd4ffe755c5f1f1275082ed0338ba637ecc56e
2022-11-23 14:13:39 +00:00
larrybr 809be56218 Add more baseNN tests, get oversize error trapping working, and sync w/trunk
FossilOrigin-Name: 03819e9368fd9f78f351147a1dc865743f9634893e43a9d1e3d7cbaf4c966069
2022-11-22 22:46:41 +00:00
drh 58c0652923 Merge enhancements and fixes from trunk.
FossilOrigin-Name: f8932e04d4d18eb9d71edda15aa08af2eb139ff14d77ca147ea6e9b173e0f5e0
2022-11-22 20:58:18 +00:00
drh b23f1572ab Add the stub function: optimizeAggregateUsingIndexedExpr(). The hope is that
we can fill this in with a routine that does useful optimizations.

FossilOrigin-Name: d85bb724fdd6fbad2b88ed7f60e4174e3f65182356f404d04620c5cf6b17f77e
2022-11-22 20:37:41 +00:00
mistachkin c620522175 Avoid naming collision between the sha1 and shathree extensions.
FossilOrigin-Name: 9ec923b5dc24d6082da8d42bc0ee8ab1c418912625c0c56de9627be2c818ef98
2022-11-22 20:04:00 +00:00
drh c7c5b8a508 Include the treetrace bitmap comment accidentally omitted from the previous
check-in.

FossilOrigin-Name: db07471c531766a8eec1d5b41c9fd0283b5e64ee13166dc3391f70a1e1946121
2022-11-22 19:56:54 +00:00
drh cbef156d93 Merge trunk fixes and enhancements into the agg-with-indexed-expr branch.
FossilOrigin-Name: 070634781a5eb41f96b001e48b064e3cd8c82314f576335eb1fcd43792179291
2022-11-22 19:51:16 +00:00
drh 5d7aef16bf Rename the SELECTTRACE macro to TREETRACE, so that is corresponds to the new
CLI command.  Renumber all of the bits in the bitmask used to enable
various kinds of tracing, and add a trace bitmap decoder in sqliteInt.h.
Changes to debugging logic only.  No (intentional) changes to production code.

FossilOrigin-Name: 8036445a36d9d982c1305935e7e37367bdf9e466b923eb6286b52524802e3ccd
2022-11-22 19:49:16 +00:00
larrybr 411929025b Sync w/trunk. Ready for merge to trunk.
FossilOrigin-Name: ff67460e1a3d21c9ca7cbd171fbc5e6cbdb3797de359887f851776b73b732fdf
2022-11-22 19:12:24 +00:00
larrybr bf82c95e08 Add test/basexx1.test to test is_base85(t), base85(x) and base64(x). Sync w/trunk.
FossilOrigin-Name: 4e4334547795f659b4a5dc5fdaf842535643750a5f1ce1af799c526931a473e4
2022-11-22 19:04:09 +00:00
drh c9b7d943ad Fix the error message in the CLI for "no such VFS".
[forum:/info/1217fdd6b3|Forum post 1217fdd6b3].

FossilOrigin-Name: 861af465fda8485dfff143dc45c659b884d826aaec2ebaa941566404d1fe427b
2022-11-22 18:24:07 +00:00
dan 63b74e02a7 Add Makefile.in targets for sqlite3r.c and sqlite3r.h, versions of the amalgamation that include the recover extension.
FossilOrigin-Name: 59a837cfc7f9f96509491c8fc45355d2e8892af25246955e22adec1cbf37327b
2022-11-22 16:12:53 +00:00
drh 3c8e438583 Since the memory registers used by the columns and functions of an
AggInfo object are sequential, it is not neecessary to remember each register
separately.  We can simply remember the first one and do the math when
others are needed.

FossilOrigin-Name: dc5bd34963b761c819c565653156d0befbf65cc2cc5dc4113b0ce952450f0352
2022-11-22 15:43:16 +00:00
larrybr 2a072506ca Fix last function pointer in sqlite3Apis init. Reported at https://sqlite.org/forum/info/eba0faa96d
FossilOrigin-Name: 6d5dc9eed54b7d883aa7dbe2eb5ff17d2a703eabdb7d548a27cccc1fe5addb4a
2022-11-22 15:41:32 +00:00
drh 89e5dfac01 Omit the unnecessary AggInfo.mnReg field.
FossilOrigin-Name: d79c58ef08b917bacc0f24d210d8eb23f659f955c219b4757af42eee8f17099b
2022-11-22 14:31:13 +00:00
drh 42b78237cd Factor out the allocation of registers for aggregates into a separate
subroutine.

FossilOrigin-Name: 4475799d5b41e93eabc32fac502ac2de240642b3b64739216e32e9af92ee191d
2022-11-22 14:10:22 +00:00
drh ff5e4ecced This is the first in what is anticipated to be a long sequence of incremental
changes aimed at improving aggregate query processing, and in particular
helping aggregate queries take better advantage of indexes on expression.
The end goal is to resolve ticket [99378177930f87bd], though it remains to
be seen whether or not I can get there with this approach.

FossilOrigin-Name: cba837eae93f6b842d4e78ef00661a4f09deb99c53f12b3e8f46763749602597
2022-11-22 13:33:58 +00:00
drh c59b7a8053 Convert an ALWAYS() in sqlite3DbSpanDup() into an assert(), for a performance
increase and size reduction.

FossilOrigin-Name: 21e80a29737c367babcc0cf2533eed61b5d0fcf3cc3c33ab88761899e394eaf3
2022-11-22 12:47:32 +00:00
drh da217c958e Split out the debugging code that dumps an AggInfo object into a separate
subroutine called printAggInfo() so that it can be invoked interactively
during debugging.  No changes to production code.

FossilOrigin-Name: dc33cfbedef4b444adeadea17f8183b7c4ce5b87432d0c712f986b34c7374ff1
2022-11-21 17:40:23 +00:00
drh 03af6d7157 Change the name of the Parse.pIdxExpr field to pIdxEpr so that the name is
distinct from all other identifiers in the code and thus less confusing.

FossilOrigin-Name: a2962d0187534f3496282b48efbf38dabcd226bf8972be925c8433c55a8853fe
2022-11-21 16:40:12 +00:00
stephan 6eb11f0a95 Merge trunk into wasi-patches branch to clean up the diff view.
FossilOrigin-Name: 95de6742d3d96d2b21eec57195dc7a2236d3f61640633ae1baa36bf142a3485b
2022-11-21 16:03:19 +00:00
stephan 9f2be5db04 OPFS contention test: give each worker a distinct recognizable name instead of a random one.
FossilOrigin-Name: 5f564bf7de7ce3ad7bedb5f06b3086ceaec55da768a60d74059fa4fba4328567
2022-11-21 16:00:26 +00:00
drh b53d8fa9f2 Performance optimization and size reduction in insertCell() by omitting
the "pRC" argument and instead returning the result code as an integer.

FossilOrigin-Name: bee94d1bb0daade023cc1e274339daafc249e1978c0765fc45042b5f9060e478
2022-11-21 15:55:57 +00:00
drh e7d5384418 Performance optimization to sqlite3BtreeInsert().
FossilOrigin-Name: b8976ebfe03fbc1e09a38d598a62493a7f19ff7a2a3acd1ec54d0dee190471e9
2022-11-21 14:13:10 +00:00
drh d01dee5e6a Small performance improvement in sqlite3BtreeTransferRow().
FossilOrigin-Name: dab959ea3edf99788bfd76352cd46a3e56876b0e7d7008c6927aa14534853c50
2022-11-21 13:35:00 +00:00
stephan a303392a2c Add new JS tests dir to those pushed to the test server.
FossilOrigin-Name: 07182dca9f2a4ffea1af0684c93e55e105465b2ee9820c70764e3e7bc1c28efc
2022-11-21 06:07:22 +00:00
stephan b38ac0986e More tweaking of OPFS concurrency measures and the related test app.
FossilOrigin-Name: a8d4da1501d411085ec2fd48c4a056c8b1d97ef3c3203c5b403a854ac2864870
2022-11-21 05:18:24 +00:00
stephan 36d5554c9a Resolve missing SQLITE_LOCKED result code which triggered a new (since last checkin) exception in the OPFS VFS. Improve output of the OPFS contention tester app.
FossilOrigin-Name: 2debbbca33bd4170a1dc4dbb5eb3e68523e51d289b06c551e5560ac4e32e433b
2022-11-21 04:12:38 +00:00
stephan 27c4cd183d Add test app for experimenting with multi-worker OPFS concurrency. Tweak OPFS VFS to significantly improve the otherwise "unfortunate" concurrency situation.
FossilOrigin-Name: 96f76e7616f8157a342b9e1c42f7b1feab200d182268871a2b25f67d4ee2564c
2022-11-21 03:50:52 +00:00
larrybr b5aaf0f285 Get clean builds for MS tools. Add build examples for extension glommer (basexx.c)
Make sure extensions build for dynamic load and built into shell.

FossilOrigin-Name: da940d507e9e31a84daedede44a56f743198d7258a9e83892511fe8dd4c78d2b
2022-11-21 02:18:12 +00:00
larrybr 423003dca9 Fix a base64 decode bug. Provide for convenient inclusion of extension(s) built into the CLI, to simplify testing and for its own sake. Improve comments. Cure collision between base64.c and base85.c when both are in the same translation unit.
FossilOrigin-Name: 07543d23a98c2a851393a2674e59d3cf1df37c244fb451cb7436f49c95c1423f
2022-11-21 00:11:09 +00:00
stephan 5aa5463e75 Remove check for WASM_WASI macro when detecting wasi compilation mode, as that macro is project-specific. Rely only on __wasi__ (exposed by clang) to detect wasi compilation mode.
FossilOrigin-Name: d469ac0c448eced26a697751ce9be316e8bc3cd029fda2b50966523c7850854f
2022-11-20 15:30:42 +00:00
stephan ae276719f0 js dist: account for a file rename in the previous checkin.
FossilOrigin-Name: 469f9011a885e19b99210c5e3e582afa140b8b5f0aa7a720334848df5ab6ae98
2022-11-20 05:47:17 +00:00
stephan 6d97c1a844 Further minor cleanups in the JS build related to vanilla vs ESM.
FossilOrigin-Name: 100a596800eca61477d9880092465d594c22be3707f2a11aaf6eb9e234fc6f2d
2022-11-20 05:36:52 +00:00
stephan 59a9654715 Remove an obsolete reference to WASMFS from ext/wasm/index.html.
FossilOrigin-Name: 51ff681864ec19844f8e7a46aef132e8a8601a1b64e1f5a243a53c6413f2a61a
2022-11-20 04:14:29 +00:00
stephan b2873a3014 Generic cleanups and doc additions in the wasm build files.
FossilOrigin-Name: d10f385e36ee7fe3077d80d8d6e7ce55732d20ef73e2a63533d8d2932ec8bf62
2022-11-20 04:14:00 +00:00
stephan 8e1bd8264d Add 'snapshot' target to create distinctly-named snapshot builds of the wasm deliverables zip file.
FossilOrigin-Name: f7620aa09e4893971e00cdee5c6f1fe15c3bd21f985bec90fbd90fdfc457ac31
2022-11-20 04:13:16 +00:00
stephan 0a94ef8b52 Minor cleanups in the ESM-related preprocessor filtering.
FossilOrigin-Name: 205884a273128bb666b496b659b4fa9f031ebdbbc1aa704fdeb4b7e015740098
2022-11-20 01:15:22 +00:00
drh ecba10730d New assert()s confirm that BTREE_PREFORMAT must be the same value as
OPFLAG_PREFORMAT.

FossilOrigin-Name: f40bf2c642643ae61d331e8d4815f601224fa258ab34344c6756966163a89f4a
2022-11-19 20:10:55 +00:00
drh 5a1d659d79 Performance optimization in sqlite3BtreeInsert().
FossilOrigin-Name: 9776fa31758161970a50995a487b6543ed71e9610460b7324304ef21d9248707
2022-11-19 19:37:26 +00:00
drh 16635fa777 Performance improvement and size reduction in balance_nonroot().
FossilOrigin-Name: 715bc81eb833ad4834d139a04085e0386c54c30d7395207e48972c4dfe5879c1
2022-11-19 18:43:32 +00:00
drh 47de1f9e19 Small performance improvement and size reduction in btree.
FossilOrigin-Name: daa07149c0a0fcb6a6a1ace6020ca68802588ed309f5aaaf99c871088bc46908
2022-11-19 18:17:40 +00:00
stephan 8d0d409876 Get tester1.js working via an ES6 worker module and add that variant to the dist zipfile.
FossilOrigin-Name: 90480586f1b2ad82118e19536b095431b8457f294c0afaa9b4f883f184cc804c
2022-11-19 16:16:40 +00:00
stephan cedc3eee70 Elide a wasi-incompatible shell.c block in SQLITE_WASI builds.
FossilOrigin-Name: 9b8b15a779158ec7c39d936f6bfa6a35e12063a36933341ade051809993a5678
2022-11-19 15:03:25 +00:00
drh 1e62057436 Improved comments on the pageFreeArray() routine of btree.c. No changes to
code.

FossilOrigin-Name: 0c2adc6d3547b07e950ae49f07f688f71a21b3ad5a51f16f0e8d49ab91564582
2022-11-19 14:18:48 +00:00
drh bf9b994f52 Streamline the decodeFlags() routine in btree.c for a small performance
increase.

FossilOrigin-Name: 4cb285210b4a2b14c80962bf2ecb35be310d3444c329c15d86b3073096455704
2022-11-19 13:09:03 +00:00
stephan 946c317ac6 Default to SQLITE_THREADSAFE=0 and SQLITE_OMIT_LOAD_EXTENSION for wasi builds.
FossilOrigin-Name: 382ba3b28409bbc54e6c5c3c9fe78980832e9f33261ee842e3b56e788300f669
2022-11-19 07:44:32 +00:00
stephan 9a92d53cf5 Account for lack of mmap(), getpid(), and shared memory APIs in wasi.
FossilOrigin-Name: 80ff026fb4b2203eea53d4930c1e9bb138db951fb408739c7d5c776fb397b665
2022-11-19 07:39:31 +00:00
stephan 8cfd2c3eeb Preliminary patches to get sqlite3.c building as-is in WASI environments.
FossilOrigin-Name: 19c04d71bf0f2f44d5a03288cbea3f63cf4eae2fa018ac6be4ae49a321e2b2f3
2022-11-19 07:17:12 +00:00
stephan 27a67968af Add build of sqlite3.mjs (ES6 module), add a test app for it, and include it in the dist build.
FossilOrigin-Name: 2e783670e10b59e67c14b0db7f4803b41790cc7730de221d54fa2d4483cfba33
2022-11-19 05:26:45 +00:00
stephan c768ef7289 Integrate a custom preprocessor to the JS build process to facilitate creation of both vanilla JS and ES6 Module builds from the same source files. There is still some build-level reworking pending to make an ESM build a first-class deliverable.
FossilOrigin-Name: 10c723d96d61d2e552ec1102563d58f1eb11bc3d30e03606fd8e0279c5a9043a
2022-11-19 02:58:03 +00:00
stephan f7d98ac764 More work towards creation of a ES6 JS module.
FossilOrigin-Name: 6b826e700f6849eebfbba38e5948b96be245994e3e03ea30743114d3f5689c42
2022-11-19 02:51:41 +00:00
larrybr fb2e0bfa8b New extensions for base85 and base64 conversion UDFs
FossilOrigin-Name: 5cc1fe1ddc2a33c59d3c006057e474c7c7975c483395ddea530df6968fe15341
2022-11-19 02:39:16 +00:00
larrybr ff4917e9a5 Create new branch named "base_convert"
FossilOrigin-Name: 0cbf55407a3a94b1c9c0ada52fa2995088bac3739876fa8d465dfb4dfcc4a6ea
2022-11-19 02:32:26 +00:00
drh 85728a21c6 Small performance optimization in btree.c.
FossilOrigin-Name: f710cce13577788cf3b95ed7089c3af2854271ff53f0a0b7b0619f315e331eff
2022-11-19 00:22:12 +00:00
drh cf3107c7d4 Databases created using sqlite3_deserialize() should report their filename
as an empty string, not as "x".  Fix for ticket [53043c9793715f08].

FossilOrigin-Name: ff494449efd475878c549728cc22ee9b12d13674068781747fc042a0c1bd09c8
2022-11-19 00:08:35 +00:00
drh 26e0f2e916 Update the version number to 3.41.0 to begin the next development cycle.
FossilOrigin-Name: 5c669f5f399fe89998b9edba6486f2a6fe5fca789ed82e8711349c8736b293d9
2022-11-18 17:57:19 +00:00
stephan 76abcfbdc0 shell.c.in: on non-Windows platforms, check for $XDG_CONFIG_HOME/sqlite3/sqliterc before ~/.sqliterc, per request in [forum:7a16582b1e403c81|forum post 7a16582b1e403c81].
FossilOrigin-Name: 17065d095d26a814acf1e13f5cc18b21fecc58eb8c9da100458029bb139fcd35
2022-11-18 15:22:45 +00:00
stephan ee026c5479 Replace use of cpp with the fit-to-purpose c-pp to avoid cpp's C-centric/JS-unfriendly quirks.
FossilOrigin-Name: 49d70f071e918d5d095c807575bb7ce2b287a123261e789e938521b3b409429a
2022-11-18 02:29:59 +00:00
drh d8e48fffdf Fix corner cases in UTF8 handling in the REGEXP extension.
[forum:/forumpost/3ffe058b04|Forum post 3ffe058b04].

FossilOrigin-Name: abb18f61c5cec0f524acc41453b4c06b61c5af51ff46417588837fc0c3967288
2022-11-17 19:24:39 +00:00
stephan c7c15d1b83 Merge trunk into js-cpp branch.
FossilOrigin-Name: e047b33d1fb7d6a32e967f03f9952249cd2da4d21dc301fe92bd7baa0da5d6a9
2022-11-17 15:21:49 +00:00
drh 3c1572ddb4 Use the log10() and log2() functions from the standard C library to implement
the equivalent SQL functions, in the hope that this will prevent reported
precision problems.
See [forum:/forumpost/cfceb1230bdcfd84|forum post cfceb1230bdcfd84] and the
surrounding thread.

FossilOrigin-Name: 7c572d02e60a83b36543ba4d9d45f61e9fc111b61fee085410c2d87558c732d6
2022-11-17 14:40:33 +00:00
drh 823872c6d6 Split out the documentation for sqlite3_value_encoding() into its own
page and make it clear that this interface is intended for testing and
debugging only.
[forum:/forumpost/c9f445453da950ad|Forum thread c9f445453da950ad].
Comment changes only - no changes to code.

FossilOrigin-Name: 9048a766ff7dfa0cd91ea74092e462f4501cb3f719033ccb55700bf5e4dfd0d3
2022-11-17 13:58:25 +00:00
drh 841c98e14c Raise an error if an attempt is made to compile the CLI with the
SQLITE_OMIT_COMPLETE option, since sqlite3_complete() really is necessary
for the CLI to operate sanely.

FossilOrigin-Name: a119a9e2ade4eac5feb1aa885c15b83e725f87386351de99d3abb49656219d50
2022-11-17 01:24:06 +00:00
stephan a872466c8e Change a self.X JS reference to X to account for a symbol resolution discrepancy between vanilla JS and ES6 modules, as explained in [forum:801d8f77e5115141|forum post 801d8f77e5115141].
FossilOrigin-Name: 0590de4da1103d842b9f9f25bcd2e69223b2ea067ae2f320f58dd3763218b39d
2022-11-16 21:52:29 +00:00
stephan 1d4f1cf426 wasm build: include FTS5 instead of FTS4, per /chat and HN discussions.
FossilOrigin-Name: 85c3f2ac5a887ba809f236c8c9d3837b2fdb578d4beae61bc78ada7ce03b58e2
2022-11-16 21:39:23 +00:00
drh 08efa45113 Fix harmless typo in comment, reported by Debian in Fossil
[https://fossil-scm.org/forum/forumpost/15f7327318].

FossilOrigin-Name: 5689f0d9ad1be532b274508938b25ff0d63027b8cc31f796dfaa2cca71d53642
2022-11-16 19:57:21 +00:00
drh a07df967f5 Avoid returning SQLITE_SCHEMA if the first query run on a connection
is "SELECT nosuchcolumn" or similar.
[forum:/forumpost/c1798f77ef|Forum post c1798f77ef]

FossilOrigin-Name: 3dc4f75d77417df1ef19be8e3191d246fb44ca7f7d1de7b161c5cb0f8aafeded
2022-11-16 19:53:39 +00:00
drh c524ff6526 Merge the 3.40.0 release into the begin-concurrent-pnu-wal2 branch.
FossilOrigin-Name: a090de41be2a727cdb98f02df3d793d94101ca7004c1023c86ae2ad55b5f9c5b
2022-11-16 16:11:07 +00:00
drh ce8a1720d6 Merge the 3.40.0 release into the wal2 branch.
FossilOrigin-Name: 2df915ef04a64cc8027330013f73ac0ee1fee8cb6eb61f3a608c21ea419d5928
2022-11-16 16:00:43 +00:00
drh 2f2c5e2061 Version 3.40.0
FossilOrigin-Name: 89c459e766ea7e9165d0beeb124708b955a4950d0f4792f457465d71b158d318
2022-11-16 12:10:08 +00:00
drh a70a01dbcb Previous change to the fake_big_file command in the test harness was not
correct.  This one should fix the problem.

FossilOrigin-Name: 6ee57a2e4e3399481100c40da0229d2d33cbe3290016185c8a60839d14b209f6
2022-11-14 19:42:01 +00:00
drh 1da477d3ab Change the fake_big_file test command so that it fails for files larger than
2MB on Windows, as we have observed that Window10 will sometimes bluescreen
when operating on very large sparse files.

FossilOrigin-Name: d39ffd2abadcc85c46ba17612f9f575005b4818e18bf39362b8ee574837cfad9
2022-11-14 18:57:21 +00:00
dan cfce889cda Avoid running bigsort.test with SQLITE_DEBUG builds. The assert() statements added in [7c96a56] make it too slow.
FossilOrigin-Name: fa68dd8928c76888d535e649794743eeefd3abe0f909792a128955710bfc948a
2022-11-14 17:48:26 +00:00
drh 859e8b07dc Merge the 3.40.0 rc1 changes into the begin-concurrent-pnu-wal2 branch.
FossilOrigin-Name: b0e2b9aa2c77af8c1e5b35a49f58610c8ac7063a63fd61c34bff6050348e068f
2022-11-14 13:18:39 +00:00
drh d9da500bd7 Merge the 3.40.0 rc1 changes into the wal2 branch.
FossilOrigin-Name: a5a610a650b6b49f4bb5fbb8e2f43c730afbe88df2c3ae4ddf3d1e5392960387
2022-11-14 13:10:42 +00:00
stephan 8d7b41302f Remove some outdated code comments. No code changes.
FossilOrigin-Name: 80ff592a9d0157bfa8bd1f9959c3aa26cab0ec16fdccf7b58a7523b8912c1a8c
2022-11-13 19:29:51 +00:00
drh 7e475e571f Disable the ability to change the schema_version cookie when
SQLITE_DBCONFIG_DEFENSIVE mode is enabled.  This is a security
enhancement inspired by the question in
[forum:/forumpost/2b9cc3dae1f1e5f6|forum post 2b9cc3dae1f1e5f6].

FossilOrigin-Name: 1d81381e8f5db5d7064cc313b8544ca3cb1ca9e8cd61e71368a2d2e598befc9c
2022-11-12 17:17:01 +00:00
drh a6303704a3 Remove unnecessary tabs in the ChaCha20 implementation.
[forum:/forumpost/0cdce5db8c|Forum post 0cdce5db8c].

FossilOrigin-Name: b7179efbdb2bdc878acec0abfe272821f7e0d7d9e5ef06cd7fd796ef295e54ab
2022-11-10 23:10:11 +00:00
dan 873a84054c Avoid returning SQLITE_SCHEMA if the first query run on a connection is "SELECT nosuchcolumn" or similar. Forum post <https://sqlite.org/forum/forumpost/c1798f77ef>.
FossilOrigin-Name: d31c019fd6849e4c3f8452e75c6cfefd613ade5355e896be368bd16fef28c627
2022-11-10 19:19:52 +00:00
stephan 2c4d62f013 Automatically relinquish implicitly-acquired OPFS file locks during VFS idle time in an attempt to help alleviate cross-tab locking contention like that described in [forum:58a377083cd24a|forum post 58a377083cd24a].
FossilOrigin-Name: 8daf24ff73dd9928057412e0e4c2e2b2e47e1dca66acfb6b07c846e8d97582cc
2022-11-10 13:22:35 +00:00
stephan da2641597a Rework automatically acquired OPFS locks to be released during idle time. This eliminates the performance hit reported in [46304ba057707c].
FossilOrigin-Name: a7fe91afca473fe55c983bc81d214df4ef3699863c7423fa4b6b9cde23d6a3b4
2022-11-10 13:14:30 +00:00
stephan aafa022f5b OPFS: if an op which needs a lock is called when no lock has been obtained, automatically lock it at the start of the op and unlock it at the end of that op. This is an attempt to alleviate the cross-tab contention described in [forum post 58a377083cd24a|forum:58a377083cd24a] but it increases speedtest1 run time by approximately 4x. Perhaps auto-lock can be combined with the older idle-time-based auto-unlock to unlock such locks (but not those from xLock()) to improve this?
FossilOrigin-Name: 46304ba057707c3b072b6e7bb8c4af774f653aa5814099f0866cd87b2b73abeb
2022-11-10 11:35:10 +00:00
dan 838865c0bb Better handle an error in the fts5 integrity-check code. dbsqlfuzz e87c62f9b67ea21aebdc36ab71cab7cc3eda8dc3.
FossilOrigin-Name: ae43e97087a3207a5ca3ffae75fbe7a33c01f4a38ce0d1d7eed8591ae3083617
2022-11-09 11:17:57 +00:00
drh ea038d3d23 Additional defense against corrupt database files in dbdata.c.
FossilOrigin-Name: 2e70d1e5c9b2c9e068be3ccf8a6062edf7bdde2e46d60ba8ce54eda851af6008
2022-11-09 11:02:39 +00:00
drh 92c0881d9d Increase the precision of the nData parameter to the dbdataValue() routine
inside the sqlite_dbdata virtual table, to avoid the possibility of integer
overflow.

FossilOrigin-Name: 295447b44fc7658891352a9b3f792379b52a76ee5dcfd498b7028f5f87b40e9e
2022-11-09 10:17:25 +00:00
dan 50ba1a0213 Fix a problem with URI formatting in calls to sqlite3rbu_vacuum(zDb, NULL) causing problems on win32. First reported here <https://sqlite.org/forum/forumpost/a7ae915a9d>.
FossilOrigin-Name: 77ccc8ea8a901df64feabe08e1210087d060f0bc133242e87a94a71b68b5b5fa
2022-11-08 20:04:05 +00:00
dan afe768193f Do not attempt to run test script dbpagefault.test with SQLITE_OMIT_VIRTUALTABLE builds.
FossilOrigin-Name: 901918c4867557e51160ef9c495e4e007de26cfe07c237addc00a388662957e8
2022-11-08 19:36:26 +00:00
dan 64c40ec2c7 Fix a problem in main.mk preventing the amalgamation-testfixture target from building.
FossilOrigin-Name: ccf00a0e22a2a9815ff7db1c6c6457fce35aaf50843c1b5ff2adda22d6f70314
2022-11-08 15:49:12 +00:00
drh 80fafc25ce Further improvements to the handling of the u32 and u16 typedefs in the
shell.c source file.

FossilOrigin-Name: 55a19677d723147aeb2b4a86bbd01756ddeb2072cba72c3145ad32d335e203b0
2022-11-07 19:40:20 +00:00
drh d9324a5226 Improve the ability of mkshellc.tcl to remove redundant typedefs from the
generated shell.c code.  This is needed to get shell.c to build on older
C compilers.

FossilOrigin-Name: 3645585f37631d60cefab1d55cdb1ee060aae87317b9b158a01329ca8a4d3e1e
2022-11-07 18:36:02 +00:00
dan 40503750fa Add the sqlite3rbu_rename_handler() API. To override the default routine that RBU uses to rename files.
FossilOrigin-Name: ebbb1f88e7b5d6cbe84d400f1a187acedb4c668d0b7e4c63bf1496e57da9b8ad
2022-11-07 18:00:18 +00:00
stephan 49507d2af6 Minor doc updates in speed-check.sh. No code changes.
FossilOrigin-Name: d2ed4116fbf1de3c840f84e05db6f29f7b489518ac07d56f61df153deab6bf6b
2022-11-07 17:21:53 +00:00
drh 54422235e9 Fix an obsolete comment.
FossilOrigin-Name: e377c0a1ef030395293c5f24d7cb8e5b36ce972e9fac31b99c8425075486a46a
2022-11-07 15:23:51 +00:00
drh e38b6e0318 Fix typo in comment.
FossilOrigin-Name: c3b94d7d4697a5c3983253f6266c544d8037617778a09d491bc12eb13f08ac75
2022-11-07 15:01:05 +00:00
stephan 690d4c545d Add sqlite3.wasm.alloc.impl() as a "public back door" into the low-level non-throwing allocator. Correct sqlite3.WasmAllocError constructor to behave like its usages expect it to and add tests for that.
FossilOrigin-Name: cea8bf9a144d842c4755c3130273524926e8c4831d7f21c4e34d4e8c74109c8c
2022-11-07 13:06:20 +00:00
drh da01757870 Fix requirements marks so that they match documentation corrections.
FossilOrigin-Name: 10d6189d23133006b39ea230045a918483721dd48f5558f77e57e23693097d16
2022-11-07 12:21:06 +00:00
stephan c0a18d6a5a Minor wasm doc touchups. No code changes.
FossilOrigin-Name: 2c448368913a844bdb5e69f8fa3bad91a2b6612ba3b7f3f650dd07b81db25a77
2022-11-07 12:12:27 +00:00
drh 1407458c6f Fix the documentation regarding negative length parameters for
sqlite3_result_text() interfaces, to point out that the length parameter
to sqlite3_result_text64() cannot be negative.
[forum:/forumpost/cf1d043b07|Forum post cf1d043b07].
Comment/documentation change only.

FossilOrigin-Name: 8016507651f377b08deb3a13cc559d56ce6e934c3073a8e63d05fd946b8403a4
2022-11-07 11:19:28 +00:00
drh b36f697b50 Make sure that the recoverStrlen() function (used internally by the new
recovery extension) is 64-bit safe.

FossilOrigin-Name: ea30a6bfc463acce26a4d710e7bf56519ad096ce2fe904ee7e199f5c55068034
2022-11-07 11:05:53 +00:00
drh b819202acf The recoverAssertMutexHeld() function is for testing and verification only
and should be a harmless no-op for production builds.
[forum/forumpost/b4f2c7d402|Forum post b4f2c7d402].

FossilOrigin-Name: d75504710c86af0037ac897106aadffe54955463195ee4dfb2cdfc81d396cbb4
2022-11-06 17:19:34 +00:00
stephan 6e8a3341ea shell.c.in: on non-Windows platforms, check for $XDG_CONFIG_HOME/sqlite3/sqliterc before ~/.sqliterc, per request in [forum:7a16582b1e403c81|forum post 7a16582b1e403c81].
FossilOrigin-Name: 49c6e438a83b9ff40ebadd3dfd5f58e6eea053575e15335909f5ee59a6dba82c
2022-11-06 13:12:11 +00:00
dan f53c0a0d4e Update test file dbpagefault.test to account for the restriction on using SQLITE_VTAB_DIRECTONLY virtual tables from within triggers.
FossilOrigin-Name: 2b68fc8aa35cc69e4d2c26aaebbf3f2b1f1c08d15ca9efcbe5be21d45735d3f1
2022-11-05 19:26:45 +00:00
drh 7bc4ce3412 Merge the latest trunk enhancements into the begin-concurrent-pnu-wal2 branch.
FossilOrigin-Name: aa2e247b58dea930c4f0af3566d287f196ea178be8e994fc9eec685a89bebac8
2022-11-04 19:09:41 +00:00
drh 82ac3d8211 Merge the latest trunk enhancements into the wal2 branch.
FossilOrigin-Name: ca63a1bee16d30677c20c7576361dfb9a359e6e1b2b2b58a574da0059d3a8822
2022-11-04 18:58:48 +00:00
dan a16edfa29b Tweaks to recover module test scripts to work with various permutations.
FossilOrigin-Name: 454c61e818f6941c9a23f6600e37828a3f2b2ad3c6dbc8d6223570aed5d9cd8b
2022-11-04 18:32:45 +00:00
dan 5f7e173abf Another test case fix for auto-vacuum builds.
FossilOrigin-Name: c1c47eff48e0831b5aed987e90d797aee99caf1861fa8318c6c3ecfb108de7e6
2022-11-04 17:07:33 +00:00
dan 80dbd818f2 Fixes to recover module test scripts so that they may be used with SQLITE_DEFAULT_AUTOVACUUM and SQLITE_SECURE_DELETE builds.
FossilOrigin-Name: 0e5597ce5353dea2cdb092b166b57ba1d60f8115eb468349f2b2869803691a2c
2022-11-04 16:39:39 +00:00
dan ffc9b1b046 Fix Tcl tests so that they again build and run with SQLITE_OMIT_VIRTUALTABLE defined.
FossilOrigin-Name: 06a9dbea40c8a0cdfae6b127a1ce1cba0547acdf1115c087e9b790a78c264b52
2022-11-04 15:17:14 +00:00
drh 881f0f4114 Remove an unimportant testcase() macro that is no longer reachable due to the
improvement in corruption detection.

FossilOrigin-Name: cd7e3568a9b0c0f5ef5ca8a29c55cefe2bbad2dbc88a3718a87bb24a8ada3b7f
2022-11-04 12:59:04 +00:00
dan 4868a86513 Add another test case for the OP_Found fix.
FossilOrigin-Name: 6cdd64a445fe16d547d7bf5ea26dad3ab6d9d5f7d1c2b49fdc9d7a2edaa031c6
2022-11-04 11:59:40 +00:00
drh 093677add4 Enhance the ability of the OP_Found and similar opcodes to detect truncated
index records and report SQLITE_CORRUPT.
dbsqlfuzz 2b12f90aeff8e081706c7e9b58834f04869f446c.  Test cases in TH3.

FossilOrigin-Name: 059a09da2c5fd9c7e723c713565fbaf71602079feef0704129cc5cbbd0033936
2022-11-04 11:54:42 +00:00
stephan c75445087f Correct sqlite3-wasm.c's SQLITE_DEFAULT_CACHE_SIZE (it's measured in kb, not bytes).
FossilOrigin-Name: 479ad980dfe509403e184e39a5aa441171e47b3297e05039f85516e72e9f15be
2022-11-04 09:02:21 +00:00
stephan 7d24ff2971 Add experimental esm target to ext/wasm/GNUmakefile to tell emcc to generate sqlite3.mjs (ES6 module) instead of sqlite3.js. Related to discussion at [forum:ed4596cf8496a39b].
FossilOrigin-Name: e55d8eba83012492d85418dc0faedce5896027ecc70295a5ca1826f61a5edbaf
2022-11-03 22:51:20 +00:00
stephan b0ab21dee4 Experimentally use clang's C preprocessor to filter #ifdef's out of the generated sqlite3-api.js, the goal being to see if we can feasibly use cpp to include ES6 module-specific code in the main code base and conditionally filter it out.
FossilOrigin-Name: 718a6d371e61359d73c8f80afdb248e3d9b4d8df4c4e5c122ac884344e31035b
2022-11-03 22:14:47 +00:00
stephan 4df2ab575f Globally replace '' with "" for empty JS strings to please C preprocessor.
FossilOrigin-Name: e92e1f42bef94a1df29f66b4111ebfde93eba3759bc5d5a9c95f714508851346
2022-11-03 21:21:10 +00:00
stephan de6186e04f Minor build cleanups and fix a harmless race condition in the OPFS part of tester1.js.
FossilOrigin-Name: 70ee6ee014bc4e2c1daa9b4a8909cf76ccecf32de1eb39055f20d3d0b1daa1bd
2022-11-02 14:08:59 +00:00
drh 2cdcc7f01a Clarify documentation regarding the --recovery-db option to ".recover" and
the magic 789 configuration option it is associated with.

FossilOrigin-Name: f6fa0cffa921ccde8910e7fa4a63c2e4ef8ddb376c8ce99e436b27ac332c4498
2022-11-02 14:08:26 +00:00
drh 12de518db4 Minor improvements to API documentation in comments for the recovery
extension.

FossilOrigin-Name: ed1c3515ad6a988e07a8b4583fbc38be257e6eae7443b01a242b98207ce78162
2022-11-02 13:09:14 +00:00
stephan f45c33701d Add sqlite3_wasm_vfs_create_file() to replace Emscripten's FS.createDataFile() in a (mostly) VFS-agnostic way. Add a test for worker1's export (to bytearray) support. Re-add worker1 open-from-bytearray using sqlite3_wasm_vfs_create_file() but it's untested (requires a new interactive test app or maybe reconsideration).
FossilOrigin-Name: b35e1225c91a3cadc0d25af1e4e790237256d194990faa13190e343ed03e11c5
2022-11-02 11:53:31 +00:00
drh faff0410dc Emcc seems confused by SQLITE_DEBUG, for reasons unknown. Use NDEBUG instead
to simplify the #ifdef logic in sqlite3recover.c.

FossilOrigin-Name: 2610779ac84ac4a1a6901b6244653faf0c49ac6f0a4710a19aaf2a13106ae742
2022-11-02 11:25:33 +00:00
stephan c18c8bf957 Fix a symbol name typo which broke and exception-handling case in OPFS xOpen().
FossilOrigin-Name: 5bc83d569594e104e90b1acef1a5fd23655b2089de393a6776e799fdef2082f5
2022-11-02 08:49:05 +00:00
stephan f170b97985 Fix a missing symbol in non-SQLITE_DEBUG builds.
FossilOrigin-Name: 17cdcad627205cdc8ae8114df44ad2a44e59603200efb238fbbb5c30ea0a453e
2022-11-02 08:10:22 +00:00
drh b9f8ff0e8f Fix a harmless compiler warning.
FossilOrigin-Name: 663a89164fdd43529a13cab4b9d9f04fe519e7c6b954f9c91e32e4d7ae25eb46
2022-11-01 21:42:50 +00:00
dan 41dbdae4e2 Instead of (const char*), use a special type - sqlite3_filename - as the filename argument passed to VFS method xOpen().
FossilOrigin-Name: 90df04e5a25907e748227c484b601b7dbd87c037556a0e87f8423f529d08bb6b
2022-11-01 19:35:54 +00:00
drh 6da7cc9b4c Fix harmless compiler warnings.
FossilOrigin-Name: de829770aa0f59b4cd3ab1dc5eec90be82663dca3199c2cf6d97f4c41ce9f799
2022-11-01 18:56:39 +00:00
dan 52d5d474f5 Instead of (const char*), use a special type - sqlite3_filename - as the filename argument passed to VFS method xOpen().
FossilOrigin-Name: bd1fe4de8a2e8b8de19a7f90b9fdafb641ad11a6f0c96c0b4495d5bf80885675
2022-11-01 17:43:19 +00:00
mistachkin 11980a6b8e Fix compilation errors in the recover API seen with MSVC.
FossilOrigin-Name: 78d17c4fedc1629bf161f9858f2b8bf35506155ccaa0acc084b419d244d4f75d
2022-11-01 14:13:05 +00:00
drh dddec5cadd Fix a #ifdef involving SQLITE_OS_KV that was adding code unnecessarily.
FossilOrigin-Name: b6c1b6e4a3bc21c40d6faca6f70842df5201f21c96a076a895410660630461c0
2022-11-01 13:12:20 +00:00
drh be44e188e3 Enforce affinity on materialized tables for subqueries and views. Also,
do not allow UNION ALL flattening if the affinity of a result column varies
between different arms of the compound.  This is a fix for
ticket [57c47526c34f01e8].

FossilOrigin-Name: 88a05141c28e5ff1357c3c599493e4ffb8f3821bab04be80244deac62e49135c
2022-11-01 12:10:39 +00:00
drh dc5e8c63bd Improvements to comments. Change the "optimization_control" TCL command
in the test harness so that it returns the new optimization mask, for
verification.

FossilOrigin-Name: a3a500127d4752ee05c183e9ac97a2bc27768bf7063b04b5c1405cd3fd3931f5
2022-11-01 12:01:10 +00:00
stephan 0f32760e3c Minor internal cleanups in the js pieces.
FossilOrigin-Name: 271391b4e32220ab4c32d69f579ecd2b03eb99da898955a1ef8fffc27216719d
2022-11-01 11:09:34 +00:00
stephan 49048b148e Significant cleanups and expansion of the sqlite3.opfs utilities. Add oo1.DB.dbVfsName(). Add VFS name to worker1:open's arguments and result.
FossilOrigin-Name: 86a341d7e061f946b39e8647ddd4743013b851b33ae9e6e755d8dbc53fba5286
2022-11-01 07:49:49 +00:00
drh b6d9167945 Do not flatten a UNION ALL subquery where the column affinities do not match
on all arms.  Force that subquery to be materialized.

FossilOrigin-Name: 1b8a128832c1633cbb86999d347fef64ac45fe134dac61132112fc97f24676a0
2022-11-01 01:07:29 +00:00
drh 5fdb9a352c Enforce column affinity on the materialization of a subquery or view.
FossilOrigin-Name: b6692de374d7f489d8cf8877bdfef19501bc5ecb0781d42750807eeb58f2e7cd
2022-11-01 00:52:22 +00:00
drh c7dd9b60eb Make the UPDATE method of the sqlite_dbpage virtual table rebust against
OOM errors.  [forum:/forumpost/bbcf0dd6ca|Forum post bbcf0dd6ca].  Test case
in TH3.

FossilOrigin-Name: d15c9a4a323b825eb80e706e12e9df95e5db458024f51e6e537940efc8234d8b
2022-10-31 18:01:05 +00:00
stephan 02d15a7ebd Worker1: use a list, rather than a single slot, to manage the default db link so that we can keep the least-recently-opened db as the default. Re-introduce worker1.export() but do not yet expose it until an equivalent import() (or open() option) is implemented.
FossilOrigin-Name: c7750c101d9b7baa31496731bc34ea0a9b2bad0c11e2b3e92a8c7b327135c1bb
2022-10-31 13:50:05 +00:00
stephan 43b442a626 Internal tweaks to the OPFS VFS result codes.
FossilOrigin-Name: 32e1a2d2eb8738ae1635e413829f6cf7f64c63d2a86b72940a573de89948e529
2022-10-31 11:53:34 +00:00
stephan 1acfe91582 Add oo1.DB.exec() 'returnValue' option, which specifies what exec() should return. Defaults to the db object and enables direct return of the result rows array or a list of the individual SQL statements. Other code-adjacent internal cleanups.
FossilOrigin-Name: 69d36a6aa5e2cd79d26c0fd3e0d20fe8838fd1be97db07725233bfff1dfe6643
2022-10-31 11:09:14 +00:00
dan 549907fd67 Avoid a segfault that could occur when running the recover API on a database handle with memory-mapping enabled.
FossilOrigin-Name: e02c697281a777c33070168af784b2d291397e488244a217620897f40aed7158
2022-10-31 10:53:23 +00:00
drh 1e84e1e632 Extra OOM check in shell_error_context() of the CLI.
[forum:/forumpost/5708841db0|forum post 5708841db0].

FossilOrigin-Name: cb12ac5de17e677f089d7b0b46803efbd9a9178972ffb0454a8b557b06633658
2022-10-31 01:22:38 +00:00
stephan 50ef01398d Add oo1.DB.selectArray() and selectObject().
FossilOrigin-Name: 7660db2a2e9c4f3a6a9343d6929744ad0f4be6820976411f9080165491da59b7
2022-10-30 11:39:47 +00:00
stephan 9163ef1f4d Adjust automatic OPFS locking retries such that they will wait up to a total of 3 seconds spanning 4 attempts.
FossilOrigin-Name: bbce49d81abc491ca666474040a53057e26e99131fea9449de253f88f4415ded
2022-10-30 10:24:53 +00:00
stephan 1fc6ffccc5 Minor WASM build cleanups. Enable custom Module.instantiateWasm() when not in WASMFS mode (where it doesn't work). Add sqlite3.debugModule URL param to enable some module-init-time debugging output.
FossilOrigin-Name: 50f678846a2b3c3d0818f0bae89f2ee86252a2e6a9c7029ebaae3953ca0fa14c
2022-10-30 09:47:33 +00:00
stephan af9cee12c1 Reduce wasm module's starting memory from 128mb to 16mb. The latter value is needed only for WASMFS-based batch-runner.js, as WASMFS builds cannot be configured for dynamic memory growth without a tremendous performance hit.
FossilOrigin-Name: ed9d93c48752ba6e18edc1238a8ccdbf6bb65c74fb52a89d515f8b0ab6bea13a
2022-10-30 08:39:18 +00:00
mistachkin f5e7691e2b Minor updates to the Windows batch tools.
FossilOrigin-Name: b42f56205fa8e06431fb1a45c48257d93605fea0ed95bd766f71e27292d1af33
2022-10-30 07:01:31 +00:00
stephan 1cc77c9543 Update ext/wasm/README.md for recent developments.
FossilOrigin-Name: c76ec31ebf4d80ec8cbd2aa2017e1d6bc32eae66bfbcc130c053ddb403599c6b
2022-10-29 12:31:50 +00:00
stephan da27e4aa02 More styling of module-symbols.html.
FossilOrigin-Name: a36f42a8364b745c410a6f837d005a193f6a93134cac087ac18bee2897cad730
2022-10-29 10:14:56 +00:00
stephan df52a0bc78 Expand and document the wasm doc-update make rules. Extend the dist rules to fail more readily if there's a problem.
FossilOrigin-Name: 362ec11ec09f441b745cbd8ac8366b6bc998aba6e2b70782a88f189a3ffa9923
2022-10-29 09:45:01 +00:00
stephan 8948fbee29 Move the sqlite3.capi.wasm namespace to sqlite3.wasm. This causes a tiny bit of naming confusion with the sqlite3.wasm *file*, but seems to make more sense than having it as a sub-namespace of capi.
FossilOrigin-Name: 3f16eb18d6186ca972fca952ccac18649e7a905213f589e53c0c9333e695448d
2022-10-29 07:54:10 +00:00
drh 053bb22f35 Fix harmless compiler warnings in testing code.
FossilOrigin-Name: 2f53612388507f9b1ecbec3c15281825beab54b8062d97d0ab6f009c5fed347b
2022-10-28 18:52:05 +00:00
drh 71b65e88e8 Add sqlite3recover() support to fuzzcheck.
FossilOrigin-Name: e65c5bdc5bd6e1989c3de59798ff0a761125fc490fdec8c7671ca70ed688c4f8
2022-10-28 18:35:06 +00:00
stephan ffbc653d85 Remove sqlite3_interrupt() from the WASM build, as it is essentially impossible to employ in JS's threading model.
FossilOrigin-Name: e85387590061edbb6cdc04de792ab86f43afaa5c8d1b0792206cb5b481c7d76a
2022-10-28 14:33:50 +00:00
stephan f03ddcca32 Add ext/wasm/module-symbols.html. Adjacent minor JS touchups.
FossilOrigin-Name: 24f12e681e06e3b71a6ac9c82255fe0270953a74c711405841f7e385eeafe874
2022-10-28 11:40:46 +00:00
stephan e681b651eb Minor sqlite3.oo1 API reorg prompted by documenting.
FossilOrigin-Name: 55237924ca363c5a3e0f8c82620eb6626a98311615866197375a86876b9da3a1
2022-10-28 10:36:18 +00:00
drh c97bb14fab Fix a minor problem with sqlite_dbdata.
FossilOrigin-Name: 58c2e49279c8f77f006b0c2ce351776bc711633f9c1a39c7a37969838d49ec0b
2022-10-27 23:17:49 +00:00
dan 676d1cda92 Fix another crash in the recover extension triggered by a corrupt database.
FossilOrigin-Name: d125377378916fb3147612a536fde15ee955176eea9ea6eaa4a2ee5f1e8e1cbe
2022-10-27 20:27:45 +00:00
dan 10f5525410 Fix another problem to do with recovering from in-memory databases.
FossilOrigin-Name: 5a7bfd4a217cd80882a1827674b479cdc547419a785e06e0ee82cb8cfca42701
2022-10-27 19:06:42 +00:00
dan ac16a4716b Ensure that the VFS wrapper in the recover extension does not attempt to invoke undefined methods on the underlying VFS.
FossilOrigin-Name: 347c575108bc8112613a02afda1247bdbed24c15def7eb412d0b27aba047715b
2022-10-27 19:02:51 +00:00
dan 895f12dfe7 Fix an assert() failure in the recover extension triggered by a zero value as the first page-number in an overflow page chain.
FossilOrigin-Name: 44c0874c33e76ce708022e5c38c74f75565a8cea9f4c5b067cc0a54079aeee20
2022-10-27 18:35:09 +00:00
drh 4853aa18f0 In the CLI, ensure that input to utf8_width_print is not NULL, or if it is,
change it to an empty string.  For for the problem reported by
[forum:/forumpost/2961cf13eec61876|forum post 2961cf13eec61876].

FossilOrigin-Name: 9ecc9d25ea62a7777bbf621989a5c577b612784121a20465cda4d4023fa9634d
2022-10-27 18:20:08 +00:00
dan f7fea5b4b8 Change the name of the ".recover" command option "--freelist-corrupt" to "--ignore-freelist".
FossilOrigin-Name: afce8443ace57c3c99c6810be98f05ed716ae504b68098726c869843c643bbf0
2022-10-27 18:19:45 +00:00
stephan 674b312ed7 speedtest1.html: when vfs==kvvfs, increase --size from 2 to 4, as enabled by [a4d40f6346e7]. --size 5 is 4.96mb out of (supposedly) 5mb, which works but is a bit too close to the edge.
FossilOrigin-Name: a608d584a8a68ebaf8566cffc9c33f59309b710e7b29229a15bef529702f35d5
2022-10-27 14:41:38 +00:00
drh fd6080637c Automatically set temp_store=MEMORY if the VFS is kvvfs.
FossilOrigin-Name: a4d40f6346e7eb2a5239684dba86f297358122768a4d4bf6786b6028f4300e04
2022-10-27 14:28:15 +00:00
drh 9c17ee4f93 Do not accept a NULL pointer for the filename in KVVFS.
FossilOrigin-Name: fdecbd3b5bcdeb358015f461ac6c863f12ab180ab4624ed50734e41e557d8186
2022-10-27 14:00:21 +00:00
drh f7b58da831 The kvvfs VFS should remember the page size when writing to the database.
FossilOrigin-Name: fc5503c8acb085042e877c9de43256a4fee27f63ab35e2ddad18e40734ed00d4
2022-10-27 12:46:17 +00:00
stephan bb5136e5cb Re-enable the .recovery and .dbinfo commands in shell.c.in in the fiddle build, as those were fixed by [3d20d77a3511] and [cd0aa27d1732].
FossilOrigin-Name: ba3a7a4a9997f62b78467d78d95fa5b58a03ecb2e2ccb68f4d316fc08fd02f86
2022-10-27 11:32:20 +00:00
dan 340b1d1a71 Do not use sqlite3_result_text16() in the recover extension if SQLITE_OMIT_UTF16 is defined.
FossilOrigin-Name: 3d20d77a3511bb223474da8491e1da8aec098c41c1b81ab78ff80fbd37d5b568
2022-10-27 11:25:08 +00:00
dan 48a4a4b73d Fix a problem with running ".recover" on an in-memory database.
FossilOrigin-Name: 87b4cca2b84845cee603d40fbf16c9635d6dc2d19d2f37ae86abfeff716c9a85
2022-10-27 11:12:54 +00:00
dan 21fd7ce098 Do not use sqlite3_mutex_xxx() functions in sqlite3recover.c when built with SQLITE_THREADSAFE=0.
FossilOrigin-Name: cd0aa27d1732abc61c8a8440118ff629f7dca185d2f515ce1f6090c49d8dc890
2022-10-27 10:51:49 +00:00
stephan 4f975c3379 Minor doc typo fixes.
FossilOrigin-Name: ed8d3f25a4d6ac04d9f7918c791d8d2c6f23ce846278ca63f8fbadb7ea27369f
2022-10-27 03:57:48 +00:00
stephan 3d420836fd Restructure and simplify the feature-detection #defines of the recovery support in shell.c.in and disable it when building fiddle because it uses features we elide from the wasm build (e.g. utf16), leading to link errors.
FossilOrigin-Name: ddd10c05c5d16d7205c347c5c1e73a75faecb2b7ec88a4a15875856765e1b143
2022-10-27 03:56:01 +00:00
stephan de868175c3 Expose sqlite3_randomness() to WASM and add a custom binding for it which can populate a JS byte array. Add WhWasmUtil.isPtr().
FossilOrigin-Name: 333e67076b4bc967bb543ef8e265c63f6e3498c38ac121a7d1eff4a1d7a71c63
2022-10-27 03:03:16 +00:00
dan 195687f1bf Disable the push-down optimization for sub-queries that are INTERSECT, UNION or EXCEPT compounds. dbsqlfuzz a34f455c91ad75a0cf8cd9476841903f42930a7a.
FossilOrigin-Name: 346a3b12b861ce7ba369e98cd336f79a1d4f7a7bb9acd7a4f63f37b391755bf5
2022-10-26 21:14:21 +00:00
drh 07dd481010 Remove an unused variable from the recovery extension.
FossilOrigin-Name: a029dddff4f4ed7275538610cbd9cea658b905b72924860ec9cda9e76dabcfac
2022-10-26 20:12:46 +00:00
dan 387f468591 Add the "recover" extension in ext/recover/, for salvaging data from corrupt databases.
FossilOrigin-Name: a820792548da596ed0f8ab3fd1c04ec16647abbe51a2a5aac9e17c4f58be6d97
2022-10-26 18:41:12 +00:00
dan 50632afa6e Remove an undefined left-shift operation from the recover extension.
FossilOrigin-Name: a67082357a2cc348faf8236aafa7f39eb5cb673b1d114a594c6d5bb257f85b73
2022-10-26 18:29:19 +00:00
dan 5ad03a486e Merge further changes from trunk, including fix to dbdata.c.
FossilOrigin-Name: bcf6b48d52c1ce656899f50bd508c0920cae8cf2ef3c0758a7633981f0f0f484
2022-10-26 18:22:22 +00:00
dan 8364ce2aa2 Make the shell .recover command and the dbdata.c module more robust in the face of corrupted databases.
FossilOrigin-Name: 4eef562a00ae988f2426c9af51f4165c0e4cbccd601061664a0c54c19b9cc70f
2022-10-26 18:04:34 +00:00
stephan 8c3e234ad5 Correct misuse of localhost-mode-only symbol S in fiddle-worker.js, which should fix the outage reported in [forum:67d985ac0bbe407f|forum post 67d985ac0bbe407f].
FossilOrigin-Name: 3b5aa50c223ac35c7d73e4629420a01408cd74d19ae5b887f91b7a657d91e026
2022-10-26 15:40:17 +00:00
stephan 7731dfca49 Corrected link to fiddle in ext/wasm/index.html (broken by [3d7b4f36b7dd]).
FossilOrigin-Name: 6efa0bae1678bf719a0693feeceb4c1e252fc86502bee9ee2a6d8af9a91d9b20
2022-10-26 15:34:51 +00:00
stephan 34b92b19f3 Replace the newly-removed oo1.DB.getFilename() with DB.dbFilename() with the hope that the distinction from the dbInstance.filename property is clearer.
FossilOrigin-Name: 0b80543d75504c733b73b32f73b96074a4b0cebad64e47032926a3fefee41726
2022-10-26 11:27:33 +00:00
stephan 14ae1a5331 Remove oo1.DB.hasFilename() and getFilename(), as they are unnecessary.
FossilOrigin-Name: 75c546b982024bf2b527c85271111d54158cb010867f6e29fc89e86cc8457cc5
2022-10-26 11:12:14 +00:00
drh 705d7be62d Remove the sqlite3StdTypeMap global constant that was made
obsolete by [aa6e908619624867].

FossilOrigin-Name: 3dfdfb3f12edb3f4267942598efd05d573e13b7c5d6cdbc3404373f41b8993dd
2022-10-26 11:11:31 +00:00
stephan 419e0d3e09 Add --download-version flag to ext/wasm/version-info.c and correct dist target's output version format to match.
FossilOrigin-Name: 6f2a40d06dd9a56491d27928ed3785e08308c7dcea2b3f768097fc98ba91a910
2022-10-26 06:05:44 +00:00
stephan e896ba94e3 Add doc page link to the wasm dist README.txt.
FossilOrigin-Name: c2380668d9a2ad04f6f27ef052190bda17cdb112895e93b94ae1da70db6983c8
2022-10-25 16:57:42 +00:00
stephan 4831f4e0f1 Correct -O level for fiddle build when built from the top-most directory (-Os instead of -O0).
FossilOrigin-Name: ff4fc29c38b78dfd471e25942304cba352469d6018f1c09158172795dbdd438c
2022-10-25 16:04:44 +00:00
stephan ce2d4ae6a6 Rename fiddle.html to index.html and move fiddle push-to-server rules from GNUmakefile into fiddle.make.
FossilOrigin-Name: 3d7b4f36b7dd01058d57fc8ef7689d9bbf1a219166796ed427180cd59d2c189d
2022-10-25 15:59:27 +00:00
stephan 16603ecfcb speedtest1: use the current (or default) vfs->xDelete method to unlink the db. This is specifically necessary when running the opfs vfs in a wasm build of speedtest1. This worked without this fix until recently because the affected test code was performing similar acrobatics in its stead.
FossilOrigin-Name: d3c830bd37a402161b486d3631feceae5d00f89c2522aed65fcd2e666bd64238
2022-10-25 15:38:38 +00:00
drh 42dddb97de Enhance the sqlite3_strglob() and sqlite3_strlike() interfaces so that they
will do sensible things with NULL string pointers.  This is an extra layer
of defense against bugs such as reported by
[forum:/forumpost/730b554179|forum post 730b554179].

FossilOrigin-Name: 8ba9c884bded52e3e044ff39c826c04838e7c31f05d802f4a14d9ce5a01ab721
2022-10-25 13:44:18 +00:00
drh d5ca2c4b3d Make the ".dump" command of the CLI more robust against malformed databases.
See [forum:/forumpost/730b554179|forum post 730b554179].

FossilOrigin-Name: 0573edfb9e0b1fd51e8da20592c71b02040968c33fdec41c31a6b1b7d3e83262
2022-10-25 13:42:10 +00:00
dan 5f726694eb Merge latest trunk changes with this branch.
FossilOrigin-Name: 757e3f585959f4f113fee48fe3d504f037604c53a95c8d47ce5c2bccfb2af8ff
2022-10-25 11:05:31 +00:00
stephan 89954fe7ce Minor doc tweaks in the JS GNUmakefile.
FossilOrigin-Name: 9c8df130b8137cb4705f805907d1ff754d7f4b07bf181e1d05f1798b7aa5f4f6
2022-10-25 08:41:00 +00:00
stephan d3c8f2ac33 Remove persistence from the 'reverse log order?' checkbox in tester1.js to eliminate an inconsistency between the main-thread and worker-thread modes (the latter having no option for persistence of that setting).
FossilOrigin-Name: ecf906b07f50b4fcc551d20326a1bb1189bd6c2274627fd293f9460658ebea07
2022-10-25 08:15:57 +00:00
stephan f861b36bf4 Minor cleanups and doc improvements in the OPFS sqlite3_vfs proxy.
FossilOrigin-Name: 48645f7bcacf81c4149f26d20ee1752fbe93a02f96b85bd7e28bfa49322137e5
2022-10-25 08:06:17 +00:00
drh 7704a535d0 Fix typo in comment.
FossilOrigin-Name: f65c95658fe4d30817da8de7eb88e823ea1cd8be40e347d626870bad3cc13359
2022-10-24 18:42:45 +00:00
drh 99a37ca779 Add a comment to a previously undescribed member of the Walker union.
FossilOrigin-Name: 69d704224e9ed022fcec591beff2ffcc4daf3e7fc586debfdcf00b25c1fddd87
2022-10-24 18:33:50 +00:00
dan 80b30f995e Use the same "PRAGMA synchronous" setting for the output of a "VACUUM INTO" as are configured for the database being vacuumed.
FossilOrigin-Name: 86cb21ca12581cae9a29f42ba707bd9d789e667e5ddc0f64b24940d5d5c9a118
2022-10-24 15:51:24 +00:00
drh 4073706a47 Improve the ability of the query planner to recognize covering indexes even
on tables with more than 63 columns and where the index is over columns
beyond the 63rd column.

FossilOrigin-Name: f058773e41495ddbae698f9e9a4f62b7003112ea8614dfad69471340058735e4
2022-10-24 13:20:48 +00:00
drh 2d75c1aa3d Update the fuzzinvariant logic in fuzzcheck to the latest code from
dbsqlfuzz.

FossilOrigin-Name: 739ad584765f1fc0eb196db870785da1726805cc2d9dd0a831a37a9e55a4afe9
2022-10-24 12:38:32 +00:00
drh 1f41a8c8a7 Fix built-in documentation for the ".mode qbox" command in the CLI.
FossilOrigin-Name: 3d7ea33be1076fd21681bdcff0c489646c017594b42b4a21a74e9e2d3947890b
2022-10-24 11:10:40 +00:00
drh 89b373ab28 Test cases for covering indexes on wide tables.
FossilOrigin-Name: 64c3777eb356b15094a9e9fcc135499ba22f4d847fb117afa6ccf19986e33585
2022-10-24 11:02:02 +00:00
drh 7d913e9aae The wide-table covering index detection must take into account aggregate
queries.

FossilOrigin-Name: 9ac73f83c53b9fba10bb1b8bbfd1a8dc8a2dc4a655e811d5b96335dbf2b6fdac
2022-10-23 20:09:13 +00:00
drh fb643592fe More frequently send the complete SELECT statement into sqlite3WhereBegin()
so that it can more accurately determine index coverage.

FossilOrigin-Name: f6d09c5a0df940dbaa716df70df28218985b83540b34c468989729a3421ef38c
2022-10-22 23:09:29 +00:00
drh 54cc766b41 Enhance the query planner with the ability to discern when an index is
covering even when it indexes columns well beyond the 63rd column.

FossilOrigin-Name: 1390417be45dd84e9118f6e761f23b8ff7476d26411e165bbaab678881e4eadd
2022-10-22 20:13:46 +00:00
drh f55a7dad19 This branch attempts to improve the detection of covering indexes. This
first check-in merely improves a parameter name to sqlite3WhereBegin() to
be more descriptive of what it contains, and ensures that a subroutine is
not inlines so that sqlite3WhereBegin() runs slightly faster.

FossilOrigin-Name: cadf5f6bb1ce0492ef858ada476288e8057afd3609caa18b09c818d3845d7244
2022-10-22 14:16:02 +00:00
drh 5723c659fd Clarification on the meaning of SrcList and SrcItem and especially the
SrcItem.colUsed and Index.colNotIdxed fields.  Comment changes only - no
changes to code.

FossilOrigin-Name: d96f6cc8475ae5509b8bff2db75e3c6f69a214d58d8979fbc0162ae488a040dc
2022-10-22 13:49:35 +00:00
stephan 3725af73b9 Add SQLITE_DQS to the compileoptions_used list, per request in [forum post 8b1060122b|forum:8b1060122b]. Force DQS=0 in sqlite3-wasm.c.
FossilOrigin-Name: fcd9e0dbe3226f3f7ccc15b11fc3aa3b8058571bef274c25a33e9753e22f7551
2022-10-21 17:48:49 +00:00
stephan 89ccfac0ac Macro name typo fix in OS_KV builds.
FossilOrigin-Name: 9cf1142b0cdb13347e3f551c862cb4714cadfe5ad637f049cf0a4b8bb6125b32
2022-10-21 17:37:05 +00:00
drh de033d078b Ensure that the RtreeCell object in rtreeUpdate has all bytes initialized.
FossilOrigin-Name: 1e38742bde09cd879eb370bcf8604ed7ed00759209bcb4ed0374350e4b9e842d
2022-10-21 17:18:58 +00:00
stephan 98147dd526 Add kvvfs tests to tester1.js. Fix a scopedAlloc() misuse in oo1.DB ctor caused by refactoring earlier this morning (and caught by these new tests).
FossilOrigin-Name: 8e0f001ab76de6dbc17295b9085f7f61ce274c43f8c432ea4d2ec3153d248ff3
2022-10-21 06:58:27 +00:00
stephan 848d27ada1 Add timing info and OPFS sanity tests to tester1.js
FossilOrigin-Name: 99915b0076422487cdd181a54e55694404fba13e4a540329b5ede9e2c9e12532
2022-10-21 06:26:17 +00:00
stephan a6ca996e4a Expose sqlite3_vfs_unregister() to WASM and unregister kvvfs in Worker threads to avoid its unintended use there (in contexts other than local/sessionStorage). Correct registration of window functions, extend oo1.DB.createFunction() to support window functions, and add window function tests to tester1.js. Correct an incorrect 1-arg handling case for DB.exec(). Add per-test assertion counts to tester1.js.
FossilOrigin-Name: f07ce15479b7224b0d1ba9f147a433136e70c1461aa667d2737d4a918f778f55
2022-10-21 05:27:40 +00:00
stephan 6f3286cafd Make semantics for UDF xFinal() result handling and error reporting handling more flexible.
FossilOrigin-Name: 89f3e1982ec32c010af67d15ef780847df20de568669e5c9d02f3cf084f51330
2022-10-20 23:48:38 +00:00
stephan 96b6371d70 Add more JS tests. Flesh out the aggregate UDF tests to use sqlite3_aggregate_context() so that they can each be used multiple times in the same statement. Add sqlite3_js_aggregate_context() convenience helper.
FossilOrigin-Name: 9d034ef5e1bab7c9651c2450dc85765fa6365d3f1414c711550de858ff8b3ece
2022-10-20 21:28:31 +00:00
stephan 875b95d4f5 Apply magic.txt correction and addition reported in [forum post 2d2366a04a0385|forum:2d2366a04a0385].
FossilOrigin-Name: 9bf26e2aa3579f354ed2d314e1bf3e3ef117cbd71500ef5f76caa1de5cce1edc
2022-10-20 18:58:14 +00:00
stephan 8a8244b5c1 Rename sqlite3_web_... to sqlite3_js_... Document the worker1.close.unlink option. Fix unlink usage in speedtest1.html. Minor JS build cleanups.
FossilOrigin-Name: ac9af71b72a749b0a040273a88480d56f49570b569389a4ea20cc055f494d8ff
2022-10-20 18:31:32 +00:00
drh c046f6d4d9 Add a bit to the optimization disabling mask for the indexed expression
optimization.

FossilOrigin-Name: 56df1482ab45d4d41f937b27c5a877b52fa082bfc6809ea682b62b47a51333de
2022-10-20 16:30:05 +00:00
dan 7e089d0c40 Add test case for previous commit.
FossilOrigin-Name: 535867a9c8719358380f89fd80473c484def9bb72744dd2fd28ada65999ee901
2022-10-20 16:22:39 +00:00
drh e180564028 Only enable the bMaybeNullRow flag on IndexedExpr for an index on an
expression, not on a virtual column.  But do enable it for the right operand
of a right join.

FossilOrigin-Name: a9657c87c53c19228a42559c82c54b504a5ad729e407e9e2c7dabcc0c949b261
2022-10-20 16:12:02 +00:00
drh 63b3a64ca6 The generalized indexed expression optimization of [2435112867fbd7b6] makes
the prior [a47efb7c8520a011] enhancement from 2017 obsolete.  This check-in
removes the older optimization.

FossilOrigin-Name: 56442c9bdd7844454c79acf83323380ea3509cb2e0b9a3d538a6b51726ae6d07
2022-10-20 13:36:32 +00:00
dan 9b6a28330b Fix minor problems in test1.c test code.
FossilOrigin-Name: 6e545e2b846795fc67f8a90c04de284861a8a429ac686a7476b41c8ed7aabf75
2022-10-20 10:46:17 +00:00
stephan 842c5ee849 Rework sqlite3_wasm_vfs_unlink(), add sqlite3_wasm_db_vfs(), update some docs.
FossilOrigin-Name: cdd46858f0e63bc7bfce8e339b3db9efdec43b6443ee76563a847f53d0176831
2022-10-20 05:14:37 +00:00
stephan d89a66ec36 Minor internal JS cleanups.
FossilOrigin-Name: 818ef0b5c909e733b643455278f7cc45533178f1cc6617058c00ed64fa44896a
2022-10-20 04:00:05 +00:00
drh bf70f1bd1a In the CLI, never use strcmp() or strncmp() directly. Instead use
wrapper functions cli_strcmp() and cli_strncmp() that work correctly even
if the input is a NULL pointer.
[forum:/forumpost/142b868da7560d0b|Forum post 142b868da7560d0b].

FossilOrigin-Name: 7450a561f8577c3ee41f84074c6ef39e29e56876cab4fd763e6489c66f603223
2022-10-19 18:04:42 +00:00
dan 361fb98b5d Move file /ext/misc/dbdata.c to the /ext/recover/ directory.
FossilOrigin-Name: f6d5ac807efcf6140016a8b8f0c74566c0bfc98ad92bd8fca03c5cc80ebf3dec
2022-10-19 18:03:39 +00:00
drh 3b88065d35 Fix references to "SrcList_item" in comments as that object has since
[bfd5bf2c73110fcb] (2021-02-21) been called "SrcItem".  Comment changes
only.  No changes to code.

FossilOrigin-Name: e3648a07f5607dbd4d6ad5b6e5a62a4a1d0b173d3f22aa36c74b768281fc86d8
2022-10-19 11:58:24 +00:00
drh 805055f734 If a query uses an index where one or more of the columns of the index is
an expression or a virtual column, and if the corresponding expression is
used elsewhere in the query, then strive to read the value of the expression
out of the index, rather than recomputing it.  This is the
"Indexed Expression Optimizations".

FossilOrigin-Name: 2435112867fbd7b6ebb7f2c2b9da57cdf1e23fab6c2869870b66133a9f9faedc
2022-10-19 10:56:05 +00:00
stephan a817b30f0b Update the wasm/js 'push' rule to account for this morning's changes.
FossilOrigin-Name: c4a0f745fbaf6f950810f92765cbe66dde66e226b78ec5dd9017473bf4c9c268
2022-10-19 07:51:12 +00:00
stephan fd31ae3bf9 Rename several demo/test files and include more of them in the end-user dist archive.
FossilOrigin-Name: 9c85835f6f50eb3b1a2b89c817816335743f04440c48bfa05aa89ec519cc0d51
2022-10-19 07:34:36 +00:00
stephan 9dc4d5e6b4 Minor wasm/js build tweaks.
FossilOrigin-Name: bcbee1ec010ae74f0749aa7fda008698198a8cd52a7aa12d6458d43d3e739eb0
2022-10-19 06:14:24 +00:00
stephan 3117dc60e3 Update the dist rules for this evening's js/wasm build changes and include an index.html specifically for the subset of apps included in the dist archive.
FossilOrigin-Name: a0ef0f32e96638b502b3951e524d590bdfb09dd39be453686b075102da67b7db
2022-10-19 06:06:42 +00:00
stephan cd0df83c15 Apply considerable acrobatics to get the JS/WASM deliverables building to and loadable from a directory other than the one which contains the app-level code. Requires an only-slightly-leaky abstraction of passing a URL argument when loading sqlite3.js but provides much greater flexibility in where the JS/WASM files are located.
FossilOrigin-Name: 6d468dab9eb84d4548f68014959f02fe4f66455472ff24fe729382bb2972e3d1
2022-10-19 04:44:58 +00:00
stephan 71de8e0241 Considerable wasm/js build cleanups and reworking. Remove wasmfs builds from the end-user deliverables and disable the wasmfs build by default, per /chat discussion, as it doubles our deliverable count for only marginal gain. Attempt to move the sqlite3.js/wasm files into subdirectories but rediscovered that that breaks loading in Worker mode because URI resolution of the wasm files differs depending on whether the main script is loaded from a script tag or a Worker.
FossilOrigin-Name: 5b23e0675efdd2f1ea7b4f5836a579e8d6aa8a25b3f1a6a950520ad845ff01bb
2022-10-19 01:07:30 +00:00
drh 72ab366192 If a virtual column whose value is constant gets indexed, do not include
that column in the IndexedExpr list.
dbsqlfuzz 59ac17a99b9a5c4930ee71cc153c9428a0c0e794

FossilOrigin-Name: c21eb903f06a16d4fafa5f895be08414fa58e6a91c148030aec8676b40c2d66d
2022-10-18 22:37:07 +00:00
stephan b5e2e6fcd3 More work on the JS end-user deliverables. Add tool/stripccomments.c to support that.
FossilOrigin-Name: 2156f0744acfe425457430a0f6a7e02de907de85edba81a6d4eef40293e561c8
2022-10-18 20:36:50 +00:00
drh 7a2a8cee5a Improved byte-code comments for the OP_Column opcodes used by the
indexed expression optimization.

FossilOrigin-Name: bf6d837fddbf74d5b3f40ad01ea4564edfac00b651d6f3f0e996cc5dc18bb06c
2022-10-18 20:27:02 +00:00
drh a331cf7e57 Fix the indexed expression resolver for generated columns so that it picks
the correct table in a self-join.

FossilOrigin-Name: 54c3eb085aca92da2b819f3532ee4750b377265f7e1e914600ea3c3d535efe51
2022-10-18 16:47:41 +00:00
drh 9489e0a5d7 Merge recent trunk fixes into the index-expr-opt branch.
FossilOrigin-Name: 6198ab4b871c464448ae303072bde0a810cac566a516cc29b1bf806fbd0a6c7b
2022-10-18 16:32:22 +00:00
dan ac79fa1c78 Add new test file windowE.test, to test the window functions modules response to an inconsistent collation sequence.
FossilOrigin-Name: 740a2eb0928d54fdf735a8e573c6a61a34dbd295b46e5b6ef39e957fd2623293
2022-10-18 15:02:08 +00:00
drh 0e522c068a Correct sort order for serial-type 10 entries in the database file. This
is a continuation of [4fb77e96fa89a23a].

FossilOrigin-Name: 904b54625d985e742888e06ba792cab316b9ec9d6669d9cf509bac48030373ca
2022-10-18 13:27:31 +00:00
drh 451393c4f4 Merge trunk fixes into the index-expr-opt branch.
FossilOrigin-Name: c7b9cc645b394a1b638a9a13ca83d321215aaa78d69ca70c6baa0ee692ed21bf
2022-10-18 11:28:33 +00:00
drh 43fce6bb0d Fix sqlite3VdbeRecordCompareWithSkip() so that it sorts the internal-use
serial-type of 10 together with NULLs.
dbsqlfuzz 5ff35e9d49a5fcca5051e23960ff2f483a538bab

FossilOrigin-Name: 4fb77e96fa89a23a9365320c4190834edd6c09cfaf1ca30b34ce19b747ebbec0
2022-10-18 10:27:06 +00:00
stephan ce152a4369 Add ext/wasm/README-dist.txt, intended to have been in the previous checkin.
FossilOrigin-Name: c9080b6505b839cc0bb5bd25ce7df0daa91a0a303d037ffe6e77888c8584b932
2022-10-17 17:36:11 +00:00
stephan e79a0943ed Initial draft of archive generation for the wasm/js deliverables.
FossilOrigin-Name: f6cee114c71e11ca1f508c8f60be5753871cbd4fd6c45ce397824bddbf00ffd9
2022-10-17 17:34:41 +00:00
stephan 1b8c246799 Repair version-info.c after a local mishap caused all of the intended edits from [36e197cb3d0c]/[e9b407a4d0a0] to get deleted before checkin. Simplify version-info build (only requires sqlite3.h, not the library).
FossilOrigin-Name: 6e511df500c63e27f932be9d601b641144562614dead99371477eca4594fb3fd
2022-10-17 16:47:59 +00:00
drh 79b99b09fa Merge trunk changes into the index-expr-opt branch.
FossilOrigin-Name: a1485ce6c8374135290eb0a1e4eb4e225497ad7096847a4e2377c9c054946ca7
2022-10-17 16:31:55 +00:00
drh 8be8924901 Fix a potential call to strlen() with a null argument in the command-line shell
following an OOM error. [forum:/forumpost/9c4f2ebe22|forum post 9c4f2ebe22].

FossilOrigin-Name: b6413a6dff8ac9b7088b1381afbbbf799e376455d11786530cc5fc825747ab53
2022-10-17 16:09:33 +00:00
stephan cdc9ddc6ec Missing file-rename for the previous checkin.
FossilOrigin-Name: e9b407a4d0a0fcd9f26d457beede19d41141327befe84be621681952be5d1e69
2022-10-17 15:53:16 +00:00
stephan 0467b01a1f Rework ext/wasm/version-info.c to make it more useful for the upcoming dist build rules.
FossilOrigin-Name: 36e197cb3d0cbb76e4c1381d77f7d8856e7391f2ef6671c13b9b6b0133315814
2022-10-17 15:52:12 +00:00
drh e70d458340 Improved comments and function names. No logic changes.
FossilOrigin-Name: 8f460b3b15bf4b88a224907d2f83e4ed6bf84d6b3f480008a1b187ea49c6ce1d
2022-10-17 14:46:39 +00:00
drh 08535840c6 Get the indexed expression optimization working for virtual generated columns.
FossilOrigin-Name: 8a510cb0880349370ad19b0852f270ca072bff7871f0dc272964af31c32a7c37
2022-10-17 14:30:01 +00:00
drh 543c368958 Merge trunk enhancements into index-expr-opt branch.
FossilOrigin-Name: 56ef97e64efb84176df570819474901d492209ff80ce4355ff896dea9d2b8a51
2022-10-17 14:29:04 +00:00
drh ce4b0fdf8a Make use of alloca() in wherePathSolver() when SQLITE_USE_ALLOCA is defined.
FossilOrigin-Name: 45752471521f8d4a6cad4bce40f907018acd403ffd3b8170020c34e4e0b6b91b
2022-10-17 10:15:41 +00:00
drh 135c9ff6f7 Remove a use of alloca() that does more harm than good.
FossilOrigin-Name: 2ab3cf4d71b23a4c0b69f977828d82df7bbc080c29ae189e0d78d38efee50c96
2022-10-17 09:56:51 +00:00
stephan e493f8f0cd Typo fix.
FossilOrigin-Name: 650126c92aa191d6361a32ec90ca7def60a2df22ba6bec1d2354692fe3504423
2022-10-16 23:33:24 +00:00
stephan 2676fc58c8 Minor tweak to get bring an edge case of oo1.DB.createFunction() into aggreement with its docs.
FossilOrigin-Name: 8d59ccd9f357fbed0b0790508ca9027194fc4b0131029f9f460c7b4bde806a5f
2022-10-16 18:57:15 +00:00
stephan 824bb5b8db Add aggregate function support to sqlite3.oo1.DB.createFunction(). Change signature of the options object used by that function so that the callback property names match those of the corresponding C APIs.
FossilOrigin-Name: a7db6e4b50beebfb1c97e0c4de49538d8199c166b18a0b1b175736c593128a00
2022-10-16 18:50:55 +00:00
stephan 8ffc98999d JS: add build-time-generated version info to the sqlite3.version object. Remove some stray debug output from tester1.js.
FossilOrigin-Name: b5f462c2d85d503f6492ec20580d57cb4c926712f6306a6be764bd09d1f5e8b8
2022-10-16 16:38:15 +00:00
stephan 956392694a Add a top-level license and build-time version info header to generated sqlite3*.js. Correct a broken link in ext/wasm/index.html.
FossilOrigin-Name: 0f1a06e8e39a1fbc74f1aff9cc59787282dfbf847d6c5c7edb3f7d410db0e4b7
2022-10-16 15:38:03 +00:00
dan 8e62d370cf Exclude slow test recovercorrupt.test from veryquick.test.
FossilOrigin-Name: ff9a733532fe11bb253246b45a94681a174c976aafee5c780af571893e8ce694
2022-10-15 19:18:10 +00:00
dan f0799fdf6d Add missing comments and fix other code issues.
FossilOrigin-Name: 8ed4e4a80d8104d20b211a0b3e299abdb955cabdd36979bf6aae54266dc9778d
2022-10-15 18:26:30 +00:00
dan 9dffd36c5d Merge latest trunk changes.
FossilOrigin-Name: 42255ead8e44b24a8abc04aca49f170c55f3b52f42366a0d8ef248ee65fcc7ce
2022-10-15 15:39:46 +00:00
drh f3adb7c430 Enable the index-on-expression optimization even when the expression is
used as an argument to an aggregate function.

FossilOrigin-Name: 462b3c7f39724dc814f55e7a225e7d0c48f81c524cdda797a66e9e198c35ce58
2022-10-15 12:01:40 +00:00
drh 7a98937db2 Only extract an expression from an index when the index is not a null row
in an outer join.

FossilOrigin-Name: 08b033c737d1a84859291f50e2985c9dad8d660a50185d55d3171165a8e08d4c
2022-10-15 11:27:01 +00:00
drh 260ac72722 Merge fixes from trunk.
FossilOrigin-Name: 1cb65f36c3539302767f551ed53082f054666a7cb2696cef0990ab6747edbc52
2022-10-14 19:56:58 +00:00
dan 879164ed74 Fix a problem with using the push-down optimization on compound SELECTs where component SELECTs use different collation sequences. dbsqlfuzz 11516f050100243e5a845f5a2b48a90ed2efaf2e.
FossilOrigin-Name: ed14863dd72e35fa3a23320c3d5a8166515faea39a555c28a27b2d35e701eac4
2022-10-14 19:30:34 +00:00
drh 0388f17988 Add missing initializer from the extension loader. Fix for
check-in [d6d449978245b4fa].

FossilOrigin-Name: 565d74c36f2dd4612860c7ee561682c50db28cfa5384346528292019dbfdf86c
2022-10-14 19:21:12 +00:00
stephan e72ddfd89b Generic minor cleanups and docs in the OPFS async proxy.
FossilOrigin-Name: a4423ca234453c14eb40db7fe5943f63b30fd9dc2207388e8a2966733a004e9d
2022-10-14 15:52:29 +00:00
dan 3fa10f82f4 Handle recovering databases with reserved bytes at the end of each page.
FossilOrigin-Name: 52d0235ef3f21683daadf2343514f7b95a67c3589266f75c1e00e04f11dc6214
2022-10-14 15:46:19 +00:00
dan 2bc4a6cad0 Fix a problem in the LIKE and GLOB operators that may occur when the character immediately following a "%" or "*" wildcard is U+80. Reported by [forum:61bf7ccbdf].
FossilOrigin-Name: 2da677c45b643482eec39e4db7079c772760bc966dc71bf6c01658cc468f5823
2022-10-14 15:10:36 +00:00
stephan 3e771c0f07 Remove a resolved TODO comment.
FossilOrigin-Name: 32fd4ac382f774189ac34f6fff80e55e6e56dd2aa67b0db88d5a88324f17f6ff
2022-10-14 13:26:18 +00:00
drh 859434d049 Cancel IndexExpr objects when they go out of scope.
FossilOrigin-Name: 0963519371bb7ac02adb323430855760fb9b82a23745e47c504aaf393d22ac34
2022-10-14 02:00:54 +00:00
drh 4bc1cc1847 This experimental branch attempts to use columns for an index-on-expression
in place of the expression that is being indexed.  This particular check-in
mostly works, but there are still issues.

FossilOrigin-Name: 2e8d4fd4cfd9e82f33c707ba246fe2bb3ca01762cf5ac5905058fbc7adf0abe7
2022-10-13 21:08:34 +00:00
dan 249efd04d7 Add tests for detecting page size of databases.
FossilOrigin-Name: 31f9e2369fcf59032b7c4c9f5bfc85e7ef7c174003b0b9e2757dad5a4c79b370
2022-10-13 20:06:17 +00:00
stephan d92c652ac1 Move the rest of testing1.js into tester1.js and eliminate the dependency on jaccwabyt_test.c. Extend the list of default config-related #defines in sqlite3-wasm.c and reorganize them for maintainability.
FossilOrigin-Name: 4e2a8aff2dd4b6e148f45184e2523ebe47815257eca97fa3d32bcbf9625f0def
2022-10-13 16:48:35 +00:00
drh 921acff927 Optimize the IS NULL and IS NOT NULL operators so that they avoid loading
large strings or blobs off of disk if all it needs to know is whether or
not the string or blob is NULL.

FossilOrigin-Name: cb94350185f555c333b628ee846c47bcc9df5f76bb82de569b8322f30dbbe1bc
2022-10-13 15:09:44 +00:00
drh eddfa9840e Improvements to the description of the OPFLAG_TYPEOFARG option to OP_Column.
FossilOrigin-Name: 5e9c67ba18b701aabbb0546acdfc532c9e8f0d27fb0a2c899415a5c47096c90b
2022-10-13 14:54:32 +00:00
drh bd1c634546 Adjust the implementation of sqlite3VdbeTypeofColumn() to make it easier
to test (and slightly smaller).

FossilOrigin-Name: 79fdd021df256c6a2b9973fd6bf8e52ffaaf4d369d40489b8395ac97880b1325
2022-10-13 14:35:20 +00:00
drh 7ca4af6a9f Fix a typo in the documentation of the OP_Column opcode.
[forum:/forumpost/a2b5bd6d43|Forum post a2b5bd6d43].

FossilOrigin-Name: 043e76e6166da5cf8e213cce46aaccb1f910e1fdbdb5556576eafb81b3bc5faa
2022-10-13 14:01:11 +00:00
drh e995d2c226 Proposed optimization to the IS NULL and NOT NULL operators that avoids
loading the entire content of larges strings and BLOBs.  Response to
[forum:/info/3c08d4715dc05b00|forum post 3c08d4715dc05b00].

FossilOrigin-Name: 45f171565442f9fd6574fb93ae7abe83c168b20be68af42531bc55571450d3ab
2022-10-13 12:47:33 +00:00
stephan d85d08398d Port the first 180-odd unit tests from testing1.* into the new tester1.*. Fix a stray-keystroke-induced typo which broke pstack.allocChunks().
FossilOrigin-Name: ef689e33e464829f5cbe4ca24a53d9dba59abe74d3d80a37a91b93a4eccccf2d
2022-10-13 08:03:31 +00:00
drh 1ffb6be119 Updates to the fuzzer query invariant checker - tracking changes made
over in dbsqlfuzz.

FossilOrigin-Name: 4ca16a304ad10fbb48f78b4384b347fe883e1a4f222f113ac981e89845c3e113
2022-10-12 18:40:25 +00:00
drh 131aa13a2a Changes to Makefile.msc to fix the build on Windows following the previous
merge.

FossilOrigin-Name: 368fa6b25bc803ded7c1a0184615980902657879370caec22ceea42496ec0566
2022-10-12 18:30:08 +00:00
stephan c9e992a197 Merge fiddle-opfs branch into trunk.
FossilOrigin-Name: 98f954942a8d4824b5c4bd1d27cfeff7a71a029dc5a54220b76d59e367996581
2022-10-12 18:13:52 +00:00
stephan 7be51f5897 Merge trunk into fiddle-opfs as a preliminary step for a clean merge in the other direction.
FossilOrigin-Name: 5e1848ce384a78fe5dfa1b5c1b2acb9c2fac34a8f0ba6c652e0541dcf24e9027
2022-10-12 16:35:22 +00:00
stephan 402da26c16 Add initial infrastructure for setting up function/regression tests for the JS/WASM APIs.
FossilOrigin-Name: 7f5db9829b6e60fadb756fea5442da1f4368c8428bb5cdaf14f97f0c3c8451d9
2022-10-12 15:54:28 +00:00
stephan b75971e6e9 Minor doc cleanups and corrections in sqlite3-wasm.c
FossilOrigin-Name: 5144c122a921e4240901cf4eb46347b92213273eec7cf0977952ab2b60722c27
2022-10-12 15:40:16 +00:00
stephan 181f7e869d Correct mismatched H1 tags in test code. Minor CSS tweaks.
FossilOrigin-Name: 4d8eb90a370054d4482c20637ab56f5e01f4d10215f2af4e35fb9a1f85ecb700
2022-10-12 14:39:15 +00:00
drh 47996ea7c4 Add the sqlite3_value_encoding() interface.
FossilOrigin-Name: d6d449978245b4fa66c152132da468eea7977eab4d1fe53bb2fe3ef543d8030f
2022-10-12 12:49:29 +00:00
drh c2777abc2d Enhance the OP_IsType opcode so that it is slightly smaller and faster and
so that it works correctly with invalid serial-type codes 10 and 11.

FossilOrigin-Name: 846f863e0d55eb7e78c8d355bbbcd73e1946d0ba566c2cb31683cde5f4353d1b
2022-10-11 13:57:55 +00:00
drh 7d23d157bb Improved the ability of the CLI to handle very long input lines.
Potentially a fix for the bug reported by
[forum:/forumpost/fa4bb2941a|forum post fa4bb2941a].

FossilOrigin-Name: d0e107ee00101f42b4c9bf372625311d04b83c96a4a9caacac866ea03d8e7fa4
2022-10-11 12:02:42 +00:00
drh 7747ffba47 Enhance the "PRAGMA integrity_check" statement so that it verifies datatype
constraints on non-STRICT tables:  (1) Columns with TEXT affinity should not
contain numeric values, and (2) columns with NUMERIC affinity should not
contain TEXT values that could be converted into numbers.

FossilOrigin-Name: aa6e9086196248675b9e33dbfaad9ef67097432535896790f80d2470002978b1
2022-10-10 23:54:17 +00:00
drh c9ef12f62b Code clean-up for the integrity_check enhancement.
FossilOrigin-Name: a140173102febe9ef8064ee9b95bee489db54caba149e577d69e4d75161bf390
2022-10-10 21:21:04 +00:00
drh db6940ab45 Fix corner-case bugs in the new integrity_check logic. All tests pass now.
FossilOrigin-Name: dbab9d52a842085da67f5d0b8e96c665adc16331accf380b4c234e9b05bdb7fb
2022-10-10 19:38:01 +00:00
drh 49d77ee642 An attempt to enhance PRAGMA integrity check so that it does data type
checking on non-STRICT tables.  Specifically:  (1) Columns with TEXT affinity
should not contain numeric values, and (2) columns with numeric affinity should
not contain text values that can be converted to numeric.

FossilOrigin-Name: 8b1e7f0524637728cebe81c7d3ff8ad8a5a55782eac6409b425dad538024f596
2022-10-10 18:25:05 +00:00
drh 005c9d8295 Improved detection of database corrupting when moving pages on an
autovacuumed database when creating a new btree.
dbsqlfuzz 9a55397eae13cec64aebf1fd35489c3a90bdaac5

FossilOrigin-Name: 327965bc71023d9380a9d6805062659108dab4bfcd386b7aba813754f270d33a
2022-10-10 12:02:53 +00:00
stephan 50661c5e31 Remove some extraneous debug output.
FossilOrigin-Name: 56ff42053f7b73b380c103a84f29e285cfbfb6e18a25b3a50a6d78e32610914e
2022-10-09 17:08:36 +00:00
stephan d8897d971b Remove a now-unnecessary --cachesize flag for the OPFS sqlite3_vfs speedtest1-worker.html invocation.
FossilOrigin-Name: 08b516d6953256cfb8792cfee9e6ed7450fd55d84196b66b77409cfaee3c8729
2022-10-09 17:03:48 +00:00
stephan ff891b4e1c Minor cleaups in the post-init async phase of sqlite3 module initialization.
FossilOrigin-Name: 2e024a6b533524b7732cea8aa27b16363e17b4c32731ddf31085e91fc728c1af
2022-10-09 15:12:37 +00:00
stephan 392fd3ce37 Remove ext/wasm/kvvfs.make which was inadvertently resurrected in [92b500da70a3].
FossilOrigin-Name: 2e7fa085fc0b1202c6502d89e2603ae76a93451754b07b32ab4f0211f213f980
2022-10-09 13:35:14 +00:00
stephan 4fbf90ee11 Refactor kvvfs JS bits to make use of [ea370b9b05f7ed7eaa]. At main-thread startup, if kvvfs is available, replace the kvvfs I/O methods with JS impls. Checkin part 2 of 2, to account for cherrypicking [ea370b9b05f7ed7eaa] into the kv-vfs branch.
FossilOrigin-Name: a9047e020a097b2259bc9935b63ca1c538a3a7f1d050e15f0d0a08cfb84acc7c
2022-10-09 13:26:15 +00:00
stephan d35b5587df Refactor os_kv.c so that the kvvfs read/write/delete methods can be swapped out at runtime by JS implementations. This eliminates the kvvfs dependency on Emscripten. Checkin part 1 of 2, to account for cherrypicking.
FossilOrigin-Name: ea370b9b05f7ed7eaa154ba58019f6642217eabc18517e721567adf948d93980
2022-10-09 13:19:27 +00:00
stephan d6555726ed Add -DSQLITE_OS_KV_OPTIONAL to speedtest1 flags for kvvfs testing purposes.
FossilOrigin-Name: 92b500da70a3dc64e910c232a2cac7620b6609162c2a5058b269d3b83d763c02
2022-10-09 11:42:12 +00:00
drh 211a1a7281 When casting an odd-length BLOB into a TEXT with encoding UTF16, omit the
last byte.  dbsqlfuzz c5df68b5bbdf6c9f3c1851e41e88f49ac05c9969

FossilOrigin-Name: 1e2796b3741a4c0b966e6c01a7d62dea8de9d1f4bededea3d7ba7004dabc2ffe
2022-10-08 17:27:05 +00:00
dan 73c586bcbf Fix a problem causing the seek-scan optimization to skip over valid rows that could occur when it is used with expressions of the form (a IN (?,?..) AND b >= ?). dbsqlfuzz ab1db6dc0efb04cba1cd3431ee6da4894fdc4520.
FossilOrigin-Name: 63d9efe277759d4daa29794846b60c6f55491496618f423f61468df72d0a4633
2022-10-07 18:57:15 +00:00
drh 8a2254fa1f Improved tracing output for showing all registers used by the
OP_SeekGE opcode.  This applies to debugging builds only.

FossilOrigin-Name: 0aa6dee7f72279114a47cbb69fdbda0b916a2c365f7a147946869f167bd018b7
2022-10-07 15:55:35 +00:00
dan 88a0637f57 Ensure the page-size, auto-vacuum and other settings are copied into the recovered database.
FossilOrigin-Name: 078520f2825bcb82d01b30209409e9bf8b6c99084c192345df41c9f531d08f51
2022-10-06 21:00:23 +00:00
dan 4df12ce5a8 Add code to determine the database page-size by searching for well-formed pages.
FossilOrigin-Name: 0dbd0ccef532e076fa82c9fad4dc9fe632c0ef43e3483cc288cbb7eca336a9b9
2022-10-06 17:20:12 +00:00
dan 73ac958c74 Fix a problem that occurred when using a bloom filter to optimize an (ipk = ?) lookup in the case where the RHS of the expression is a TEXT value. First reported by [forum:/forumpost/f61a8b7053|forum post f61a8b7053].
FossilOrigin-Name: 8e14c351b29bb434ac4e2510681e95716424a73e340d39feff4919f0431c2e00
2022-10-06 14:10:11 +00:00
dan adc686043e Improve handling of corruption in the sqlite_schema table b-tree structure.
FossilOrigin-Name: 2be0dba12fb9b98cd9632f1ff3f30aeb29661a2e121b68c2f3335617b027797b
2022-10-05 16:58:28 +00:00
dan 461fa44be0 Merge latest trunk changes.
FossilOrigin-Name: b0feecaa84b88f534edb49056cf7e92256f4a687c8b4c44e1d1186709a304378
2022-10-04 19:47:17 +00:00
dan 041cc06139 Better handling for databases with corrupt headers.
FossilOrigin-Name: 17f68d803685405d880025134c71bac01077962d677f2902a608952d0b9cdb5a
2022-10-04 19:43:12 +00:00
stephan 9a55773b2f Replace time-based auto-unlock of opfs sync handles with lock acquisition/release via sqlite3_io_methods::xLock/xUnlock().
FossilOrigin-Name: 2625b7cfe1640c1d7e779ec1f37db970541598c0dc3e22e5eecf3c772d95ad40
2022-10-04 17:06:51 +00:00
drh 2eca06141f Attempt to clarify the operation of the xLock and xUnlock VFS I/O methods.
Assert() statements added to prove that they behave as the documentation says.

FossilOrigin-Name: 3efa811251d4510a074231a1e0b667783cc5f90777466b9dcae675cd9892b423
2022-10-04 14:50:46 +00:00
stephan ed3182f233 Tweaks to the opfs async wait/relinquish times.
FossilOrigin-Name: 35f33c23e5849de1c43c4499ee0a7fa11d26ae34949c1e820c3fa8e8873f9c2b
2022-10-04 11:14:23 +00:00
drh a4b2f41938 Change the name of the autoconf-generated configuration file from "config.h"
to "sqlite_cfg.h" to work around a bug in a dependency checker, reported
in [https://twitter.com/josecastillo/status/1577136108097314816|a tweet from Jose Castillo].

FossilOrigin-Name: 897e187cad981126fc5f6ea8e382e937825af10b0c9589b0a23b9fbaa4ca58b8
2022-10-04 10:35:10 +00:00
stephan 98e120fca0 Update ext/wasm/api/README.md to account for recent changes.
FossilOrigin-Name: 9d488081fc82a9abe3e81c7897fe6b29b6b337f62d0c62cb9cec7517bd54b53f
2022-10-04 09:12:43 +00:00
stephan c7fb48d4b6 OPFS async proxy: add a wait-and-retry policy to the get-sync-handle step to help account for cross-tab locking.
FossilOrigin-Name: 45c48c63d311052105d102189208495b2b53fa3c4174884ecf63b757aa4016e5
2022-10-04 09:12:05 +00:00
stephan f6b6188bdd Update the 'start in speedtest kvvfs' link in speedtest1-worker to account for recent changes.
FossilOrigin-Name: f4d5ba412e0c1a031430a32c9ed1e11a09af33f9883150e660cc05e49fe56e6a
2022-10-04 08:58:22 +00:00
stephan 2716cd2528 Minor pstack doc update.
FossilOrigin-Name: 6b4fb7c4e399ecc948ad41f5fa3139a9293cc822cd7efa2b83dc5fb1660b3ec6
2022-10-04 01:21:34 +00:00
stephan 2b776ee2d6 Minor cleanups and additions in sqlite3.capi.wasm.pstack.
FossilOrigin-Name: 97bd670d3859bd3f2e62c15065b238b85ef614f26479e56a14be42c5784aedf4
2022-10-04 01:11:10 +00:00
stephan 3c272ba380 Add a test/debug mechanism to shut down the OPFS async listener so that it can be inspected (it normally can't be because its tight event-listening loop ties up the thread) and then restarted.
FossilOrigin-Name: 7d0bcff4e9b899cd25b393b9f0a02c5dcee2e229f0a0fa01719c7dcd7dcbe7c1
2022-10-04 00:54:00 +00:00
stephan 88838f6b95 Set default page cache size to 16mb in wasm builds. Fix an off-by-one counter in sqlite3_wasm_enum_json(). Minor coding style conformance tweaks.
FossilOrigin-Name: 72a9e589cc318ec50941739e2edf2c0636284b316c3bf87b71fd363b37619da3
2022-10-03 23:13:05 +00:00
stephan faf6a07965 Add a --cachesize flag to the speedtest1-worker-opfs link in index.html because opfs is much faster with that.
FossilOrigin-Name: 5b8f8e33afc47c9f0c64dbe34643d7d1c37a0a82afb4656e714a86ef54a9ce6f
2022-10-03 22:51:24 +00:00
stephan 8d9e595513 Minor JS doc updates and typo fixes.
FossilOrigin-Name: 3cfcc14dfd220536141aeffb902fdc8db1cea055b2a0609b88e092fc3df94688
2022-10-03 22:38:00 +00:00
stephan d234902b7c Add configurable cache_size to batch-runner and speedtest1-worker. Add SQL tracing to demo-123.js just for demonstration's sake.
FossilOrigin-Name: 2ab065133bec19f71a1dc382946fdbd020920b2e38a526f4126a3a1e8df5de87
2022-10-03 18:07:25 +00:00
stephan 4b074692f4 Omit WAL mode from the wasm build.
FossilOrigin-Name: 88efe2a62b81e5ee9119f54204e397fda59666a327158c7b7c972e84a716869e
2022-10-03 18:05:27 +00:00
drh 16292a5e52 Adjust the --help output for speedtest1 to clarify that the argument to
the --cachesize option is in pages, not bytes.

FossilOrigin-Name: d7c2be6bdc3402e5a339d0f7b76d2c25f3a444dcbf54a77c1ccb790d1195a7cf
2022-10-03 15:22:59 +00:00
drh 277c7d3516 Copy all of the fiddle-opfs changes to speedtest1 into trunk.
FossilOrigin-Name: f3d31c9982cab517a3907a28c571225cee614ee78375c70eb2efb9a2b6b4b042
2022-10-03 13:56:52 +00:00
stephan 57956a9c86 Resolve/remove a TODO.
FossilOrigin-Name: 2ea35fea3bac2637e935643edfa7913b792e529de3a8980dbb8678d98ed5bcf9
2022-10-03 13:46:20 +00:00
drh 3044e77b4a Add the --vfs option to speedtest1.
FossilOrigin-Name: aeb884029ac1c68f0d7361dda0a545cff19ed265b608c1e329a1926397c5c8a3
2022-10-03 13:45:03 +00:00
stephan 4f5bbedb3a Export sqlite3_trace_v2() to wasm and use it to ensure that the new per-VFS post-open SQL support in the DB ctor works. Default opfs vfs to journal_mode=truncate, as it's faster in that mode. Add 't' DB open-mode flag to enable SQL tracing to console.log().
FossilOrigin-Name: 508f7f6d63e52f61fae5abe817579a4e130fa7fbd18733d741d521a5bdabb7ce
2022-10-03 13:03:41 +00:00
stephan a4c357f94c Reimplement fiddle_reset_db() so that it works with all VFSes.
FossilOrigin-Name: 18462052903bfeb037dd9af3a1fe1898d141eff5d6f527721858120553de808d
2022-10-03 11:42:45 +00:00
stephan 5f0b67cb7a Experimentally relinquish the OPFS VFS sync access handle when the db is idle and reacquire it on demand, the goal being to help alleviate cross-tab locking issues.
FossilOrigin-Name: 2703ac9842335962e488e597168d70b1389b95a6ad39edf70a211b95979b4708
2022-10-03 11:33:35 +00:00
stephan 33ded6edaf Add journal=MODE to the list of supported URL flags for speedtest1-worker.html.
FossilOrigin-Name: 5c43e8d2ec23f28fdce63874d9d4c8ccbb4a8f81030b19d65ff5711a7c32697e
2022-10-03 11:23:47 +00:00
stephan 7ff8da876d OPFS VFS now lazily opens its sync access handle, as a step towards experimenting with relinquishing it during idle times to help avoid cross-tab and page-reload locking issues.
FossilOrigin-Name: a984e1ba435731413a541f86c50232bc7d6e33aff6ba4cca90f89188e7b82a2c
2022-10-03 09:21:37 +00:00
stephan bdfd7ea03a Partial revert of [a82e6faaa642] to remove the 'I' alias for 'j' wasm function signature letter. In hindsight, that change seems premature.
FossilOrigin-Name: dcd46af9141f4edf816010923941a76d0edd3f18cfe429c52f599ad2a0d52651
2022-10-03 08:30:22 +00:00
stephan ef9cd12ec4 Minor JS API tweaks prompted by documenting them.
FossilOrigin-Name: a82e6faaa642b09d241232c4daa67134d4dfa24bf3ca3725740346ca5269b381
2022-10-03 08:21:06 +00:00
stephan 510a9d1c64 More cleanups in the UDF argument and result handling, in particular int64. Consolidate some duplicate int64/bigint range checking code. Expose the UDF low-level utilities (arg/result conversion) to client code. Add the sqlite3_context pointer to the JS-side UDF wrappers for API consistency.
FossilOrigin-Name: 10ab77af952abf09f93f342a9d07a3b133f2c4c0a3588df3390cd3a923cafae4
2022-10-02 22:50:04 +00:00
stephan 65e6522f8f Minor doc typo fix in sqlite.h.in.
FossilOrigin-Name: 372802aaa22a830bf498bb0d9561ae02b32e5e191c4d7d771ea152140f6685a1
2022-10-02 20:13:46 +00:00
stephan 92ede964e1 JS: clean up create_function() wrapper and add support for create_window_function(). Eliminate an extraneous blob copy when a UDF returns a blob. Make use of newfound JS-fu to clean up how sqlite3ApiBootstrap() config is initialized.
FossilOrigin-Name: d3bad9347c5423fa7f19ae729461636f1043c99a1f01f168efa10bebefb1cdd1
2022-10-02 20:08:53 +00:00
stephan 9892883e9e js: implement a hand-written wrapper for sqlite3_create_function_v2() which converts, if necessary, JS-function-type args to WASM function wrappers. Replace DB.createFunction() impl with the new one.
FossilOrigin-Name: 435ab33384017967e46f52b70bee851a85a28808990a0e58dd5288f606b89c9c
2022-10-02 18:47:39 +00:00
stephan 9fdce595eb Doc typo fixes.
FossilOrigin-Name: e528675da3971907666c7d2d09763975a105ec585dc5122145f65905d535bed8
2022-10-02 03:14:38 +00:00
stephan 63e9ec2f9c More fleshing out of sqlite3.capi.wasm.pstack.
FossilOrigin-Name: eb5726677a727a958df11f1fba078d30c7c0ba2a9bdb158e8641b35b5f971af3
2022-10-02 03:11:13 +00:00
stephan 6479c5a359 Document the roles of the new (this past week) JS files added to the build process.
FossilOrigin-Name: 8b3bc7313aff551e5ee0b7aeb927095cf19b9b96abbdd922066c130656b8aa7d
2022-10-02 01:48:14 +00:00
stephan 193ee11fe1 General internal cleanups in the oo1 API.
FossilOrigin-Name: f9db664f756f3707afcb5dce87f6d946625848f27ea84337af68de72d4ad6c6b
2022-10-02 00:09:40 +00:00
drh 7d34b8d628 Performance optimization in sqlite3_prepare() (discovered while working
on the stmt-cache branch).

FossilOrigin-Name: cd494240cc6184bb028d1f1049086f57c0a71a43d7ed1405fd4205240e1b9e61
2022-10-01 20:27:29 +00:00
stephan f064a3bbe4 Correct fiddle db export breakage caused by a post-testing API change made in [1fa019c88dea].
FossilOrigin-Name: e73cc44ec36c7585ebb914bdeed5b39480fca7c7a8d3c4426bfe769c87b98d17
2022-10-01 19:21:52 +00:00
stephan f446af57b5 Pedantic constness tweak.
FossilOrigin-Name: c8a173cf16d2567bf9b13f52904077a2e7209776c76613c7bff59cd66e24bf11
2022-10-01 18:55:10 +00:00
stephan 3afad4d432 wasm: correct a memleak caused by a shadowed var in the previous checkin. Add a stack-like allocator, sqlite3.capi.wasm.pstack, as a faster way of managing short-lived pointers (like the one which got shadowed).
FossilOrigin-Name: 1fa019c88deac6b6e5155b620bdab1d7145846290daafb9adbefcf4f0fe542cf
2022-10-01 18:47:42 +00:00
stephan 32781427d7 Fiddle: fix makefile dependency issue and duplicate inclusion of post-js.js. Reimplement db export using sqlite3_serialize().
FossilOrigin-Name: 29db7de79232c21d19b91bb0fc253114e02e21dd9bf90619453dbe72a4c8bf7f
2022-10-01 16:01:41 +00:00
stephan be9d203665 Merge trunk into fiddle-opfs branch.
FossilOrigin-Name: 64ebcbe41615a6d7776597564105ea7638e4a9095a764ea558c2620640429cf8
2022-10-01 13:45:14 +00:00
stephan c9099d2d73 Tweak sqlite3-worker1.js to be able to load either sqlite3.js or sqlite3-wasmfs.js, noting that the latter still does not load in a Worker because of an Emscripten loader bug.
FossilOrigin-Name: 000ef7059bfb54dc4f829b81a8d8c927c0382980218d8a3d60cd2c4d89151c90
2022-10-01 13:38:27 +00:00
drh 7424aeff41 Replace the Parse.disableVtab field with Parse.prepFlags for increased
generality, a small size reduction, and a small performance increase.

FossilOrigin-Name: b7da0bcdf70e53ab1ec00a0694e17c7429e23bc6eb3f39b622d06a930aa2f6a3
2022-10-01 13:17:53 +00:00
stephan 6167d5cf17 Note two wasm-related potential TODOs.
FossilOrigin-Name: 5636e82864457d870754ee7125c307dc5d2195197a5c0266579da9f102938b89
2022-10-01 12:04:49 +00:00
stephan 8c1a4b89af Remove a couple of weird quirks of the Worker1 and Promiser APIs. The Worker1 (now undocumented) unlink capability needs to be reevaluated to work equivalently for all storage backends.
FossilOrigin-Name: 9f2b331a32cbaadfd20d04c9908171355322c1aa1d9d6df3628f3b2fb4391ec1
2022-09-30 23:49:43 +00:00
stephan ae589b69db Tweaks to the Worker1 and Promiser APIs prompted by documenting them.
FossilOrigin-Name: c68b9aa160e2c1197ae7eb06a634017ac2b281393074afa4582762d5458c6889
2022-09-30 23:02:11 +00:00
drh 7bc6a818f0 Remove one of the NEVER() macros added by [5166acf3f0040459] because it
is reachable after all.

FossilOrigin-Name: bd8ae5febbb8d3122f1e2e09b9fcdd6ac20c075ae1cf60bcdbd36080f3736417
2022-09-30 22:40:57 +00:00
drh 1337a2291d New test cases in fuzzdata8.db
FossilOrigin-Name: aa367a3c974041323edb759855f0b83803439ceef2846da2d73eadd80efeaac4
2022-09-30 20:59:09 +00:00
drh 2d3261fe86 Fix a bug in the demoDelete fix from [d76a301e22e05722].
FossilOrigin-Name: 7e761a02530ffbb055a766eb6c995f11fc2adbabb9a52502124f672fdcff0d10
2022-09-30 20:53:36 +00:00
drh 0a803bd984 Add NEVER() to two error-detection tests that were made unreachable by
the previous check-in.

FossilOrigin-Name: 5166acf3f00404597e77910aee88920ab833af8447766f50bf4f70525606cf95
2022-09-30 20:48:22 +00:00
stephan e67a0f40e4 Add JS wrapper for sqlite3_exec() which knows how to handle a JS callback. Add some console.error() reporting of module-load failures, as they otherwise often get silently swallowed up by the loader's mechanisms. Add 'flexible-string' JS-to-WASM argument converter which performs more X-to-string conversions than the 'string' arg converter does.
FossilOrigin-Name: 96818aa83f4ccc574f558231249ecbdd39763b4351cf4cf6d33f53774a3ee5e6
2022-09-30 20:35:37 +00:00
dan ad658b2d82 Better handle a case of database corruption where a b-tree page is one of its own ancestor pages.
FossilOrigin-Name: a24e46804893a7e6e8f3130abca882f9a3aa3b7b2b82aa39a0a20856172766d5
2022-09-30 20:15:21 +00:00
stephan d18f1bbfe0 Add sqlite3.version object. Add more state to the Worker #1 config-get response, including sqlite3.version.
FossilOrigin-Name: 711f458d188a0dbe6612069c856ade29323ab426dfa4f80e7b82757ccc474cb8
2022-09-30 16:49:03 +00:00
stephan 07c0b72253 Add oo1.JsStorageDb.clearStorage/storageSize() methods, copies of capi.sqlite3_web_kvvfs_clear/size().
FossilOrigin-Name: 1e09efe7fa15b8908f8b8353164a8361de778e27ea6c0b11c402bf4e1c56333d
2022-09-30 15:46:08 +00:00
stephan 359d62395e batch-runner.js: force WebSQL batches to not abort for a failed statement (necessary for apples-to-apples-ish benchmark comparisons).
FossilOrigin-Name: 60f0c5cb04d2baf8431e523434d8753e39377f4b2c6bad225c2e5487a4be419b
2022-09-30 15:24:58 +00:00
drh cb57716c65 Improvements to the "demoDelete" method in the demovfs, suggested by
[forum:/forumpost/cad18961c2|forum post cad18961c2].

FossilOrigin-Name: d76a301e22e0572262917dfe9557a5051339a65416a473dd2d9989a49975403e
2022-09-30 14:35:18 +00:00
drh e62a79d8d9 Merge the latest trunk enhancements into the begin-concurrent-pnu-wal2 branch.
FossilOrigin-Name: 68a61513f9e064ed9e79638ec40f8bff1a8ee678793683e2725a4ce63563db6a
2022-09-30 14:04:15 +00:00
drh 73b1ef33ef Merge recent trunk enhancements into the wal2 branch.
FossilOrigin-Name: c22c7c879846b1357c00741700b7f6d1111c4b94895b4e7f552c92299d35712e
2022-09-30 13:54:27 +00:00
stephan dc6ae60226 Expose sqlite3_msize() to wasm.
FossilOrigin-Name: aa6ad34f3b693b554be58ee2895f79397162c5b8877c709653c8508fc625e8f4
2022-09-30 12:10:36 +00:00
stephan f6c686c9f4 Add oo1.JsStorageDb() as a convenience wrapper for oo1.DB(...,'kvvfs'). Minor doc cleanups.
FossilOrigin-Name: 8a7998709f859a562cf6829485cb9921f8823af0efabe003741348ab1169fb89
2022-09-30 11:01:44 +00:00
stephan 53d4e01d06 wasm: expose sqlite3_de/serialize(), sqlite3_malloc/free() and friends, noting that the former explicitly lies on use of the latter for memory management so is not generically safe for use in wasm.
FossilOrigin-Name: fbc0edb5d31aa0dea92460e853f15f08c642451a7878994116b530cf172325cc
2022-09-30 10:55:28 +00:00
stephan f71c954cbc Add JS infrastructure to ostensibly allow us to customize the wasm imports, which will hypothetically allow us to eliminate the dependency on EM_JS(), but the corresponding Emscripten glue-level feature currently breaks fatally with WASMFS builds so it's disabled.
FossilOrigin-Name: 88d9253b0db5494bf1c9b6d24f22524eeec856b89e64a66ffb30d945f0df21d3
2022-09-29 22:08:22 +00:00
stephan b0ccf50cbd Ensure that the wasm builds always, even when passing -Ox flags from the CLI, use -g3 to avoid problems caused by symbol minification with optimization levels -O2 and higher.
FossilOrigin-Name: 27d24d5d270e272af07d96cf07bdbe79ba07a9c69ba3c70beb76ebf4f90fc97a
2022-09-29 17:35:25 +00:00
stephan eb97743cc7 Finish eliminating explicit Emscripten module dependencies in test code and fiddle. The only remnant in public code is the Emscripten-generated module load/init interface.
FossilOrigin-Name: 7be78dd4efc410f13ff1ceda1fad82b309cc24de2d5106c8bd6b2adeaa13b106
2022-09-29 16:54:23 +00:00
stephan b94a98607a Rework the Emscripten-emitted module loader/init function such that it passes on the sqlite3 module, instead of the Emscripten module, to the first then() of sqlite3InitModule()'s returned Promise. This eliminates any need to mention the Emscripten module object in client-side code unless they want to configure it in advance for loading-status reports.
FossilOrigin-Name: 0dbaa0e2b5abf5c23e2039ec90a3055ebb3c063aaf4e556c42546defe6fbb86d
2022-09-29 13:17:50 +00:00
drh 31474ce57e Fix misuse of the sqlite3_set_auxdata() interface in the ICU extension.
FossilOrigin-Name: f25cf63471cbed1edb27591e57fead62550d4046dbdcb61312288f0f6f24c646
2022-09-28 19:14:01 +00:00
stephan 4b884bb4c7 Wasm: expose sqlite3_exec() and use it to simplify the db-reset logic in batch-runner.js a bit.
FossilOrigin-Name: 2e2821f782511b9d2274a89a5a922582aba18c7e9dc7ce01080e713942a56d7d
2022-09-28 18:10:50 +00:00
stephan cb22bd80d2 More work on batch-runner.html/js to facilitate speed comparisons between various VFSes and WebSQL.
FossilOrigin-Name: 3bd1bc240676e56da87fc49f3c67a1edc4fafcf2a2416298d19ae4f80b676a72
2022-09-28 17:52:52 +00:00
stephan 5ad3631915 Correct duplicate copies of sqlite3-api.js being embedded in the wasmfs-based builds.
FossilOrigin-Name: bbfcfba260f39a9c91e82d87e06b1c2cb297c03498b4530aa3e7e01ef9916012
2022-09-28 13:01:49 +00:00
stephan d98011852d Work around broken -Os wasm builds by adding the -g3 flag. Unrelated documentation tweaks.
FossilOrigin-Name: f5d6bf8616341037fa3e229edf820d19acef3e0a6207a652b2b143de0a493214
2022-09-28 07:53:47 +00:00
stephan 489a8a9d3c Minor doc correction.
FossilOrigin-Name: 093f6e4b36db294e8e47df2fc75a4bc4fc101e2b6ff0201c912ccd1dcf394479
2022-09-27 17:03:01 +00:00
drh 5d48e0cdc5 Provide the SQLITE_MAX_ALLOCATION_SIZE compile-time option for limiting the
maximum memory allocation size.

FossilOrigin-Name: 584de6a996c78b8e41bdfcd05a8e2a3844664c6b4efedb5883c8b8af388462b5
2022-09-27 16:35:06 +00:00
stephan 6559e0ab15 Comment the -O2 wasm builds as being broken (mysteriously missing JS bits).
FossilOrigin-Name: f61475ef8244fd1f6553eca9cc2317e5de2cc04443e24afb7b91ebd3fc1bd402
2022-09-27 14:31:34 +00:00
stephan 5b9973d898 WASM API renaming. Reworked JS API bootstrap's async post-init into a generic mechanism, no longer OPFS-specific.
FossilOrigin-Name: c42a8cb090cad1108dfd6be574202d744c59e053b505bc4c17252dc6b65d26bf
2022-09-27 13:40:12 +00:00
stephan 3d64548491 wasm/js: rename /persistent to /opfs to account for potential future persistent storage options. Minor flag-handling cleanups in the speedtest1 pages. Minor API tweaks in oo1.
FossilOrigin-Name: 4dc972a3656b2a9ec915bfb3f653136560c753ce4024c3f0d0d0c28f66db7a0a
2022-09-27 09:17:37 +00:00
drh 6319a8003c Make the legacy FTS3 code more robust against integer overflow during
memory allocation.

FossilOrigin-Name: 5517bc50988b5339c2fd071b29de1b5ca03037b0b635c3b112cf7108fab54d5f
2022-09-27 01:53:05 +00:00
drh 80cf891792 Enhance defensive mode so that it disallows CREATE TRIGGER statements if the
statements within the trigger attempt to write on a shadow table.

FossilOrigin-Name: 3283bbd12a60f472ed03cb7c6209a605a3bf9f3d9083371e17840b56e8b3f559
2022-09-27 00:56:45 +00:00
dan 7e33f8f43b Modify implementation of recovery to break recovery of the lost_and_found table into multiple steps.
FossilOrigin-Name: adedfd040bb2d1c2dcda0b91c6073f12953e45d9b860a8f217d990e861d66dfb
2022-09-26 20:51:16 +00:00
stephan 278d3faf1f Fiddle: replace db export routine with a C-side one which works for both Emscripten FS-hosted and OPFS-hosted db files. Minor code-adjacent cleanups.
FossilOrigin-Name: 3579a8d6f1f6cd3cd8aad9949536870c5fe7bae8c1778f700dd85d763e266b94
2022-09-26 13:55:10 +00:00
stephan 1f095d482d Get fiddle db export working for OPFS VFS. Add root dir handle to the main OPFS VFS worker to enable creation of certain utility functions without delegating to the async worker. Add sqlite3.capi.sqlite3_wasm_rc_str() to map integer result codes back to their SQLITE_xxx counterparts. Minor doc touchups.
FossilOrigin-Name: 9b2244e1c8a40efe6547094a1b57acc8f2173145a8731abb0e36268ce0a8ef41
2022-09-26 11:38:58 +00:00
stephan 6a643e4bcd wasm: change StructBinder signature for sqlite3_file::pMethods from 'P' to 'p' to eliminate an unnecessary and inconsistent level of magic.
FossilOrigin-Name: 85f2e877e53899860af4dc6630044b471a9c7c82faba1f4e1e60ae991460b943
2022-09-26 11:34:31 +00:00
dan 8d8fc58413 Add the SQLITE_RECOVER_SLOWINDEXES option, for specifying that indexes should be created and populated along with tables, instead of separately at the end of the recovery operation.
FossilOrigin-Name: ad9dba9d1eae786575c7f31e34b342b6f5b26e719bbe27b61609cad8cfd0a505
2022-09-24 19:54:49 +00:00
dan 351563efe1 Add sqlite3_recover_step() to header file sqlite3recover.h. Update the API docs in this file.
FossilOrigin-Name: 47f416153035d6bf0ae27587583d2957c87c02caecbd4e7f363bcb2bc27cf159
2022-09-24 19:17:20 +00:00
dan aee4049128 Update things to use sqlite3_recover_step() internally.
FossilOrigin-Name: f4b15aad3005237b7ac507eed2b9b07e0f5c9407ab28f2656a21c9845f13d35f
2022-09-24 18:05:52 +00:00
stephan 395012e58e Fiddle: move, rather than copy, data buffers between the threads. Fix the case of an uploaded db failing to install because its filename is the same as the opened db.
FossilOrigin-Name: bcec4f964a7b02f59be05286ff715bac654a32b19f05a743e402f4cdb207cab8
2022-09-24 11:32:00 +00:00
stephan ef11fb915d Resolve "already configured" warnings from shell's main() when starting it up in fiddle mode.
FossilOrigin-Name: 114ef3552af977b272a0baddeb9a2326484b60acfc75284e43c55530f86b413f
2022-09-24 10:15:08 +00:00
stephan 56fae744d4 Reworked out the OPFS async proxy metrics are fetched so that they play more nicely with the tight event-polling loop.
FossilOrigin-Name: ef503ced5c2ca842be9aea9ef13719a378ed3020e884032db09afee1b8eba0a1
2022-09-24 10:12:19 +00:00
stephan 60d9aa7c59 Refactoring towards getting fiddle to support OPFS as a first-class citizen. Certain operations, e.g. import, export, and unlink, are not OPFS-aware.
FossilOrigin-Name: 1b923ed6438d7fef4508936e0c4bc026a368721698b1539961e3fb3140a185cb
2022-09-24 07:36:45 +00:00
dan 9bbd91b39d Internal changes to the recover extension towards a step() style interface.
FossilOrigin-Name: 73926d5c8cd1ecece134b5a73b44ee1dfca74dc200606e3f009b06cdecf8cee9
2022-09-23 21:01:10 +00:00
dan 80b1f6fcde Fix various compiler warnings in new code on this branch.
FossilOrigin-Name: ae49e9efde3012158061def6e0a8d993abbc5933514a21f84bc10f700f61b5e2
2022-09-23 11:40:43 +00:00
dan bdbd097280 Merge latest trunk changes into this branch.
FossilOrigin-Name: e87fa70ab0f9b95bbcde18567f47906a222a3fd02b4f3c2903d2bb087d361b7c
2022-09-23 11:30:24 +00:00
stephan f75c0c7036 Reformulate some JS to work around a buggy/broken code transformation in one of the Emscripten-driven code optimizers.
FossilOrigin-Name: e1249369d5ec1c582c280b1f578b35d53637fdf1cbd97c16d5ed95b136b83e56
2022-09-21 20:24:12 +00:00
stephan 6110a5d0ab Put pieces in place for fiddle to support opfs, but more cleanup is required in the sqlite3.js/fiddle connection. bigIntEnabled now defaults to whether the Emscripten's module has bigint enabled. Add wasm-sensible defaults for several SQLITE_ENABLE/OMIT flags in sqlite3-wasm.c.
FossilOrigin-Name: 7c7fb7535e86b3960eea7f29ab7e6d5197c166b4ee64ad4a9bc0749f2869badc
2022-09-21 19:51:25 +00:00
drh f333370e1f Reduced and improved comments on the constraintCompatibleWithOuterJoin()
function.

FossilOrigin-Name: 38aaf26e082bd95df6b64df43e1772fe6e20c4eb71307dcd97559cac7f11f8f1
2022-09-21 18:21:31 +00:00
stephan 65022716b3 shell.c.in: when building in fiddle mode, define _POSIX_SOURCE (ifndef) so that emcc's string.h reveals strdup().
FossilOrigin-Name: fb85b269c43147f153977606dd8ede7b93744bf955e4c1a2b198907fd8a94620
2022-09-21 16:21:21 +00:00
stephan e8afca3f16 Correct mistyped --shrink-memory flag in speedtest1-worker. Minor OPFS proxy cleanups.
FossilOrigin-Name: 86e2b55ec9483fa5add51a479c6509d73461f1ac6fca5d49e057b1c66f4314d2
2022-09-21 14:02:47 +00:00
stephan 72ab400d4d Doc cleanups and additions. Add a way for the OPFS async runner to propagate exception text to the calling thread.
FossilOrigin-Name: 5c5e80652825cf883e6c17809cb98f2bf17d5feac2d263f6f492479154730dab
2022-09-21 12:27:35 +00:00
stephan 171b168b3e Added some structure to the wasm demo link list.
FossilOrigin-Name: 777077c4c2249e1ec78390d4f65aaf281c1fbefcef4bcc7609199e995645ceb6
2022-09-21 12:25:40 +00:00
drh 2897b88f9e The RIGHT JOIN logic added to version 3.39.0 added a new restriction on the
query flattener which resulted in a performance regression for some queries,
as documented by [forum:/forumpost/96b9e5709cf47cda|forum post 96b9e5709cf47cda].
This change removes that restriction (thus restoring performance) and fixes the
problem that RIGHT JOIN was having in a different way.

FossilOrigin-Name: 501609eddf2a46d51e8cd56477a22d4df142a166e78fe4322c0db11d1eed6687
2022-09-21 09:54:41 +00:00
stephan ebe1423098 Add a link to fiddle.html in the wasm test app index.html.
FossilOrigin-Name: 96c734c07acfbea153d9aaf293a28a2d49d78de19fec4fb90c4c08fb1fd0ddbd
2022-09-21 08:41:40 +00:00
stephan 4bc2f6b415 Move fiddle build rules into the wasm-centric build files. Add rule to push wasm bits to the wasm test server.
FossilOrigin-Name: 113f8204dc4ac849d5632d3de1680b6e0da871e107ef484c8d7273799bee3d88
2022-09-21 08:39:03 +00:00
drh faaacd3fbb Break out the decision of whether or not a constraint term is usable by
an outer join into a subroutine: constraintCompatibleWithOuterJoin().

FossilOrigin-Name: c7fbc9b0453050e2746af27e3a11e0c3701bef8c56da8e19173242c6ea3aff8b
2022-09-21 00:16:59 +00:00
drh 0f51aa3972 Do not allow EP_InnerON terms to be used on a outer join.
FossilOrigin-Name: f47aa745690c018800243bf76930b2499ff4537411c4e27b4b16ba3854cc2bf8
2022-09-20 22:19:13 +00:00
drh 0cc6f78bcf Add new test cases that (current) get an incorrect answer.
FossilOrigin-Name: fe5c37736444e397ace387b761ec2491229785d7a437f3f60232aedf9d1f00ec
2022-09-20 21:07:52 +00:00
drh 4c5c941b6e New test cases for query flattening when there are ON clauses and outer joins.
FossilOrigin-Name: db55387b1972bc7bc6855bd3497af744a111e7d28d09893cd985fbb01be41bcc
2022-09-20 19:45:04 +00:00
drh 8b9a3d1fc8 Modify the OP_IfNotOpen opcode so that the jump is taken if the cursor is open
but was previously set to a NULL row using OP_NullRow.

FossilOrigin-Name: 1292d68caa7086610ddda343f3852e63de8da1eb66536ee4716b6529f5a31bc6
2022-09-20 19:22:17 +00:00
drh 5a0771a1f4 Remove code that has been previously commented out.
FossilOrigin-Name: 852b385a5de622aa32026824210d4bd23db52a4a8a697b83d22d6000edeba487
2022-09-20 18:18:00 +00:00
drh a5f8c42c98 Merge the fuzzcheck fix from trunk into the flattener improvement branch.
FossilOrigin-Name: 930a8c1561d0d5dc70dc5378edac77f79ac59ab6569b561de3dfb02f160c38c5
2022-09-20 17:43:40 +00:00
drh 703c208d30 Update the fuzz invariant checker to conform to the latest changes in
dbsqlfuzz.

FossilOrigin-Name: 8d8c124349d2a25200ace9a60d76751594b722468ad0c39d927a5d2c6e2464e0
2022-09-20 17:21:54 +00:00
drh 40c9becc81 Is the query flattener restriction 29 (do not allow both EP_InnerON and
EP_OuterON constraints on the same FROM clause term) really necessary?   This
branch explores what happens if we omit that restriction.  This is an effort
to address the performance regression reported by
[forum:/forumpost/96b9e5709cf47cda|forum post 96b9e5709cf47cda] that is caused
by flattener restriction 29.

FossilOrigin-Name: 19270577ad1d08c9e6e5f0600396b8a1ccda9df1e67a0df845aee6b60437f379
2022-09-20 16:57:49 +00:00
stephan 4cffb644c7 Correct a too-strict is-opfs-available check.
FossilOrigin-Name: 1b5f1b4a6c1457f98c258459e23e321fc59793de298fecb84031b87f02156cd5
2022-09-20 16:20:35 +00:00
stephan fa5aac7475 Merge kv-vfs-magic-names branch into fiddle-opfs branch and make some kvvfs-relevant tweaks.
FossilOrigin-Name: e3d36dcdd37e59f17a07d3611d08744eb86f439fab82a648490dd608bcaa3185
2022-09-20 16:10:39 +00:00
stephan 89071030db Export the sqlite3_uri_...() family of functions to wasm.
FossilOrigin-Name: 72bebc848fce5c3b4766017d016ccb2360de2bd0cb3e47e710c80dbcb6b8b707
2022-09-20 14:52:26 +00:00
stephan fea03bc0ca Merge opfs-proxy-atomics branch into the fiddle-opfs branch.
FossilOrigin-Name: 25a36920d44544547a84161681cd41e292b4a70df60ac3630791873a79237d98
2022-09-20 14:39:54 +00:00
drh c3b6fdaead When compiled with SQLITE_OS_KV_OPTIONAL, the magic names ":localStorage:"
and ":sessionStorage:" are recognized and converted to use the kv-vfs.

FossilOrigin-Name: c5db9262d0388ccb0e84c6a4b4e2e786dd634f13874e4034ba7b175befa4ce90
2022-09-20 14:36:53 +00:00
stephan 0e0aafc698 Remove an obsolete file.
FossilOrigin-Name: 1e43855498b4329f733bd6e0731841dc884e885de7d2d338d402d05d54309427
2022-09-20 14:21:58 +00:00
stephan b8c8d4e4a0 Speed up de/serialization of func args and return values in the OPFS VFS proxy.
FossilOrigin-Name: 5bf235bbe035e4ace7a54851e190742528af6b4266328a1b8bbb9fb3dd7f2118
2022-09-20 13:25:39 +00:00
stephan c2ccd67689 OPFS proxy: remove one sanity-checking-only level of proxy function to shave off a few microseconds.
FossilOrigin-Name: b534831f3efb8910a17e29956e3e87cc80055ea66e15dbef992b6a556ff042f8
2022-09-20 10:47:36 +00:00
stephan c9e2602e8f Remove extraneous result-code SAB slots. Doc additions.
FossilOrigin-Name: 88de20712bbe3e11ea74af95edc34e9ab9f346f9aa0a30931e5a9e1f96ce57b0
2022-09-20 10:11:52 +00:00
stephan 5e8bb0aa13 An alternative messaging strategy for the OPFS VFS proxy which uses only SharedArrayBuffer and Atomics, instead of worker messages, for communication (only the initial one-time handshake during initialization uses worker messages). It runs speedtest1 approx. 15-20% faster but still 20-ish% slower than WASMFS.
FossilOrigin-Name: a83ee3082d89439ea3ad5737e63e25bebb0f91895aca006ce5fecf5b93a2651a
2022-09-20 08:27:57 +00:00
stephan 138647a552 OPFS VFS: further internal refactoring towards experimenting with a new comms model.
FossilOrigin-Name: 5ca412ced24b4e3af5f467e710a597ed440badf7b8335346aade11d3cad3d1a1
2022-09-20 03:31:02 +00:00
stephan c4b87be3e8 OPFS VFS: moved i/o buffer from per-file to the VFS, and related refactoring, in prep for experimentation with a new inter-worker comms model.
FossilOrigin-Name: d4d63e4580ad8d497310608175308c03c517e051d7865cb66aa0b10356612d7d
2022-09-20 01:28:47 +00:00
drh b22b493338 Clarify comments that describe query flattener restrictions associated with
outer joins.  No code changes.

FossilOrigin-Name: 2647d73bb1876d01851f52cb3fd6af08a962ab336b6d51c160d27b1baf94f75f
2022-09-19 19:25:15 +00:00
stephan aec046a264 Further metrics and buffer-copy optimizations in the OPFS proxy, but with little effect.
FossilOrigin-Name: d1f1fe6f1c60640f7770dfb9245c459a09b8d24ec2ddf664dff77c810bd51f96
2022-09-19 18:22:29 +00:00
drh 316cd40122 Fix a minor comment typo. No changes to code or documentation.
FossilOrigin-Name: 8032db4ade66f70a102abb9d3a49e20620e204c45ad7a954a49d20403985bf07
2022-09-19 17:41:59 +00:00
stephan f815011a08 Cut the speedtest1 runtime of the OPFS VFS proxy by approximately 3/4ths via xRead/xWrite buffer-copying optimizations. Still slower than the WASMFS impl by approx. 1/5th.
FossilOrigin-Name: fb7f287310d74a3e236125ae9c49b859f9263c29ae85161c1bcf9dd0778d8a51
2022-09-19 17:09:09 +00:00
stephan 61418d5a67 Remove an obsolete file. Update wasmfs build for today's changes. Fix the case that OPFS.xTruncate() is passed a BigInt instead of a Number.
FossilOrigin-Name: 976053925013bf1975f5f9222e28ba648af28e305bb6bdae600eb24d0e136bec
2022-09-19 14:56:13 +00:00
stephan 0e0687ccfc Add sqlite3_web_vfs_list() to JS API. Corrected OPFS VFS's registering itself as the default VFS. speedtest1-worker now uses the xDelete() of both the default VFS and OPFS, to avoid that it starts up with a persistent OPFS test db (the native app calls unlink(), but that unlink call operates on a different virtual filesystem than the OPFS VFS).
FossilOrigin-Name: 2ec7e09139a510b9fd29e4c97283b20740a00f369193c6fecbb734f187e81b48
2022-09-19 13:44:23 +00:00
stephan b5ae85eca2 Merge kv-vfs branch into fiddle-opfs. Adjust various JS APIs and apps to deal with the new method of handling kvvfs. Adjust speedtest1 JS build to include sqlite3-api.js so that it can use kvvfs and opfs VFSes. Permit passing of the vfs as a URL parameter to certain demo/test apps. Milestone: speedtest-worker.html?vfs=opfs runs with the standalone OPFS impl.
FossilOrigin-Name: ec09f32f7ae2249aaf27388ad2062982afa8bbbb5f88d236d6d9068bf33ad93d
2022-09-19 13:16:35 +00:00
stephan 5ef7f04775 speedtest1: add --vfs NAME flag and eliminate code duplication in argc size verification for all flags which take a value.
FossilOrigin-Name: 281d09867134e0a057cfadd9bfcbf0e21b8ac9737a278c41973d33a3101be7bc
2022-09-19 11:47:38 +00:00
stephan ba0b157583 Minor doc improvements in demo-123.js.
FossilOrigin-Name: 9370ce94780e974da03d5c74cf9a5983eed333e7b3bc54e605c5f30b3fa99287
2022-09-19 10:13:01 +00:00
stephan 509f40526e Generic cleanups in the OPFS VFS proxies.
FossilOrigin-Name: 7d5f6adc3b964413fc96ad8d2735312c3e58348024cabdd2099682cbf696eaf7
2022-09-19 09:58:01 +00:00
stephan 862281fc47 Correct OPFS VFS xRead() to copy the result buffer if the result code is SQLITE_IOERR_SHORT_READ.
FossilOrigin-Name: 56668f9902c6e896b6c63621a444444c6f58ee20f88a5feae97f1699be35892d
2022-09-19 09:25:25 +00:00
stephan 8766fd20d1 Replace OPFS VFS xSleep() impl with a more efficient one (no Worker round-trip needed).
FossilOrigin-Name: b9773f164878b0a1b7c88cc7a6d1374ea95f64920065e8b2b178a1afffd82fe5
2022-09-19 05:19:04 +00:00
stephan ac51eb7754 Rename demo-oo1.* to demo-123.* and add demo-123-worker.html, which runs the same demo via a Worker. Doc typo fixes.
FossilOrigin-Name: 2e4a005bd35424caeaa99ace23162cf79e2ebdb159475ffad92b85dc864ad764
2022-09-19 03:57:31 +00:00
stephan ed85333977 Disable build of wasmfs-using components on aarch64 (ARM), as wasmfs won't build there.
FossilOrigin-Name: 9a9eeebc2c27b734041089131b4952d7c0440df48ef32f355641aca61d4b30a0
2022-09-19 00:40:53 +00:00
drh bcdb4cc71b Remove a NEVER macro in defragmentPage() that dbsqlfuzz discovered can be
true.  crash-32d9312f145cdce41613573f6431d9a3e439e3d7

FossilOrigin-Name: 90403759e1cf30f7c11fa8170d6df59a0dd138e1de59d2b60bec7ba15d74f67c
2022-09-18 17:59:28 +00:00
stephan f386012069 Numerous cleanups in the JS bits. Removed some now-defunct wasm test files. Expose sqlite3.opfs object containing various OPFS-specific utilities.
FossilOrigin-Name: 26e625d05d9820033b23536f18ad3ddc59ed712ad507d4b0c7fe88abd15d2be8
2022-09-18 17:32:35 +00:00
stephan 0db3089576 Move the OPFS VFS bits back into api/sqlite3-api-opfs.js. Refactor the OPFS VFS init process to use a Promise-returning function which the client must call, as that eliminates any uncertainty about when the VFS (necessarily activated asynchronously) actually becomes available to the client. Rename x-sync-async.* to test-opfs-vfs.* Milestone: first successful test of OPFS without WASMFS.
FossilOrigin-Name: b2abf60dbfa6648f671a3932cb65feb28d05a0d5b7f792351d14f9c13d9798c5
2022-09-18 03:05:55 +00:00
stephan c5313afea7 Move the OPFS VFS bits back into api/sqlite3-api-opfs.js. Refactor the OPFS VFS init process to use a Promise-returning function which the client must call, as that eliminates any uncertainty about when the VFS (necessarily activated asynchronously) actually becomes available to the client.
FossilOrigin-Name: 1c660970d0f62bcfd6e698a72b050d99972a1e39f45a5ac24194a190f8f78ab3
2022-09-18 02:35:30 +00:00
stephan 3e2823cbb6 Plug OPFS methods in to their sqlite3_vfs/io_methods counterparts. Add URL args to control debug output and running of sanity-checks in the OPFS init code.
FossilOrigin-Name: a0e93ed20b2463606a63b03ce8ca41ec1fb22886db5c5c898ace86ba24636f70
2022-09-18 00:16:12 +00:00
stephan 5f4ad92681 Correct OPFS xRead() and xWrite() impls and add a very basic sanity test for them.
FossilOrigin-Name: cd06cc670029763955cf60ffcf944b36d41cb005b859d9b9fd0eea1b6741d0e9
2022-09-17 23:47:20 +00:00
stephan 8200a6d8d7 Implement OPFS xAccess(), albeit with more limited semantics than the VFS API calls for. Add a way for OPFS xDelete() to optionally recursively remove empty dirs left over after deleting a file.
FossilOrigin-Name: c342b5d745f301104c59851c753287ebbbe95a69a56cb522d376d0f3e352c30f
2022-09-17 23:29:27 +00:00
stephan e6f8a09532 Generic cleanups in the OPFS sync/async proxy.
FossilOrigin-Name: f36bddbe54c3acbfaa958042e4d24724f130bdca551401033f9bc63f3da73492
2022-09-17 21:13:26 +00:00
stephan 0731554629 Add the remaining vfs/io_methods wrappers to the OPFS sync/async proxy, but most are not yet tested.
FossilOrigin-Name: 44db9132145b3072488ea91db53f6c06be74544beccad5fd07efd22c0f03dc04
2022-09-17 20:50:12 +00:00
drh a50d3b7a5d Fix an uninitialized variable in the decoder kv-vfs.
FossilOrigin-Name: 354726aa6c399053785f29104de15091629ce4bc275b9e2205cb3656a9e81cd7
2022-09-17 18:31:31 +00:00
drh 20a9ed1dc6 Include the kv-vfs as an optional VFS on unix builds if the
SQLITE_OS_KV_OPTIONAL compile-time option is present.

FossilOrigin-Name: 852812d1e2ec3c53ad7c6c64662b37d861fefcf1baeee3d58eba88bcb3f6d8df
2022-09-17 18:29:49 +00:00
drh b9fbc558f7 Merge the latest trunk enhancements into the kv-vfs branch.
FossilOrigin-Name: 37c7ca25a6e837e7f83b96a32e2b1a455730e48a48012c458ae2e734a3ccdd1f
2022-09-17 17:26:44 +00:00
stephan 132a87baa3 Add initial bits of an experimental async-impl-via-synchronous-interface proxy intended to marshal OPFS via sqlite3_vfs API.
FossilOrigin-Name: 38da059b472415da52f57de7332fbeb8a91e3add1f4be3ff9c1924b52672f77c
2022-09-17 15:08:22 +00:00
stephan ce260f4e43 Merge kv-vfs branch into fiddle-opfs branch. Adjust speedtest1 --size flags to account for new size limit.
FossilOrigin-Name: afb79050e635f3c698e51f06c346cbf23b096cfda7d0f1d8e68514ea0c25b7b7
2022-09-16 20:16:50 +00:00
drh 756440febc Corrections to the xRead method for databases in the os_kv.c VFS so that it
correctly reads the header, and thus avoids unnecessary cache flushes.  This
changes also includes an optimization to the header read logic.

FossilOrigin-Name: ef54961ce69fddb4cfeeff0860288de2858a6f7a5aa396691e8e99933eb9af54
2022-09-16 15:19:04 +00:00
stephan 81439a07f0 When calculating kvvfs sessionStorage/localStorage size, multiply it by 2 to account for JS using 16-bit characters.
FossilOrigin-Name: 52d1b185b9f6cee1eb3dec436f47e0f52e4621a127abfad8c27f92fd78147889
2022-09-16 12:49:45 +00:00
stephan 4c72171b60 Merge kv-vfs branch into fiddle-opfs branch for [21915af560b1|synchronous=off fix]. Remove some duplicate debug output in OPFS test code.
FossilOrigin-Name: 13899bb98c80525276d2484598b94e4206358f243f06d45c02700024f7e226fd
2022-09-16 11:45:09 +00:00
drh 84e5076787 Fix os_kv.c so that it uses SQLITE_FCNTL_SYNC and hence
works even with PRAGMA synchronous=OFF.

FossilOrigin-Name: 21915af560b111aeeaee751790356151a5f063c2fc703dd4b35b22dc393409fb
2022-09-16 11:37:01 +00:00
stephan 57db2174d5 Add batch-runner-kvvfs.html, a kvvfs-specific build of batch-runner.html. Reduce the speedtest1 --size X value for the batch list generation to 5 so that the kvvfs batch runner can handle it.
FossilOrigin-Name: d8df25920a047d5cf2093cc6233128c5e6057a9104d0c4397e643645bd646fe1
2022-09-16 02:30:49 +00:00
stephan 58bcfe075d Remove --nosync flag from speedtest1-kvvfs.html, as that seems to be the trigger for the 'no such table: z1' error. That app now warns if that flag is passed to it via URL args.
FossilOrigin-Name: ad0677e8abcc077636d1cf1d8485be4943506382581edf832e6b8a2021560040
2022-09-16 02:28:03 +00:00
stephan d22cfa8c80 Pull the src/os_kv.c part of [13839759f8f4] into the kv-vfs branch.
FossilOrigin-Name: e334449912d5176e355d02024a07ed867741f71c9d10ce6744ca800414bf3eeb
2022-09-16 01:08:06 +00:00
stephan 0144a8482a Add return value checks to kvstorageWrite(), necessary for recognizing storage-full errors in the JS binding. speedtest1-kvvfs.html runs successfully on localStorage with --size 1 to 6 and starts failing with storage shortage at --size 7.
FossilOrigin-Name: 13839759f8f45e4eb0aa6a5052801f3964d5b2dc2e427e5e91ee6692176381eb
2022-09-16 01:05:19 +00:00
stephan 28ef9bddb3 More work on the synchronous OPFS experimentation. Numerous wasm/js build tweaks. Add speeedtest-wasmfs.html, the wasmfs/opfs counterpart of speedtest1.html.
FossilOrigin-Name: 00ee49a3a2c148480f614e49a0ee5b35a255758c0a53693f0b464b31e7a4045b
2022-09-15 06:42:41 +00:00
dan 73390f09c2 Fix windows builds of recover extension.
FossilOrigin-Name: abcbb6abfe08fc590123f0aa1bd72cfdecc75908f078f1348dc0e957de98bf52
2022-09-15 03:54:52 +00:00
stephan dae7518ae4 Remove some stale comments. Get scratchpad-opfs-worker2.js running again in prep for reuse in experimenting with the OPFS all-synchronous API.
FossilOrigin-Name: 4cbbd370186f84e440f496e7f11c641d7dc1723acc46e4a31483392e0eb046e9
2022-09-15 03:16:49 +00:00
stephan 0117693f62 Split wasmfs-enabled build of sqlite3.js/wasm into sqlite3-wasmfs.js/wasm, as enabling wasmfs breaks all tests/demos which run from a Worker thread.
FossilOrigin-Name: 08476f3c218d45846e7496bdae0b06e2122466111fdf2aa2aabb1805b15ef982
2022-09-15 03:09:00 +00:00
stephan 409505c743 Clean up some JS documentation.
FossilOrigin-Name: 925fdbfc6c1f9f06951346bc0d4fb81ffb5797d5f1123728931973f6cb1a0efa
2022-09-15 02:27:48 +00:00
dan 6fa0a11c7b Include recovery extension in configure/make builds of testfixture.
FossilOrigin-Name: d2f4652144c4a54012a2176c178c5ade3463bd017515f472cfa8dafdde192df3
2022-09-14 20:45:46 +00:00
dan 3c24342841 Extra tests and fixes.
FossilOrigin-Name: deed5336931b23bb507c064d08e9899b33f04e4f7eee03beb8d7147eb4caa030
2022-09-14 18:57:46 +00:00
dan b8eaf9a10d Fix recovery of utf-16 databases.
FossilOrigin-Name: 5b05be0861f35804270fbd184ad4b89c23e98cc2fbd56b9e4fe6197daef5fe49
2022-09-14 16:37:59 +00:00
dan 322967df59 Add OOM tests for the recovery extension.
FossilOrigin-Name: 9b6b4c7162439a889144edb561356afc66436db921a867c20871f0c556716502
2022-09-13 20:40:57 +00:00
dan a3a6d63b40 Fix a broken error handling case in the recovery extension.
FossilOrigin-Name: 60089547e1fc77ecc02f207ebf75ee3160e5ff25f41d12e02e170fd7fde66602
2022-09-13 19:47:01 +00:00
stephan 5b91500796 Add/apply various kvvfs-specific utility APIs to the JS layer to assist in testing and analysis. Correct a backwards default arg check for sqlite3ApiBootstrap(). Add exports for sqlite3_db_handle(), sqlite3_file_control(), and the SQLITE_FCNTL_xxx enum.
FossilOrigin-Name: 0d78961870ee9f22f1ba16d423377d28dcc36e04b1e31ffd57f3e2fd51f8f0f2
2022-09-13 19:27:03 +00:00
dan 7985e05a97 Add tests. Deal issues surrounding aborting recovery from within the SQL callback, and avoiding the pending-byte page.
FossilOrigin-Name: 4e97dd31f5240d9231167ae172a5116426c42177a1ed3c5422b9d51b762d5a87
2022-09-13 18:08:24 +00:00
stephan 23d39ce54d Get testing1.js working with a kvvfs build.
FossilOrigin-Name: 333a45725d1708e0fefa559c33ce1c7eeb425cdb04f594b1f2b48166c1478c79
2022-09-12 22:27:00 +00:00
stephan 5fd8f27bfb Remove an extraneous key-copy op in the EM_JS impl of kvstorageDelete().
FossilOrigin-Name: 1c5aeee45564c14e7e2e7730f3f52106339ea3148fb5aa786fa767a35db46f51
2022-09-12 20:21:34 +00:00
stephan 0b769f5b4a Merge kv-vfs branch into fiddle-opfs and add speedtest1-kvvfs.html test app (which currently fails early on).
FossilOrigin-Name: 8b1608a9a37131121cb3a0cb1e8dbad5e39dc1fc2a341d056f09537f179b2e7a
2022-09-12 20:18:28 +00:00
dan 5ca0b38605 Fix some problems with handling corrupt database in the recovery extension.
FossilOrigin-Name: ed318be8241981ef96334ba13d3201a717cc812a59aed64e3dc67f7e7e71854b
2022-09-12 20:02:33 +00:00
drh b126ec18b6 Fix the &lt;sys/time.h&gt; include in os_kv.c.
FossilOrigin-Name: 878cc93e779c3857adc711f212520f007256274fcd3f6f3d72c754a937fcd198
2022-09-12 19:33:20 +00:00
drh d8bb0b5ae6 Add a gettimeofday()-based implementation of xCurrentTimeInt64() to os_kv.c.
FossilOrigin-Name: e393ed650ef124143f84e9d787fb996e308dd7af6b8f50df72a6f085b67bf9c3
2022-09-12 19:29:34 +00:00
dan aede9b5bd3 More tests for the recover module.
FossilOrigin-Name: 37fb093b95c6b7d7ad07a275697df73b69f9fb5c5549aea8544b26e38f24833f
2022-09-12 19:23:50 +00:00
drh bb19512438 Fix a debugging/testing edit in the previous check-in.
FossilOrigin-Name: 6fc8d34c0ae1f8277544be741f2f0835fad8e475d35bd24573224ccc1699b8bd
2022-09-12 18:10:41 +00:00
stephan a002cc174f Add sqlite3.oo1.DB.clearKvvfsStorage(). Add controls to kvvfs1.js demo to reset and query the db without requiring the dev console.
FossilOrigin-Name: d845c6c22bd5d3fffc66e0566df346d690dd8bd1fc1688e312161b1a1edcfd79
2022-09-12 17:59:04 +00:00
drh 6acd7b0a75 Faster text decoder for kv_os.c.
FossilOrigin-Name: 3354a2edb762d70ccc31d4d25f81b70e644d76e01cb1e81d2e97b5414d809d30
2022-09-12 17:44:35 +00:00
stephan 5360f5fcff Merge kv-vfs branch into fiddle-opfs branch to add kvvfs-based wasm build and demo.
FossilOrigin-Name: a7d8b26acd3c1ae344369e4d70804c0cab45272c0983cfd32d616a0a7b28acb9
2022-09-12 16:09:50 +00:00
drh 3e4e697d1c New run-time options to the speed-check.sh script.
FossilOrigin-Name: ff46bc9d2a844dde9dafa157114f4d9ee157205ca5c08ed9ee32f018de310e62
2022-09-12 16:07:24 +00:00
drh 1ad51ffe21 Fix uninitialized variable in rollback-journal processing in os_kv.c
FossilOrigin-Name: e49682c5eac91958f143e639c5656ca54560d14f5805d514bf4aa0c206e63844
2022-09-12 15:59:35 +00:00
stephan 2811759d5e Resolve missing kvvfs JS test: sqlite3_initialize() must be called by the client in this build. Add basic sanity test of kvvfs using sessionStorage().
FossilOrigin-Name: 250a935aeb94d3fadec0d3fe22de85de4e658e2fdb3be3aa9a8bbc8f7b7d8414
2022-09-12 13:05:54 +00:00
drh 6b013afb88 Fix the amalgamation so that it builds with the standard unix backend again.
FossilOrigin-Name: 0adf158f4e5791339988185cc29347c75056b062e0db65b5999fdfa1f433c1ed
2022-09-12 12:41:07 +00:00
stephan ad5125a624 Initial build of kvvfs in wasm. It loads but cannot find the VFS for as-yet-unknown reasons (sqlite3 shell works fine), and most APIs throw "null function or function signature mismatch" from deep within wasm, presumably as a side effect of the "missing" VFS.
FossilOrigin-Name: 1a2f24a0bdfc6eaae478916b8f4f9c6b63ead9964534fc2951fb4e995ffe61f1
2022-09-12 12:39:28 +00:00
stephan 73079dba00 Minor cleanups and documentation in the wasm pieces.
FossilOrigin-Name: 4e6ce329872eb733ba2f7f7879747c52761ae97790fd8ed169a25a79854cc3d9
2022-09-11 16:59:40 +00:00
stephan 2a91126d76 Correct the result code from kvstorageRead() for the OOM case to be a negative value.
FossilOrigin-Name: cdbf09fa1b0c93aeb3222a157de33a4688ae629c2b829ffff0f1f62364c5ae1c
2022-09-11 05:44:15 +00:00
stephan ade4f4d645 Minor cleanups and docs for the EM_JS() impls of the kvvfs read/write/delete ops.
FossilOrigin-Name: 53b7572e441be4b4b29b4228d0f95b24457d7faaf6c0b568ed5c970a55e21ffb
2022-09-11 05:38:39 +00:00
stephan e99f742a32 Add EM_JS() impl for kvstorageRead().
FossilOrigin-Name: 06610314fcf644f323c2f7ae11d7f4349b195e66d0ebbee590438dd99d97eb96
2022-09-11 04:49:29 +00:00
stephan d15822965b Initial prototype impls of write/delete-key ops for the pending kvvfs which use sessionStorage or localStorage for db page storage. read-key op is pending.
FossilOrigin-Name: a4bd96f5348e607d0fcb627b751f9d86a188173a4b3bfb2c95f92913a78bd31e
2022-09-11 02:43:08 +00:00
dan 65660916dc Rework recover extension code for readability.
FossilOrigin-Name: 1a2540960e40e3c8c622448fd3862e249bd463c29ae4ce5e39942e942533f60a
2022-09-10 20:01:49 +00:00
drh c6c9c6aad0 Improved comments in os_kv.c. Better names for the key/value name spaces.
FossilOrigin-Name: 8e1652a3856765c9146cc2a7e3b598a2dc7dc84c556b35eab9a35184514384b9
2022-09-10 18:38:21 +00:00
drh 7585f49a0a Move the vfskv.c extension to src/os_kv.c and make it part of the amalgamation,
activated if and only if SQLITE_OS_KV is true.

FossilOrigin-Name: f6632e69c2ec1a7ddc4e51f3567e3bc082ee94a6dd51fdafdc0c3bf386a32d4c
2022-09-10 18:20:59 +00:00
dan bc2e7fc228 Further fixes to comments in sqlite3recover.h. Also rework some data structures in sqlite3recover.c.
FossilOrigin-Name: 599d1f8ec2f9e24924a6f9e66c85664360c7b95531b07a4efe1dd8c096b3fc99
2022-09-09 20:44:56 +00:00
mistachkin bdb6368d72 Fix harmless compiler warning seen with MSVC.
FossilOrigin-Name: 73c4c68d3b4c16caf8281dabddd7365d24691e5ee2da89b1783eca66bfbd9280
2022-09-09 17:50:29 +00:00
dan 7920162093 Handle recovery of virtual tables by recovering each shadow table individually, then writing the CREATE VIRTUAL TABLE statement directly into the sqlite_schema table.
FossilOrigin-Name: 5f2d5ccd56c06c3468377126acfd4be39b79b05bb6fb09b674b2e185df143aa3
2022-09-09 16:25:19 +00:00
drh 25f62508f5 Database must be named either "local" or "session". Each has a distinct
namespace.

FossilOrigin-Name: c8e41279294ea7c2f041c7d4cbbd85fce47d55f2f56180ca837316f7dfd75237
2022-09-09 16:16:33 +00:00
drh f7377d5763 Separate sqlite3_io_methods objects for database files and journals.
FossilOrigin-Name: 74fbf6c2a97729e481d5562344e960d0c237aeb1f07240262d32a24fb1f4b5a3
2022-09-09 14:22:41 +00:00
drh 126976c9f8 Simple reading and writing now works.
FossilOrigin-Name: 00845ac9ef2616e90f5f6a20da3dd040fa8bdcfe73f72fe9c06039561150a82d
2022-09-09 11:41:54 +00:00
stephan f5bf66c875 Remove a reference to a completed TODO.
FossilOrigin-Name: 757ad69607201a26802a304839b8454b1634712458539d6aa8cf999ab2447e59
2022-09-09 05:39:36 +00:00
stephan e66b26818b speedtest1 wasm: add a link in the worker variant which launches the main-thread variant with the selected CLI flags. Append collected stdout/stderr to the main-thread page after execution is finished to avoid having to open the dev tools (which inexplicably slows down wasm execution).
FossilOrigin-Name: 02709ee2beab36d144b807fd9ffaee639e3c1bdd1908a34e05f3fd23dad2ef66
2022-09-09 04:50:18 +00:00
dan 46d4398440 Update comments in sqlite3recover.h.
FossilOrigin-Name: 14164047c43e3ca43aa010c77ea00dfa85400e15645ee0f5b90a677898b6a836
2022-09-08 21:43:18 +00:00
stephan dd628ed58b Make use of the --big-transactions flag in the speedtest1 JS apps.
FossilOrigin-Name: f2846dcbcaac7880394fb14597c3a60ed310419128c4c5b863cd771a7e5cdeb5
2022-09-08 21:33:50 +00:00
stephan e07ce18be5 Add speedtest1 --big-transactions flag to cause its large tests which rely on implicit transactions to be wrapped in BEGIN/COMMIT, per /chat discussion. Added to support of the WASMFS build, which slows down significantly when thousands of implicit transactions are used.
FossilOrigin-Name: 51395c005da46b7fa5a5f28809dd8fea263a6bac2b1492759b3d5a5aa7d399ca
2022-09-08 21:09:42 +00:00
stephan 1174c23e0d Minor build cleanups and tweaks in the speedtest1 wasm apps.
FossilOrigin-Name: 5240fb4d795dea826f23cf5d2152b519f5a46f49bb2499ea868fa7c7f4ce788b
2022-09-08 20:04:52 +00:00
stephan fa7098d543 Experimentally add --nomutex flag to speedtest1 to have it open db with the SQLITE_OPEN_NOMUTEX flag.
FossilOrigin-Name: 45ef576b2ee6369ab1e3aa2ad284764bb334f92a69a33d88b7292bbc0a1b1fda
2022-09-08 19:53:59 +00:00
dan 9a27d65044 Update the shell to use the recover extension for the .recover command.
FossilOrigin-Name: ae832e77084eddd696c80cb926d070a5db9d45dce34156a02522b3140e8f5e8b
2022-09-08 19:22:29 +00:00
dan 3887ffe82a Fix problems with recovering the sqlite_sequence table.
FossilOrigin-Name: 356d2209ea5f6b69ce15b62027c63419c2d039e52f01c74a3810a6317abf4fb0
2022-09-08 17:42:33 +00:00
drh 7cd81af3a9 Compiles and loads as an extension. Starts to run but quickly hits issues.
FossilOrigin-Name: 2e38726f46918b28b5c638967f960a20afd3fe84ad245f3bea39a1d64980435b
2022-09-08 17:12:12 +00:00
drh e3485ee996 Compiles without error, but untested.
FossilOrigin-Name: a329939c32e33d8d00066ba3d4327f07cca1b4e67de0b8d85f1e7f98afe40954
2022-09-08 16:27:50 +00:00
drh cb94132d9f Code snapshot. Completely untested. Probably does not compile.
FossilOrigin-Name: 51e267696d25f2f83b5bb847d5254ab15573ef1a16ea3d7dd8279c73c0bee539
2022-09-08 15:48:01 +00:00
stephan 44a87f08ef Add speedtest1-worker.html, an interactive Worker-thread variant of speedtest1.html. Add ext/wasm/index.html to act as a gateway to the various test pages.
FossilOrigin-Name: f16c68ee6d5ebb8dec2ab656dbab2ddb5f1d5133153ad553f986b31020adaa38
2022-09-08 15:30:59 +00:00
dan 4a08e793c6 Add new test file recoverclobber.test.
FossilOrigin-Name: cb4e950c472bd24a79a8505a7f8e4c3a0f7821648297d05cc760738b777d5149
2022-09-08 11:04:23 +00:00
drh b53e0992b2 Enhance an assert() to impose for tighter constraints on the operation of pcache.
FossilOrigin-Name: 42105eb43a916a7cd839fa6c582eaffaef17f50b23c3e05142400c379feb73c7
2022-09-07 20:11:22 +00:00
drh 8c983ddb67 An improved technique for ensuring that the Rekey() operation does not
overwrite an existing page number.  This approach does not change the
semantics of the underlying pluggable pcache and it is easier to prove
correct.  This replaces the changes at
[aadd38f99a3e5abc] and [81cff9f1955874aa].

FossilOrigin-Name: 7dbddde79e3ca1b81e00288616fc27434698ce6dcb47714728fce8602ae91f45
2022-09-07 19:28:18 +00:00
drh 59ece7e106 Non-working code towards a VFS for text key/value storage.
FossilOrigin-Name: f9c89ee8d5ef46342bea78fa8d4da058d9ea628183ec985642bbda1cdfeacd5f
2022-09-07 17:29:22 +00:00
dan abb28667f1 Ensure that the recover extension properly escapes CR and NL characters in text mode. Also that it holds transactions open on both input and output databases for the duration of a recovery operation.
FossilOrigin-Name: 6cca8913e703635ad89415a60fc84000ac188d9df43f45594b8ad87facb91d54
2022-09-07 16:41:33 +00:00
stephan 3a59dd12d2 Add a note about Emscripten's -sSINGLE_FILE flag, why it would be nice, and why we can't use it.
FossilOrigin-Name: 5ea0623630d769a8f3f07a40cd119be86b631192cdb5178131876b01b40ee5e0
2022-09-06 23:04:51 +00:00
stephan 8fc8b5b35f Get speedtest1.js working with WASMFS/OPFS.
FossilOrigin-Name: 40e60f570d4f489d58d12e27c1c067b41d6c5a5e374c5fce0baa8881ef183216
2022-09-06 20:17:15 +00:00
dan 497b3e6a11 Tests and a fix for the SQLITE_RECOVER_ROWIDS option.
FossilOrigin-Name: 1d5000f5718004110776ff58284d824854a93fe1cd1f6d8a4e8b8b0c2b2c3076
2022-09-06 19:38:06 +00:00
stephan 100b496dd2 Initial build of speedtest1.wasm and speedtest1.html with which to run it.
FossilOrigin-Name: 4441535e3e54dc1881f700fa3878964eb8554a6790fd6aa32945f7cc104a8467
2022-09-06 16:47:43 +00:00
stephan 09aa80d109 Minor internal doc clarifications.
FossilOrigin-Name: 09796cc2bfce7bc4ce11db9673d20737259e9928f0484660cba3a9751f7d01c5
2022-09-06 16:35:54 +00:00
stephan ae1ed83831 Merge trunk into fiddle-opfs branch.
FossilOrigin-Name: 7f76eaec793720db87415a476ddf539bc4dea3e74c8e5406d6739206aebdacc2
2022-09-06 09:59:06 +00:00
drh 2e440a0801 Add the tool/omittest-msvc.tcl script that will try to build using MSVC
using various compile-time options, to ensure that the compile-time options
all build without errors.

FossilOrigin-Name: 6b00ecb59fd303f7985902c35a46db9e729201d4beaedea46596b728d9e4b1c8
2022-09-05 22:54:36 +00:00
drh 0c4517a61a Get the tool/omittest-msvc.tcl script working.
FossilOrigin-Name: 82b89f8a074858a81d841dfc26436e8d39ce4907b8f989eba4d848db16758303
2022-09-05 22:53:24 +00:00
dan 108c51b6ae Fix a problem with script mode and lost-and-found tables.
FossilOrigin-Name: 09ec588d2fe24dd321e88318fe90a9ae912cbc73c8a2d59a10c821625dd12d9d
2022-09-05 21:22:35 +00:00
dan d5e3fe50f8 Add a mode to output SQL statements instead of populating a database to the recover extension.
FossilOrigin-Name: 73058416e7da6581000898b7988a7010e2ce6632246f4c12b4398700c7744b83
2022-09-05 21:00:22 +00:00
drh 4c8404e555 Add a prototype for a script, similar to tool/omittest.tcl, that is designed
to test builds using MSVC to ensure they work with various OMIT options.

FossilOrigin-Name: b1e37a0bd31fa9f8e58c2859a80c23a682b4702eefab2c55b243f1e21d475d6c
2022-09-05 20:42:01 +00:00
drh 6e440ab3de Fix the windows build so that it works with -DSQLITE_OMIT_AUTOINIT.
FossilOrigin-Name: f74a5ea8c986dc33d3afcda169c38abbe55728c56716cf9991a5e2ef7fc4917a
2022-09-05 18:17:03 +00:00
dan a768b67dcc Add the SQLITE_RECOVER_ROWIDS option. To specify that rowid values that are not also explicit INTEGER PRIMARY KEY values should be preserved.
FossilOrigin-Name: 69cc9aba56a196bbd159bd24868aa5ccc60bed0dc612d09ed8a3ae898f156809
2022-09-05 15:56:09 +00:00
stephan 49cb8d7314 Minor cleanups in OO API #1. Add Sudoku SQL to batch-runner.js's list.
FossilOrigin-Name: 4488cb5798f34cbba14eb8e2aa5da8420c14a85d37d278d357406aedd5c3a9e5
2022-09-05 13:24:08 +00:00
dan 7302079dbe Add an option to assume the freelist is corrupt when recovering data.
FossilOrigin-Name: 253e498f5200b8b3e2bc309587af108dd1cec8a884b3d2a49d5406525c9a4b4c
2022-09-03 21:08:38 +00:00
dan be2e212cf9 Take the freelist into account when recovering data that is not linked in to any tree associated with a schema entry.
FossilOrigin-Name: dbd1f1efb349a9c8886e42b3f07d3f4c576924136f111558c7294d0a272e415a
2022-09-03 20:31:36 +00:00
dan f2f8a3a348 Further work on making the recover extension compatible with the .recover command.
FossilOrigin-Name: f2ac315844d8db1bd1c6950a4fef7c459ddd37cc21a8f3daafa5639fad8118e2
2022-09-03 20:07:39 +00:00
stephan cdefd5d046 wasm: minor text and build cleanups.
FossilOrigin-Name: 36ceef94e1935f5e85f79e489cd0ed265a77820fb68329c190794df5e076bf84
2022-09-03 11:41:44 +00:00
dan ad617b4d6d Ensure the Pager.journalOff variable is zeroed if an error occurs while writing the initial header to the journal file.
FossilOrigin-Name: da7af290960ab8a04a1f55cdc5eeac36b47fa194edf67f0a05daa4b7f2a4071c
2022-09-02 21:19:24 +00:00
drh 173b418e60 Fix an assert() associated with the dbsqlfuzz error at [8372468bb5d8922c].
FossilOrigin-Name: 54251f81ebc43839cc97f91d2df4c320c1b96b5690cc11ce45b60c087c9becd6
2022-09-02 17:25:25 +00:00
drh 5fc3a8a32f If sqlite3PcacheMove() really performs a swap, then be sure to renumber the
second page.

FossilOrigin-Name: 81cff9f1955874aade55daa94263f58f06d966bd51ad558755f92777069a7347
2022-09-02 15:09:55 +00:00
drh 4e9bf5ace3 Enhance the pcache1 implementation so that during an xRekey operation if
another page already exists at the destination, that other page gets moved
to the source key.

FossilOrigin-Name: aadd38f99a3e5abcf9bef49f4367752f163cc79500a28f812bb71969d7de419c
2022-09-02 14:29:54 +00:00
drh e04c9f4b33 Mutex protect access to the sqlite3_test_directory and sqlite3_data_directory
global variables.  See
[forum:/forumpost/719a11e1314d1c70|forum thread 719a11e1314d1c70].

FossilOrigin-Name: 8e6ad3a3e942a326cf16432e16d6596c7206c05b6f45cd0ff3a9b836bcfc9deb
2022-09-02 11:45:26 +00:00
drh fee6431392 Improved mutex protection of the sqlite3_temp_directory and
sqlite3_data_directory global variables.

FossilOrigin-Name: ebbe9634d6dde9e097f61fb98a79111e46de422b7bbbd9ed3af7b6f22aacf5ec
2022-09-02 11:12:16 +00:00
drh 18a3a48db1 Experimental changes to put sqlite3_temp_directory behind a mutex.
FossilOrigin-Name: 5ee3515fbb88bf1ae5f8b507844f82dcc429380b6ebeab9b09b52b25ee60a60d
2022-09-02 00:36:16 +00:00
dan 6c86783f9a Further work on making recovery extension compatible with the shell tool ".recover" code.
FossilOrigin-Name: 8df7c7d0dcd1b2fcdad00e765a9868407f0ced02ac4432ee2cdf25d83b130759
2022-09-01 21:00:39 +00:00
drh 44132244fa Defer deleting a transient SELECT statement associated with a flattening
of one arm of a compound SELECT until after the parse has completed.

FossilOrigin-Name: 1c4157c71cd1e062a9c2c79787d17e34e340f28ce1e40573851dfe174f5da7d7
2022-09-01 13:51:09 +00:00
drh a3fc683c80 In the query planner, add a heuristic that will reduce the cost of a full
table scan for a materialized view or subquery if the full scan is the
outer-most loop.  This is shown to speed up some queries.

FossilOrigin-Name: 609fbb94b8f01d6792e5941ab23ce041313d359f6788c4dde6b1ca749ab49137
2022-09-01 10:29:02 +00:00
dan 73b09b87d5 Add new files for an extension to recover data from corrupted databases.
FossilOrigin-Name: f8298eeba01cb5b02ac4d642c06f3801331ca90edea533ea898a3283981a9e49
2022-08-31 20:45:43 +00:00
drh 9c3a114ca0 Enhance the b-tree page sorting code to ensure that sqlite3PagerRekey() never
overloads a page number and uses only the PENDING_BYTE page for temporary
storage.

FossilOrigin-Name: 5007742886bd20de20be3973737cf46b010359911615eb3da69cd262bd9a2435
2022-08-31 15:04:42 +00:00
stephan 53f635df55 batch-runner.js: add rudimentary metrics export to CSV. Add a toggle to reverse the log output mode (in normal order or most recent first).
FossilOrigin-Name: b26e2bc05537a1f451db1bdd36a824d71c3ea25a130b6285f31ff9799d65fa7a
2022-08-30 17:57:50 +00:00
drh 78d15f097a Improved comments in pcache1.c. No changes to code.
FossilOrigin-Name: dd017bb1b3e31c7692d29dc4865d6bda871e429978c8738a39160d0114e5bf9b
2022-08-30 16:54:41 +00:00
stephan ffc0cbb024 batch-runner.js: re-enable unlink-before-run so that OPFS speedtest1 batch can work.
FossilOrigin-Name: ba08f7a7f12601aa580a8dc493b43cdd5ab65c87d5e4b3fd2f549343e45c2d2a
2022-08-30 10:26:33 +00:00
stephan fbf0f488cf batch-runner.js: move generated SQL files into ./sql and start adding hand-written ones (like the Mandelbrot) to that set.
FossilOrigin-Name: 06c106a7d23e4486dbed092757b7588688226ad35539ecc31378a8497f59d1ec
2022-08-30 10:04:08 +00:00
stephan 429899ddad Remove the .timer script entry from fiddle app because emscripten has removed getrusage(), which breaks the timer.
FossilOrigin-Name: 9034f19ae50f196cc2b94478edfcc8d765b08bcf383166f6212b3522dea63c01
2022-08-30 09:51:24 +00:00
stephan 5bb5965710 wasm util: remove superfluous function.bind() calls to eliminate a level of call() indirection.
FossilOrigin-Name: 1aab9627983ef0f016b01f78564e79cf815ed14d4b1b6dc04ec627b96f1b2f70
2022-08-30 09:49:30 +00:00
drh 4d28830861 Any function call can abort. Take this into account when deciding if a DML
statement needs a statement journal.  See
[forum:/forumpost/9b9e4716c0d7bbd1|forum thread 9b9e4716c0d7bbd1] for more
information.

FossilOrigin-Name: 5c95ae6c9b93b9bcf698bb1cad93b2da2e28121b35e7c539b1ddc0ef2de33cfe
2022-08-30 00:11:51 +00:00
stephan 2f06bf2541 Lots of tweaking in batch-runner.js. Minor internal API update in OO #1 API.
FossilOrigin-Name: 24b82b9504db3d8e1335c2300b133f897dc1a541026dc24be5b0ffd8be66d977
2022-08-29 18:58:38 +00:00
stephan 49a785fa34 Add test descriptions to the speedtest1 '-- begin' markers for potential display by the downstream JS code which uses those markers.
FossilOrigin-Name: e5b7006f0f57f10a490d7eaeb7df77251a2f684602fed8ff161d8ce60033e7bc
2022-08-29 17:41:16 +00:00
stephan 2cae138fba Add batch-runner.js for running batch SQL scripts with timing info.
FossilOrigin-Name: 11f3ed61150c5940da6c157e5063e70c3aa0628dfd0023c47bb65b00af74ab1f
2022-08-29 12:39:34 +00:00
stephan f2e624ea91 Add get/setPtrValue() to the common wasm utils.
FossilOrigin-Name: 24d70fc458c6002d5ff3c9f8ce7c66bde299b32aca6417c2dd1236e1412b036d
2022-08-29 12:31:57 +00:00
stephan 9a2efd8e1f Add begin/end comment markers to the test blocks in speedtest1 --script output to facilitate chopping up the output into chunks for JS-side processing.
FossilOrigin-Name: ef0b7ef2d2e19b1f9437fdd7e24f040f662d9907d1fa17c6a3892fcf091b849e
2022-08-29 09:51:17 +00:00
stephan d7d1098baa oo #1 api: correct a case where a null callback is called. Rename some vars for clarity. Increase wasm-side memory in order to be able to load the speedtest1 output.
FossilOrigin-Name: b5058f14fadbc8a1886f27cff08593dd2c8e2b2cb6d7bed3b8733a55f031989f
2022-08-29 08:04:55 +00:00
drh f0f025b340 More assert() statements in the page cache.
FossilOrigin-Name: 7c96a56c0d1de966844da6c5a84c0cd6066130a10724797ce6f228a9182d9b85
2022-08-27 16:11:00 +00:00
drh f6e3e31625 Ensure that every pcache page always has either the PGHDR_DIRTY or the
PGHDR_CLEAN bit set, even during transient page reshuffling during btree
balancing.

FossilOrigin-Name: e67f03c166277951725c194674c84da61c152a5820aa6df55a726f7dda838e92
2022-08-27 14:43:34 +00:00
stephan a3825e78bd Merge changes from trunk into the fiddle-opfs branch.
FossilOrigin-Name: 34279797be0922b9df05e0d4f327f8582ab25fb6fb21583f8441ddf5deded891
2022-08-25 19:38:54 +00:00
drh 23593f8e99 Ensure that sqlite3_prepare() inputs always have a semicolon terminator in
speedtest1.

FossilOrigin-Name: c72756b0f2db30c622f00f43be9245d50e36049bd7740ee6332164f0c48f9c3d
2022-08-25 19:29:35 +00:00
drh 09cb5eb5ac Add the --script option to speedtest1.
FossilOrigin-Name: 1c87d7c58d5aec83f9e2ae3771a81aa17cfae0cf06169025a5db085e2d5749f9
2022-08-25 19:19:25 +00:00
dan aa07b36dd5 Remove a NEVER() macro for a condition that is actually reachable following an OOM. dbsqlfuzz crash-6ef3cd3b18ccc5de86120950a0498641acd90a33.txt.
FossilOrigin-Name: b573e2cffa5fedc893ed30e76e47022b3617ac5583e1eb486afa810b2514c419
2022-08-25 13:32:55 +00:00
stephan 335ad5264f Consolidate oo1.DB.exec() and oo1.DB.execMulti() into oo1.DB.exec(). This is a bit less efficient but certainly easier for a client to deal with and lightens the maintenance burden.
FossilOrigin-Name: 7eff7213dff553b76d7ce45063e3c4a19544716611a0b609593d704076b38d0b
2022-08-25 13:27:52 +00:00
stephan 9afff9f3c5 Refactor and expand the worker1 docs, consolidating them into the top of their file instead of scattered around the internals. Accommodate an API change from yesterday in demo-oo1.js.
FossilOrigin-Name: 0a65747047322b7b585e281ac275e437ce3f46e1d06105c19117213929a906ad
2022-08-25 11:39:12 +00:00
stephan 407f75378e Change DB.exec() rowMode default from 'stmt' to 'array', per /chat discussion. Add DB.exec() rowMode option for fetching a specific column by name. Add result column names to worker1 exec() callback interface, as there's otherwise no way to get that info from a worker.
FossilOrigin-Name: 1bb37e5c477b9eb098362f74a45a55be23d450fe45cdff58c1cbff08b5b3998f
2022-08-24 20:57:37 +00:00
stephan 3734401a95 Expand the worker1 'exec' op handling for per-row callbacks for API-level consistency and smooth some edges between worker1 core and worker1-promiser. Add worker1 'config-get' message to fetch the serializable parts of the sqlite3.config state. Improve the 'open' op's handling of the 'persistent' option (noting that we cannot yet test that case from a worker).
FossilOrigin-Name: 509f8839201ec1ea4863bd31493e6c29a0721ca6340755bb96656b828758fea7
2022-08-24 18:39:46 +00:00
drh 7383f5a7d9 Add a mutex to the sqlite3_db_config() interface so that it is threadsafe
when two or more threads call it on the same database connection at the same
time.

FossilOrigin-Name: 459ad8846ee1ee2d3b277a291c47121692bdf477e779b06e77be8338f62237a6
2022-08-24 17:59:00 +00:00
stephan 9c765e7945 js: resolve the mysterious "extra" unhandled exception notification, caused by inadvertently forking one promise into two separate ones (failing to properly reassign a then() result). Fix a typo in new Worker 1 code which caused the DB(filename) name to be incorrect.
FossilOrigin-Name: 7467ac88801224089b51c6ba7924f93283dd87beca602a186c83632df26cfc85
2022-08-24 14:50:10 +00:00
dan 2ce2b779bb Fix an inefficiency in the ".dump" command when outputing large blob literals.
FossilOrigin-Name: c055d05dbdfd4643d5052c6e6a736c78651fdfcd78f15f791af903df3814832d
2022-08-24 11:51:31 +00:00
stephan 9a34509a06 More work on how to configure the sqlite3 JS API bootstrapping process from higher-level code. Initial version of sqlite3-worker1-promiser, a Promise-based proxy for the Worker API #1.
FossilOrigin-Name: b030f321bd5a38cdd5d6f6735f201afa62d30d2b0ba02e67f055b4895553a878
2022-08-24 05:59:23 +00:00
stephan efeee19a95 The very basics of a Promise-based proxy for the Worker #1 API. Still requires considerable cleanup, testing, and a solution for the exec-callback-via-event-type-name problem.
FossilOrigin-Name: 1e447849fb65887e806e3348a8a68f70ea6802bc0a1e56c385a279f27cc0cdda
2022-08-24 00:51:39 +00:00
stephan a9ac2ed069 Significant restructuring of the Worker #1 request/response object structures to improve readability and clarity.
FossilOrigin-Name: 03b9db9b98cb36faa7de5a8a64d2e13c4aeaadfefb33ac92bb41056f6be3f121
2022-08-24 00:10:45 +00:00
drh e5928b17ca Change the names of the Vdbe.pVNext and Vdbe.ppVPrev fields to make them
unique.  Change to ppVPrev to save a few bytes and a few CPU cycles.

FossilOrigin-Name: 34b8ea31877ae8b40729d37b3f51ae7e15f38be841881ea4a37c9c8f0a52896d
2022-08-23 20:11:01 +00:00
drh 35908b167f Enhance assert() statements to enforce tighter bounds on the P5 operand of
the OP_Next and OP_Prev opcodes.

FossilOrigin-Name: 4e0a07fc6f96e6e7726506b7a5ee942461c1381501f253893fa94d76023b634f
2022-08-23 17:51:39 +00:00
stephan 8c45a9f952 Wasm build flag tweaks and documentation.
FossilOrigin-Name: c8eb3aa8e0f487f14791214caf70d1aa03866e01345c7fa1d5607c24c39dde1d
2022-08-23 17:02:46 +00:00
stephan f7415736d1 Add mksourceid binary to the clean rules and $(TESTPROGS) to distclean.
FossilOrigin-Name: ae3e82f5a3332bea261aca6a0dd02063cf7056c023204cc10f1ae18d75249f91
2022-08-23 16:55:19 +00:00
stephan ae708b2b01 More experimentation with how client-side config state can be passed on to initialize the JS-side sqlite3 API.
FossilOrigin-Name: ea2acc454c012a62556f6d0623d6eff60736d24aa214a64462b423623ef44d47
2022-08-22 21:37:17 +00:00
drh 12f7d4d0a2 Merge all recent trunk changes into the begin-concurrent-pnu-wal2 branch.
FossilOrigin-Name: dde76e91d3cb5aa66fcfab91ab02adfcfcf1eafc1e7e9d3520f4f396d27655d7
2022-08-22 16:08:12 +00:00
drh b77b56bd52 Merge all recent trunk enhancements into the wal2 branch.
FossilOrigin-Name: 0b7578bf3d389f323df5da2ade9f6455d57db92aacfd219f3136dd31588c3a9c
2022-08-22 15:57:51 +00:00
drh 376860ba88 Performance enhancement for sqlite3DbFree().
FossilOrigin-Name: c296a9d958ec360fc8d217363348b4918d665bccb3c4f27503a2dcef7db49052
2022-08-22 15:18:37 +00:00
stephan 7a8a0fceec Merge in trunk.
FossilOrigin-Name: e215d55a97e1ccbca3101621374444d2381d87ef8e8fde5271e31c8b714e43e9
2022-08-22 14:03:11 +00:00
stephan e3cd67603d Refactor JS API amalgamation such that the bootstrapping/configuration is deferred until the whole amalgamation is available, to facilitate providing clients with a way to initialize the API with their own config (noting that we're still one small level of refactoring away from being able to actually do that).
FossilOrigin-Name: 9dbe9a6aecec43b51057375ef1d2d632db0d17eac8b7552c20cc91fc2f1a55d1
2022-08-22 13:34:13 +00:00
stephan 64d04a8d9f wasm: accommodated a JS API rename.
FossilOrigin-Name: 00991335c4dae56232e999398e5e82d8161903ba7d084b16a73a150e83f1f782
2022-08-22 08:55:10 +00:00
drh 41ce47c4f4 Add the new internal interface sqlite3DbNNFreeNN(db,ptr) where both the
db and ptr parameters are guaranteed to be non-NULL.  Use this where
appropriate to save more than 2 million CPU cycles on the standard
performance test.

FossilOrigin-Name: e5eaa80e81fdf86f2875a912b880272b8d099b82b08e945a7988c5dd0fe9d6b5
2022-08-22 02:00:26 +00:00
drh 05cb948bf7 Merge the test case for dbsqlfuzz 18fe4e257be7fa3ecfb0424ab7427e41e97ef9e3
assertion fault, which is fixed with the previous check-in.

FossilOrigin-Name: 2d13ec086e96a5446462ce0f689c40c8196e740cd693f5967bfe9eb961f03463
2022-08-20 19:45:41 +00:00
drh 00378fde23 Enhance the enforcement of SQLITE_VTAB_DIRECTONLY so that it applies to
DML statements within triggers.  Do not allow DML stratements against
virtual tables within triggers unless either the virtual table is
SQLITE_VTAB_INNOCUOUS or PRAGMA trusted_schema is ON.

FossilOrigin-Name: 9433ea4070f52135be64569057f439e7bdb4b3f425d87167c9ebda50011210c1
2022-08-20 19:33:04 +00:00
dan f56291ed02 Add test case for fuzzer case crash-18fe4e.
FossilOrigin-Name: 8372468bb5d8922cf20b8bbee34cfd6044ceb09c26a4efa79a5e6df2c7c4b730
2022-08-19 20:10:51 +00:00
drh 4ff42a82f7 Add notes to ext/wasm/README.md explaining how to run tests on a remote
machine.

FossilOrigin-Name: 7a3c444fb515413254b426908e4d3528ccc664a629628c23b7b85bd21c060d0e
2022-08-18 15:53:27 +00:00
stephan d869a21d13 javascript: rename and simplify DB.callInTransaction() as DB.transaction(). Add DB.savepoint(), which works the same as transaction() but uses savepoints. Correct concatenation of arguments to SQLite3Error to use spaces instead of commas. Test that demo-oo1.js works with persistent OPFS storage (output differs due to persistent rows, but the demo works).
FossilOrigin-Name: e8c323f12b6223bc9dcbbec40698969c7917128aa50cf599c247df23f607ae61
2022-08-18 12:21:58 +00:00
stephan fed91737f5 Improve an exception check in demo-oo1.js.
FossilOrigin-Name: 55e1b775fa7ec492817cc847036df7794c3c22b78558e9e9c513f24c775dedab
2022-08-18 11:16:27 +00:00
drh 78ed74ef8e In the ".dump" command of the CLI, if a schema statement ends with an
unterminated comment, try to terminate that comment prior to appending
the ";" at the end.  [forum:/forumpost/d7be961c5c|Forum post d7be961c5c].

FossilOrigin-Name: 96e112da1ac56767cf49f26132833745b6020d2c60f5c36f86ca84f87ad30c81
2022-08-17 20:18:34 +00:00
drh 16b03c0196 Enhance the "PRAGMA integrity_check" statement so that it verifies the rows of
a WITHOUT ROWID table are in the correct order.

FossilOrigin-Name: 62f934bff495850d0763e07ffa44a557f066ecba9d039363f32287213cba819f
2022-08-17 18:07:52 +00:00
drh 0348fc722f Minor code simplification in the code generator for PRAGMA integrity check.
FossilOrigin-Name: a3b9c7a6c9be5c78a93e5125f16237d2d84b977eca81f527af0198e96435a995
2022-08-17 17:14:36 +00:00
stephan 453af2f6cc Minor cleanups, reorgs, and doc updates for the JS APIs. Renamed sqlite3(-api)-worker.js to sqlite3(-api)-worker1.js, for symmetry with sqlite3-api-oo1.js.
FossilOrigin-Name: f5059ee6f9fc55a381cbf08a30dfb9a5636c0b44341e42f4e9f12a3b109b5507
2022-08-17 16:44:05 +00:00
stephan 6e901b07a3 wasm: minor cleanups in the OO API #1 demo.
FossilOrigin-Name: b9cdcc06a8f70694d10ee5b1a0fd9f08f4c705ce576e5103bbafb36fc9cc2122
2022-08-16 17:29:59 +00:00
drh a8771a1736 Change the legacy RC4-based PRNG to use the RFC-7539 chacha20 algorithm.
FossilOrigin-Name: a0f801151925e882e120f6ab685dcacb9d3268d25b52bc665c5b927bcc7dda1e
2022-08-16 17:18:00 +00:00
drh cd05aafa56 Fix obsolete comments. Add new comments. Fix non-standard spacing.
FossilOrigin-Name: a0d224c6a69941dad1f2b35edcc7ddee343b99eae2aeed74043461f3e97ef5b4
2022-08-16 16:57:33 +00:00
drh 534945ad07 Fix an off-by-one error in the ChaCha20 initialization code.
FossilOrigin-Name: 72e220eed446ea9a02a6ef03e09a01bcb8bbca1f3b32d2e0bf52a17d9722e2f0
2022-08-16 16:40:54 +00:00
stephan e0c582850e wasm: add a small demo/presentation app for JS OO API #1 and make a few minor additions to that API.
FossilOrigin-Name: d6d79b661a1c6137d4693393e02416da4858d58dc84d144081a48d523655b483
2022-08-16 16:36:19 +00:00
stephan ba851ae72e wasm: move another file and update testing1/testing2 to account for [e38d00c2b82d]. Disable wasmfs by default as it breaks the worker-based module loader (reason as yet unknown).
FossilOrigin-Name: 6dad5e0573ee866657ee10b43e55b86fc9caac7a66c13bdbd35c3625a4783f14
2022-08-16 16:16:25 +00:00
stephan 8ac2d8510e wasm: accommodate moving generated sqlite3.js in the previous checkin.
FossilOrigin-Name: 41762f9518bb51b8b23ae6507628d6d3256044e1f2aca6e7251dc57722062c42
2022-08-16 16:11:20 +00:00
stephan ab7de45084 wasm: disable shared cache mode by default. Experimentally move wasm-build generated files up one dir to rule the extra dir out as a problem for Emscripten-related worker-loading failures.
FossilOrigin-Name: e38d00c2b82d7e51ec04cd739514f255edde679b8ddab31fa944b3517d45c3ea
2022-08-16 16:06:12 +00:00
drh 0f201fccc0 Omit an unnecessary test case from corruptN.test that does not work with the
new RFC-7539 PRNG.

FossilOrigin-Name: d9e8c65ed25c4f6222c737f4244d5362dbe433d357f7d133765157446cf4e925
2022-08-16 14:09:51 +00:00
dan 2756f806f2 Add assert() statements to verify that the SQLITE_OPEN_EXCLUSIVE flag is always passed to the VFS when opening a temporary file.
FossilOrigin-Name: e123da49ccae61d591abded52f4721aa10f20d75935c9a3e3fe826a9b8df2317
2022-08-16 10:52:35 +00:00
drh 9113c87ef3 Replace the RC4-based PRNG with one based on ChaCha20. 3x faster.
FossilOrigin-Name: 084d8776fa95c75440530028171c56547a341c9a952ba2f29bb533b538603c78
2022-08-16 00:04:40 +00:00
dan 207cb6f670 Extra test for SQLITE_MAX_COMPOUND_SELECT.
FossilOrigin-Name: c271096736889530f392ff37e631a77f3bc9c46b290dbda245fa05249f4410fc
2022-08-15 19:23:15 +00:00
drh 5187c955f6 The new --script option to dbtotxt.c looks for an SQL script at the head of
the file and generates a CLI script that will construct the database then run
the SQL.

FossilOrigin-Name: 6170e638ebeb12cc40c3247324237978401f701afc270de88ad03e183e82cefc
2022-08-15 18:36:08 +00:00
drh 9c5e1e40e3 In the documentation emphasize that the use of shared cache is discouraged.
Fix test cases so that they all work with shared cache disabled.

FossilOrigin-Name: 52c3ae063e133c5f88cad2c3084cf46218f865ac817ab937e3529d2c6a7af300
2022-08-15 12:26:26 +00:00
stephan 0761780bc4 Add worker-style variant of the tests added in [ae24ac0f7dd9], but building this with wasmfs causes them to throw inexplicable exceptions from the Emscripten glue (without wasmfs it builds and runs fine, but storage is not persistent).
FossilOrigin-Name: 6401595e59179c5c0f6e51c5362cf4391787e7a55b9c6ca655746e30d3251f2b
2022-08-13 17:13:16 +00:00
stephan a7234901b2 wasmfs: use unix-none VFS by default to avoid locking errors in non-OPFS mode.
FossilOrigin-Name: 75561dea1a1afe9cb0a7d58dd82fa519e51cf42e330922cfd8e9ccdf6db4dc0f
2022-08-13 16:36:06 +00:00
stephan 1025227272 Add scratchpad/test app for WASMFS/OPFS running in the main window thread. Enable WASMFS by default in the library build.
FossilOrigin-Name: ae24ac0f7dd9e12a40de0f6ccd61a16f010804da454085f886c217cc600cdba4
2022-08-13 16:11:38 +00:00
stephan 9a4c63b0fc Cleanups in the wasmfs/opfs integration but disable it in order to get the build into a known-working state before continuing with experimentation.
FossilOrigin-Name: 41045be752a5bd7966849638f3ca56f4905308df70f79f2cb6196ca7dce9d525
2022-08-13 13:56:00 +00:00
stephan 90218aec7a Remove OPFS from the fiddle build for the time being - will re-enable once the breakage is figured out via testing with the core API.
FossilOrigin-Name: 3bc510a614973eafa60960a99bedb063594a693bdbfd80d7eb480b293b4ab811
2022-08-13 13:51:56 +00:00
stephan f79451eea7 wasm OO API #1: added DB.callInTransaction() and Stmt.stepFinalize().
FossilOrigin-Name: e37dddc1dd9c0530e4b1c6cb0ca7cba7451caa37734d383c9b47f378d7222242
2022-08-13 13:46:19 +00:00
stephan 2cf599cff8 Corrected TextDecoder.decode() usage to run when its input references a SharedArrayBuffer.
FossilOrigin-Name: d4d773405c579e7efd95be8d81fe14d71218e62e44c523d38e02f89424ba6ce8
2022-08-13 13:42:07 +00:00
stephan 587f9a0a99 Use new -DSQLITE_DEFAULT_UNIX_VFS="unix-none" for fiddle build to bypass OPFS locking errors.
FossilOrigin-Name: b3a93ec75acb38535691d6eaceb5a1a218a5ee3f755a1e12c9255b90567fc795
2022-08-12 18:54:08 +00:00
drh 087f143f87 Add the (undocumented) SQLITE_DEFAULT_UNIX_VFS compile-time option.
FossilOrigin-Name: 49828bdec5f926cd18a069d39a5db0b1e1f3528a2affcfbaa1cf7b98aca51b3b
2022-08-12 18:46:01 +00:00
stephan 24ddbca8d3 Updated fiddle deps to ensure that sqlite3.c is built first.
FossilOrigin-Name: f0ca02611484f5031076d7fae88845e9931715e9108ec0572629200d4f6636ef
2022-08-12 18:07:22 +00:00
stephan a579f4400d Build fiddle with WASMFS OPFS support and attempt to use it if available. It does not work because of an inexplicable exception in Emscripten-generated code and perpetually-locked db, but it's not yet clear why.
FossilOrigin-Name: a16f0a46ec88c560f73d5664e4bf53fb5dd1a22e99a92c11b5c8d784816c3282
2022-08-12 17:57:09 +00:00
stephan 5b0e63eb4c Minor wasm-related doc clarification and remove an obsolete code comment.
FossilOrigin-Name: 1b1f650a08da93da97ed3a96b9a3e4eac567472c266188c02a9bffe1cf620e53
2022-08-12 17:55:18 +00:00
stephan e1c3462484 wasm: document the role of sqlite3-wasm.c. Other minor doc updates.
FossilOrigin-Name: 4c10b9b18feca82440273f8192fef951ad051bbfd8aad4d3c840cf6375130afd
2022-08-12 13:07:21 +00:00
stephan e6d1650c5b wasm: corrected (in as-yet unused code) a flags check.
FossilOrigin-Name: 06b6113783dd81252a5dd125e1cddb52a9d4e387c74cfb3ba88475e47a31060a
2022-08-12 09:32:51 +00:00
stephan fbfe5af903 Wasm-related doc additions and tweaks.
FossilOrigin-Name: ec55a3aa44a80f661dd451b9ea567449ea4c7353901cfd44a4b50ca10f00ddbd
2022-08-11 15:45:32 +00:00
stephan 9289c47df7 Merge in wasm-cleanups branch, reorganizing and updating the wasm-related components.
FossilOrigin-Name: c072594d3de3d6893c5d4a9d68439b84d043325f105b0d065575765a6e66c196
2022-08-11 09:18:09 +00:00
drh f0d12d87fa Fix another harmless comment typo that causes a typo in the documentation.
FossilOrigin-Name: bb084adb53386d8e52ca1e818a8322d2ec641b73fd8568cee01cc74c0ee9f265
2022-08-10 18:40:43 +00:00
drh 778d334085 Remove (harmless) duplicate words in comments used to generate parts of the
documentation.  [forum:/forumpost/54abbe9ccd|forum post 54abbe9ccd]

FossilOrigin-Name: b5e4f0db09ff7790862357df3c6fd38a2dcdfc82ff51b9f0d9470517648c844d
2022-08-10 18:33:57 +00:00
dan 217635f709 Fix handling of columns with names that are SQL keywords in the ".expert" command.
FossilOrigin-Name: 5e1b8221c385deb04a3ff5aafb2e9fc55aecc6ffc68328674e3afe56c4273e29
2022-08-10 15:29:21 +00:00
stephan 2315e83463 wasm opfs: error handling fix for an impossible-to-reach error case. Minor cosmetic tweaks in the wasm JSON enum.
FossilOrigin-Name: 683a3b937e608a5ecaf7f63f054e8a63179d67c8b2348bf843e5e68f27a369f5
2022-08-10 13:22:44 +00:00
stephan 3961b26363 wasm refactoring part 2 of (apparently) 2: moved ext/fiddle/... into ext/wasm and restructured the core API-related parts of the JS/WASM considerably.
FossilOrigin-Name: 27f9da4eaaff39d1d58e9ffef7ddccf1e41b3726914f754b920e3e1fb572cba6
2022-08-10 11:26:08 +00:00
stephan 8c3b7501af wasm/fiddle refactoring part 1 of N: move fiddle app from ext/fiddle to ext/wasm/fiddle, which only contains files intended to be pushed to the live site. Disabled build of the non-fiddle wasm parts, pending a later step of the refactoring.
FossilOrigin-Name: fb4eb93080288b60815be14afd7ddbbca470ce363fa3735352ea9a558fef583e
2022-08-10 09:36:10 +00:00
stephan 3822e8ac47 Merged in trunk for pending tree refactoring.
FossilOrigin-Name: c3a3cb0103126210692bbeb703e7b8793974042e1fc2473be6d0a0d9b07d5770
2022-08-10 07:58:57 +00:00
drh 626bcc88dd Allow the name of an index to collide with a table in a different schema.
FossilOrigin-Name: f963c2523872b59b8a7a14971f703f2eb0d021501b288597a958f6596885d0de
2022-08-09 16:13:21 +00:00
drh c8e9f6818b Omit the (undocumented) SQLITE_PCACHE_SEPARATE_HEADER compile-time option.
FossilOrigin-Name: 60947da687d525e72e62bcddcdf4c5c819818855074cc318c9ee656e83897aef
2022-08-09 14:28:54 +00:00
dan 6aafd74853 Avoid some cases of signed integer overflow in fts5 by casting to unsigned values.
FossilOrigin-Name: 46a78c8c0ed518c4521e6e0bdebeb065bab07076abc444775002e7f4361d2242
2022-08-08 19:29:53 +00:00
drh 26e817f69b Avoid trying to cast an over-sized floating point value into an integer.
FossilOrigin-Name: 3518cd7cb1feeefc3963da72c2d258d81d8914f1e1f427da28a00b6228cf126c
2022-08-08 16:25:13 +00:00
drh 5e10d892d6 Remove an unreachable branch in the numericType() helper routine.
FossilOrigin-Name: a99cc008e46ab1a4fdbe2fa87202d026a10c57be55e3e9fedd935795ed6dc694
2022-08-08 13:04:08 +00:00
drh a3a4da0922 Do not attempt to convert an oversized floating point value into an integer.
FossilOrigin-Name: 6c4fc8385ee22516e0b87fb647327ee7d1a564040ebe2a4d66fc999ef2908df2
2022-08-08 12:19:13 +00:00
dan 8c3cc71a5a Fix a rounding error caused by scalar->logarithm->scalar conversion when using stat4 data to estimate some range scans.
FossilOrigin-Name: 86e30d7c4bb1cebe25b66686d6d1ee170c1a88e5a6c1bed6652ebbb024d62062
2022-08-06 15:28:37 +00:00
dan 8e658f0a13 Avoid having fts3 read uninitialized values when processing deferred tokens.
FossilOrigin-Name: 98729c5fa6c4c418ca9779c35e6874c930e5cee5a90b5eb32b284a0f3762269e
2022-08-05 17:16:00 +00:00
mistachkin e2848932a5 Fix harmless compiler warning seen with MSVC.
FossilOrigin-Name: e36217f2536d89f7a15a879fe3ead32307909c71ced42e1ae7b35f43d4744a80
2022-08-05 05:30:07 +00:00
drh bffd5c1ece Fix a problem with the query optimizer for LIMIT/OFFSET queries when
underlying query is a UNION ALL and both arms of the UNION ALL are
subqueries with an ORDER BY clause.  This bug was reported at
[forum:/forumpost/6b5e9188f0657616|forum post 6b5e9188f0657616].  The
problem was introduced in 2015 (SQLite version 3.9.0) by check-in
[4b631364354068af].  See also ticket [b65cb2c8d91f6685].

FossilOrigin-Name: 6c806f64bbc9e98891bad0868575ee2ec5d0951ceb0c71d3ed417b45d5f27561
2022-08-04 17:15:00 +00:00
drh e24a6f58ae Fix a typo on a comment used to generate bytecode documentation.
FossilOrigin-Name: 64348d0b665f61c3a89f51341f23ca87f939dd9e306dc58c2004e46ab04e696e
2022-08-04 14:02:35 +00:00
drh 3cf46ee508 Performance optimization to the math opcodes of the bytecode engine.
FossilOrigin-Name: 597347d9c39f6ed068ac942a8c9853b1b745fce1eb5e9dd30f6c1cb312b36f53
2022-08-03 19:53:54 +00:00
drh 403f00219b Small performance increase on the binding interfaces.
FossilOrigin-Name: aab24c37fb444804fb91177b4b522909ef1bb85383444d97d82a758f919047d2
2022-08-03 19:37:25 +00:00
drh 3a17e2e5cf Improvement on the previous check-in.
FossilOrigin-Name: c1ab0ea2959259cbd29f624bfd5e7366035f2dbcaab72c0bf55723c0919b7a79
2022-08-03 17:26:32 +00:00
drh b5a6f33e50 For an IN operator used with a RIGHT JOIN, use the number of terms in the
vector, not the number of equality terms, to size the column map.
dbsqlfuzz 14cfdad6ca45e607163f54049ddf5065183dc657.

FossilOrigin-Name: b184c8d9222da6b420b5d7c439bbe4b737d92ffa27f5e933f4e1a9c8117b0ee5
2022-08-03 15:58:44 +00:00
drh f98109e7cb In the xUpdate method of the GeoPoly virtual table, make sure that the
number of updated columns does not exceed the underlying implementation, even
if the virtual table object records an excess number of column in the nAux
field due to table constraints in the table definition.
Fix for the problem reported by
[forum:/forumpost/a096ab7d96bb057a|forum post a096ab7d96bb057a].

FossilOrigin-Name: a04dd475c2c8afdded0afecfd34c6c4c2f48cce520e09b7fa5172ff30f09209e
2022-08-01 13:14:03 +00:00
drh 3b01dd0f5d Restore a VDBE coverage macro that is mistakenly deleted for
check-in [92ac01d41d46ab73].

FossilOrigin-Name: 01de7ec44fb1e8e6f847eaef433216b3163a44a8789451718e1c419ef840ddfb
2022-07-26 19:10:13 +00:00
dan b4d7f1eb88 Fix another problem related to aggregate query LEFT JOIN flattening.
FossilOrigin-Name: 8e12c6f59bcb67be4ff131feecc5af68f4bb0afbcced2e2f7e73289aaf95d99d
2022-07-26 18:43:45 +00:00
drh 1dd1303600 Omit a unused variable initialization in order to suppress a harmless compiler
warning.

FossilOrigin-Name: 2d703c5e941cac72efc21d58e1e35c446b338b2927a42f587ab788826fd7f3f1
2022-07-26 15:41:34 +00:00
dan 4784a78d1c Add test cases related to fix [e717e029].
FossilOrigin-Name: 2909f36bfba8f34b818e5d9ecd8c759f05bbe25765089e78bb022de2b77a4f0f
2022-07-26 15:39:32 +00:00
drh 4b1b65ca2e Make sure IF_NULL_ROW expressions receive a separate slot in the sorter used
to implement GROUP BY.

FossilOrigin-Name: 2bda4fca06ab6be5ad02377a7d1fd9fb9586e3181f1052e4b4937958bdd45efe
2022-07-26 15:32:02 +00:00
drh f4c291275b Improved AggInfo.aCol debugging output, intended to debug a problem with
LEFT JOIN flattening into an aggregate query with GROUP BY.

FossilOrigin-Name: e717e029bde4ee68b6ea77a68721c02ddb6e296f1d310a368137ea3c4164f68c
2022-07-26 14:37:04 +00:00
drh e6710e83fc Add a OOM term to a single assert() statement.
FossilOrigin-Name: 217fc3ebd423e050508d6af97e0664f301824ef9a997ed2d302c006a41227917
2022-07-26 10:16:25 +00:00
drh 3c8fb6fb84 Performance optimization: avoid unnecessary calls to computeLimitRegisters().
FossilOrigin-Name: f48bd8f85d86fd93329dda7ba57f468854a732b89e4126e9c5beec3495d24b31
2022-07-26 01:20:18 +00:00
drh fe888bcf11 Fix an assert() that was made out-of-date by [b52393ac28debe98].
FossilOrigin-Name: db8230e80ed894fc2801cdbb53c2cd9de00192d7f4d8328c29cc214e2e0adde9
2022-07-25 23:34:14 +00:00
drh 509a6303d1 Performance optimization by only invoking sqlite3FkCheck() when it is actually
needed.

FossilOrigin-Name: 98b0e830bc7effa3c9cbb77aebe7c128afc3b210af336b7516108d6435705ba0
2022-07-25 23:01:41 +00:00
drh ef69d2b277 Performance optimization in sqlite3ViewGetColumnNames().
FossilOrigin-Name: 390717e68800af9b71acd635cf6cb123f9a591276df511f11462b42960f9a70c
2022-07-25 22:31:04 +00:00
drh f89812fe52 Fix a harmless compiler warning.
FossilOrigin-Name: 9aba7417c60b30b44c051580f0f997a4d2fc86bc811b532d175305ad4c181fc6
2022-07-25 22:02:35 +00:00
drh 2d2e528e1f In-line a call to sqlite3ExprCode() in insert.c, for a size reduction and
performance increase.

FossilOrigin-Name: 35066b1446228bf030795e7868509c7b54a5681984ac28bf43123f8fac2e361e
2022-07-25 21:37:13 +00:00
drh 47e2fe3ce7 Performance optimization in computing the Expr.nHeight field.
FossilOrigin-Name: 1798ce97c8763d75315e1716d10f6c5be301042c174f41ee8c1fb8d9db99d52b
2022-07-25 20:21:57 +00:00
drh 058e99502a Small performance increase and size reduction by splitting out the
sqlite3VdbeGetLastOp() from sqlite3VdbeGetOp().

FossilOrigin-Name: 92ac01d41d46ab73e189b1e5596ea63e5edb5b15639c5d7bdb981b95366c069b
2022-07-25 19:05:24 +00:00
drh 6aabb72618 Fix an error in the aggregate query LEFT JOIN flattening optimization from
[2cf373b10c9bc4cb] and further enhance that optimization so that it works
even if there is a GROUP BY clause.

FossilOrigin-Name: b52393ac28debe9867227f901d05cccf54f1b467272474500a549d956a5fb4d7
2022-07-25 16:06:14 +00:00
drh ee37302095 Allow subqueries on the right-hand side of a LEFT JOIN to be flattened even
if they contain a GROUP BY clause.

FossilOrigin-Name: 816da9a893ae97a21463562479edb419a8b511ae731d86eccee3fa6e3e7dc96e
2022-07-25 15:54:23 +00:00
drh 6b6d6c6bd2 TK_IF_NULL_ROW expressions must be accumulated in the same way as TK_COLUMN
expressions in an aggregate query.  Proposed fix for the problem identifyed by
dbsqlfuzz 8e17857db2c5a9294c975123ac807156a6559f13.

FossilOrigin-Name: 40d08807209638aad728be2cedbc904e342e76c8e486c364bd571b55dd2e1e87
2022-07-25 14:05:11 +00:00
dan 8bd0b250b7 Reduce a timeout in walsetlk.test from 2000ms to 1100ms so that the test runs a bit faster.
FossilOrigin-Name: 836fa097060dadeb2dc5d4ee2e40621c4af606b1ef7241e2264823e23e4ceb1f
2022-07-25 11:04:13 +00:00
drh 7bace9e9bb Simplifications to sqlite3FinishCoding() for a small size reduction and
performance increase.

FossilOrigin-Name: a995614b9aedf4492e6d7b777293770f268837f8246e1678ef0523738c8a8339
2022-07-23 12:51:48 +00:00
drh 3dedb87bd7 Remove a branch that is no longer reachable due to the previous check-in.
FossilOrigin-Name: 8b4d1b9317624f443fe526858bfe3da5281ba83de1828c62935f48b3e7cf2a88
2022-07-23 00:53:48 +00:00
drh ee6b80c3b5 Use sqlite3ParserAddCleanup() rather than pParse->pConstExpr to implement
sqlite3ExprDeferredDelete().  This is a better solution than check-in 
[c538d07535092722].

FossilOrigin-Name: 2a6f6971fa010219323b976ff53b2606e39fb43fd36c394837c3a8528bf3c425
2022-07-23 00:44:44 +00:00
drh 825fa17b1f Omit the EP_MemToken flag that was made obsolete by [e1f1cfe7f4387b60], for
a size reduction and performance increase.

FossilOrigin-Name: 28934a9d92d5e5ac862a0dc7169f071f39047f98dc79441db697cf353a4b9433
2022-07-22 19:28:04 +00:00
drh 034d111807 In a TK_BLOB Expr node, the Expr.zToken might not be a well-formed
BLOB literal if there has been a prior OOM.
dbsqlfuzz 23871e5805d6c45b392f9b7aa1e8a2b98f3c27cd.

FossilOrigin-Name: c538d075350927222ab0a6598b844f7b15153c5dc008d71b921a2b73c4f4a7a4
2022-07-22 18:25:04 +00:00
drh d00505dc97 Update the documentation for SQLITE_OPEN_NOFOLLOW to state more clearly that
nothing in the database path is allowed to be a symbolic link.

FossilOrigin-Name: de9222697b2ca8cae2cc7aa9082cca3910038ebbab1e8ee15270b06711711a0b
2022-07-22 14:52:29 +00:00
mistachkin e50998859d Fix harmless compiler warning seen with MSVC.
FossilOrigin-Name: 648172de20d70532ed0fb9713b76161dd481e09bbd973c03dffb51fb61b731cc
2022-07-21 18:37:50 +00:00
drh 9553016377 Make use of the sqlite3ExprDeferredDelete() interface in the previous
check-in, and in another place where it might be helpful.

FossilOrigin-Name: 22f90e9683d5cd6619ccdb06a02e9dde9f4b7457391c0dbb4c3216c22fc0db47
2022-07-20 20:36:26 +00:00
drh 3245f3be67 Simplify the logic that converts the "1" expression in "ORDER BY 1" into a
copy of the expression that defines the first output column.

FossilOrigin-Name: e1f1cfe7f4387b60443bd31742e2f49db1a2d0443200318a898ba0da216619be
2022-07-20 16:42:40 +00:00
stephan 4b5919e2ce Merged in trunk.
FossilOrigin-Name: d662796c658997be13fdc3b77ad97101b9513da53fd0b824d7a4050cac3f7eba
2022-07-20 10:09:19 +00:00
larrybr a5cc692422 Improve accuracy of julian day milliseconds calculation.
FossilOrigin-Name: e5e9311863544ef30dccd3bd0b3a048a864a650e69cdf9aab13dbe32a4777b51
2022-07-19 21:12:54 +00:00
dan cf2ad7ae36 Remove a few unsuitable scripts from the "veryquick" test suite. Also have every second testrunner.tcl process favour running test scripts that contain text like "testrunner: slow" before any others.
FossilOrigin-Name: 22d280a5cd395abbedcfffbac3d3b3a614c327be25763ca380c1338a2a7bd33a
2022-07-18 19:32:30 +00:00
drh 2a7b27f1c6 Add the "testrunner" makefile target.
FossilOrigin-Name: 954c6593152f8c7372ed1233b32cce153d0ce4804869cf4ec5504d106a4920a2
2022-07-18 18:13:02 +00:00
drh 077e17b59a Increase the size of loop variables in the printf() implementation to avoid
harmless compiler warnings.

FossilOrigin-Name: aab790a16e1bdff78759f9c9ae87a2559ba82dd34ef3dedfb66035a0db7067a7
2022-07-18 15:02:00 +00:00
drh 449b473ee6 Enhance the REGEXP extension so that it will accept the start-of-input
mark ("^") in the middle of parentheses.
[forum:/forumpost/0d6a9160f81ef1a8|Forum post 0d6a9160f81ef1a8].

FossilOrigin-Name: ed8a8ebd62a319b5dabbdf67ee27141153b9899d7c8f08eeb4bdf35271015c71
2022-07-18 13:55:21 +00:00
drh 18934137dd Fix a problem in the REGEXP extension for the {M,N} construct where M is zero.
See [forum:/forumpost/8694e55a2c29963c|forum post 8694e55a2c29963c] for more
information.

FossilOrigin-Name: af15bb75306a4b94593b8431a34768b3de3d6689293e85ca02db16bf3e9f39e2
2022-07-18 13:10:53 +00:00
drh 67a0bf383f Enhance the ext/misc/regexp.c code so that when it is compiled with
SQLITE_DEBUG, a new function named regexp_bytecode() is available that
prints out the compiled NFA as human-readable text, for debugging purposes.

FossilOrigin-Name: cb5c08978fe8f074e6ae16953575213709e98b8bbae4359e0d2e6de67a7ea9e5
2022-07-18 11:44:16 +00:00
dan b496eef690 Fixes for the generated "mallocs.tcl" and "leaks.tcl" scripts generated by running tcl tests with the --malloctrace=1 option.
FossilOrigin-Name: 449799e2d5902464540e8fda53ab429e0518278dab3b17c86911759114cddea0
2022-07-16 18:08:48 +00:00
drh 02cb3ff004 In the query planner, restore the former aggressiveness in reordering of
FROM clause terms that existed prior to version 3.39.0 for queries that
contain no RIGHT or FULL JOINs.

FossilOrigin-Name: 92d60b64ebfc2d1f0a9cabaa88e7bf0d11737ed01a77b627af10dd1b96a5321c
2022-07-15 20:39:39 +00:00
dan 40e3fa3035 Fix a memory leak in fts3 that could occur when processing a corrupt database.
FossilOrigin-Name: d74f6f6d5136995b8bf900eb671e4b15ca81e03cc1ab5b7a1aa43dc4f3617760
2022-07-15 15:08:48 +00:00
drh b3623e0af7 Fix the whereKeyStats() routine (part of STAT4 processing only) so that it
is able to cope with row-value comparisons against the primary key index
of a WITHOUT ROWID table.
[forum:/forumpost/3607259d3c|Forum post 3607259d3c].

FossilOrigin-Name: 0620e419a927a3da6ebe921aaa3471686f0fdc2e485f4c2d5c88f32092228724
2022-07-15 12:16:11 +00:00
dan b1c034b2f1 Update some faulty assert() statements in fts3.
FossilOrigin-Name: b072851be1a5e49441469dbfa4580e132ebb1d8bc2de98fe1aa67bb9c7cf13a3
2022-07-15 11:34:23 +00:00
dan de353fb5c2 Ensure all testrunner.tcl processes use the same pending-byte value when
accessing testrunner.db. Otherwise locking doesn't work and the db is corrupted.

FossilOrigin-Name: b65225653e4e5c20cc43f4dfdde6c110f6d14b87bff02a65fd61beabe50ec8db
2022-07-14 21:17:22 +00:00
dan ca0720a9d8 Update testrunner.tcl so that it can run the test suites defined in permutation.test.
FossilOrigin-Name: 15ce937ef42491c503cb91f5bb3ce4dc5cdb3a927ff2f2b873fb6bf96808f3aa
2022-07-14 18:09:56 +00:00
drh e834484d89 When applying the omit-ORDER-BY optimization, defer deleting the AST of
the deleted ORDER BY clause until after code generation ends.

FossilOrigin-Name: f22f95b838873f1d2a320afe3d0f4e4847948fcd343097b93a9f684a6f66d6ba
2022-07-14 01:48:27 +00:00
dan 615aeceaff Fix testrunner.tcl so that it can detect the number of logical cores on osx.
FossilOrigin-Name: 14918f28221a3124b78a490fbb483279551ccc5a0032ea854ff0ac365684cc60
2022-07-13 21:28:19 +00:00
dan 4f28ee9d76 Minor change to help message in testrunner.tcl.
FossilOrigin-Name: e4f9cb01d790c27394cc021e7d4a16f4d5feda2d2493abd82e9660d7e6e9fec0
2022-07-13 21:10:11 +00:00
dan 2bb2d53bca Add new script test/testrunner.tcl. For running a set of test scripts using
multiple processes.

FossilOrigin-Name: 0122e93dc19c228546908b9ef5c58f88d27d79233523e2d09a4bbd56b0c492f9
2022-07-13 21:02:07 +00:00
dan cda165ff9a Fix a problem preventing "testrunnter.tcl all" from working.
FossilOrigin-Name: 0ed1e83c6fc12acd06ecf7210a869bebaf5e5e75762e5f16bf1834ecab717d59
2022-07-13 20:26:37 +00:00
dan 95cc6a5e11 Update testrunner.tcl to allow the user to specify which tests to run on the command line.
FossilOrigin-Name: 900febcf362fa5f592c640d16177f33c13aab11ce31a61c7e18ff1be6e70bf9b
2022-07-13 19:57:35 +00:00
dan e9a7ebe19f Fix testrunner.tcl so that it checks for memory leaks.
FossilOrigin-Name: 106f6724d54ccec3edf8c9a0422b89c4f227adb26021ed6f0fc91392ef4b3fc5
2022-07-13 17:46:42 +00:00
drh b88bf865c3 The query flattener should not run if the subquery is a compound that contains
a RIGHT JOIN in any arm and the subquery is not the first element of the
outer query.  Otherwise, prior elements of the outer query will not have
the JT_LTORJ flag set.  Fix for the problem reported in
[forum:/forumpost/174afeae5734d42d|forum post 174afeae5734d42d].

FossilOrigin-Name: 274e244c85935084b2f0f85176283f018bf9b74e7703f985bd5a2f6f8bdcff5d
2022-07-13 15:52:15 +00:00
dan 0505851451 Add test/testrunner.tcl, an experimental script for distributing the work of veryquick.test between multiple processes.
FossilOrigin-Name: ef229cbb7ffbeb8c8877dff70e9d6d43050d2297dee582a37df3a0caaebd2a41
2022-07-12 20:31:16 +00:00
stephan 4413ec7255 Renamed the SQLITE_SHELL_WASM_WEB_MODE to SQLITE_SHELL_FIDDLE, which seems to be more in line with project convensions and indicates that that flag is only intended for /fiddle mode, as opposed to arbitrary wasm-on-the-web use.
FossilOrigin-Name: d1d019bfa2f62b0dc39bba42e17786ca2e4362b6d11d206e5445a051a0f93ae0
2022-07-12 15:53:02 +00:00
dan 03437f7886 Fix another test case error message similar to those fixed in [b3d6b3c3].
FossilOrigin-Name: 6d0f677291d2b5ec68c86292da240c5557422aae1290c0844223974449ce539b
2022-07-12 15:17:50 +00:00
dan 8d48e30b39 Update makefiles to fix building the non-amalgamation testfixture with SQLITE_DEBUG.
FossilOrigin-Name: d9c4a9d09b6b22d7d95420b495dc7d7a42a0638be5824f6af6630539fe787cd4
2022-07-12 15:10:16 +00:00
drh f0af3db770 Fix a bug in wapptest.tcl introduced by [51255bad4c1fb607].
FossilOrigin-Name: b26d097e099b18ff434467adafe83fc55429488dbb8ce9c879bb03d8c5b7d3eb
2022-07-12 10:46:02 +00:00
stephan ea1e3b4251 Renamed SQLITE_SHELL_WASM_MODE to SQLITE_SHELL_WASM_WEB_MODE and no longer automatically enable it if __EMSCRIPTEN__ is defined, in order to facilitate using Emscripten to build the shell for CLI-based WASM runtimes (which cannot make use of the web-specific user input changes). The fiddle build now explicitly passes the new flag on at compile-time.
FossilOrigin-Name: ee059ad5a811a1511e37158f041a7bf9070529d530410d2f1c4395cdd25c6d33
2022-07-12 09:40:27 +00:00
drh b2d7137154 Back out the pager performance enhancement at [a1c090e08139f99d3], because it
turns out we should never allow a zero key into the pcache interface according
to the design specs, even if that page is immediately released without ever
being used.

FossilOrigin-Name: ec96293ead83603ebe5d7f250d6fdc11f22172f05a9513f175331437c3eaa4c8
2022-07-12 07:13:38 +00:00
drh ebdf4c0240 Add the --config option to wapptest.tcl. The argument is a glob pattern. All
configurations must match the glob pattern in order to run.  The default value
is "*".

FossilOrigin-Name: 51255bad4c1fb6074f602586fabad675ae310b9c37f4b80ecf60e2959fb0db12
2022-07-11 22:20:39 +00:00
dan f703b42dc4 Update the expected error messages in some OOM test cases to account for [44d77a7f807].
FossilOrigin-Name: b3d6b3c3fc68dca7e20418eefa35ce3b583322b884b88a11c6773419f027a7a4
2022-07-11 21:40:34 +00:00
drh 90120b9f4a Use 64-bit memory allocation APIs in the sqlite_stmt virtual table, to avoid
harmless compiler warnings.

FossilOrigin-Name: afb9e60ee3b194f33664a6722356e5d3c69ce1d4da1a0affd92b8e6aaf4dd2da
2022-07-11 19:47:57 +00:00
drh d5c2e08449 In the sqlite_stmt extension, store the result of strlen() in a 64-bit
integer to avoid a compiler warning, even though we know that the length
will always fit comfortably in 32 bits.

FossilOrigin-Name: 3fe19452499afc8e6b38905e1ce7e9153adbfebf10dccf39da4b7f1b0cd24f05
2022-07-11 19:12:47 +00:00
drh f1ab642cde Fix harmless compiler warnings about unused debugging functions in treeview.c.
FossilOrigin-Name: 4d6f907712e35eddf6af36eb823c3ccdfcdff1c63b2c224b3bcf34ffec95d511
2022-07-11 18:26:14 +00:00
drh fc7f8f81da Back out the optimization at [1a8c2e54375ee2cf7] because there are some
cases where it does not work.

FossilOrigin-Name: fe39c8d5fd813308fb27a05ce257ff003d3c09c0372f500e8def5a528a2558b7
2022-07-11 18:11:51 +00:00
drh cd9e863079 Ensure that the Parse.nErr flag is set following an SQLITE_TOOBIG error on
a nested parse.  Fix for the problem identified by
[forum/forumpost/d5a82ba9eedee30c | forum post d5a82ba9eedee30c].
Also, remove unnecessary clearing of the Parse.zErrMsg field
following a nested parse.

FossilOrigin-Name: 44d77a7f807f5dc3e94e6cd88a27bea79257f0f2ccf332891bdaa4668d0bb987
2022-07-11 14:36:03 +00:00
dan 3dfab26028 Fix a problem in fts3 to do with deferred tokens and OR expressions.
FossilOrigin-Name: d0bfe5c574dbc467ba97d96306d78a62a55b78dbbbe4d25aa962db6aa40406f6
2022-07-11 14:26:38 +00:00
drh 25bb72a7a8 When an OOM occurs and sets the Parse.nErr value, also set the Parse.nErr
value for all outer Parse objects.
dbsqlfuzz d33f60aaa67733aa700cd69dacf8e0e23a327a29

FossilOrigin-Name: 9a494d25944f4f640026e7a7ae2948e555d7af31487c55ed2ec2818a0789b887
2022-07-10 21:12:54 +00:00
drh 74879e137d Performance optimizations in the WHERE clause processing of the query planner.
FossilOrigin-Name: 50c8e8de52a6ba328bc9eb38e1a67033b643ddf41ff507d2240e97e0072b6db3
2022-07-08 20:03:41 +00:00
drh 5f289e8400 Omit an unnecessary structure initialization in whereLoopAddAll(), replacing
it with an assert() to show that the initialization has already occurred, for
a small performance increase and size reduction.

FossilOrigin-Name: 27be9e963459ca0b20c339264125e175550debf50fdbec7d78412768a6326b42
2022-07-08 18:23:04 +00:00
drh 4efa360f89 Size reduction and performance optimization in whereLoopAddBtreeIndex().
FossilOrigin-Name: 64d2312c121e3dabbc1f86effe9dfc57915395466ea2c3746821848f743b8d8f
2022-07-08 17:57:10 +00:00
drh 72f38795b4 In wherePathSolver(), defer initializing variables until they are actually
needed, in case they are not needed.  This gives a small performance increase.

FossilOrigin-Name: 6f28a9652a3f1e5354afb76314db8f33f92f5e888e336d025f34881c0a8d64ae
2022-07-08 16:56:47 +00:00
dan 43bb445f77 Upgrade the TEA build system used to build the Tcl package.
FossilOrigin-Name: ace65da8fa947926566037762ee9ff1b7686173eb7a50c7b55c4204171228b08
2022-07-08 15:06:07 +00:00
dan 2337e97949 Fix typo in autoconf/tea/win/makefile.vc.
FossilOrigin-Name: 094b2aadd55b0bd93cab5e3172e3a3c1bea83cd312920ffc79088d4a30d2689a
2022-07-08 13:57:45 +00:00
drh 0e400f4e7d Remove an obsolete statement from a comment. No changes to code.
FossilOrigin-Name: 99afb3f41b923f06211d655d46c902da13c8f7aadf2421d6e93c048f5b49e660
2022-07-08 11:55:54 +00:00
dan 554cb87d96 Update options in the TEA package msvc makefile to match other builds.
FossilOrigin-Name: d72e12b99dd1f8abdd855130adc27dbfc8794cfe9550a8495f68c4498a89d07e
2022-07-08 11:21:30 +00:00
dan 5ca48c0b35 Update the build options used for the TEA package to match those used to build the binaries at download.html.
FossilOrigin-Name: 9d6d9dba6680b62c24f5671109f3c28b6829645bffc5ed4e92d016fd2ad02a3e
2022-07-08 11:14:19 +00:00
drh e6aec72312 Very small size reduction and performance increase in btree.c.
FossilOrigin-Name: 9a7c031a822246ee36bc440c7492e9c8ebeec6bc7066e56f114756b1881b2eb8
2022-07-07 22:59:35 +00:00
drh f15b77b7cf Size reduction and performance increase in defragementPage() of btree.c.
FossilOrigin-Name: 1b03f197b5572084177012a58990f8dba7ff10382ff5657fda62867a4d0b1af9
2022-07-07 21:04:03 +00:00
dan c59b7b1fb0 Upgrade the TEA build system in autoconf/tea/. To match tclconfig commit 20fe9e6f59 and Tcl Sample Extension be47fb0446.
FossilOrigin-Name: 1531f7391890d7d3cd091c2d1284230f128e5282bf676967ebcb212210e51e71
2022-07-07 20:49:22 +00:00
drh ebaa9477d7 Size reduction and performance increase in the pageFindSlot() routine of
btree.c.

FossilOrigin-Name: 5d247e38560b97975568e8a48324acaca1002ec2ef7fa3efc9c8aa7d83412aad
2022-07-07 20:29:49 +00:00
drh dce232a298 Size reduction and performance increase in the freeSpace() routine of btree.c.
FossilOrigin-Name: 7d7aed053f600659c63d8bac6d5da77879936c3fc57bfd058a4943b3bc530575
2022-07-07 20:11:35 +00:00
mistachkin 07fae32db0 Fix harmless compiler warnings seen with MSVC.
FossilOrigin-Name: 61e2094afbbcbd5fdf5c3ec06b96134fafb7b854dc9bfa7d0619bed6d35efbe4
2022-07-06 23:50:01 +00:00
dan e7ebe0aa81 Avoid ignoring the last line of a csv file if the final field is empty and
there is no trailing CFLS. Also have the csv extension treat the last line of a
file in the same way as any other line if it is short fields.

FossilOrigin-Name: 587795d47fcaf5142526fabbcc4d5a632f561f258414c2846e8932a49b5b2e6b
2022-07-06 15:44:57 +00:00
dan e36281fc31 Avoid dropping error codes in the xBegin() method of virtual table sqlite_dbpage.
FossilOrigin-Name: 570e2bce595d3a48977dca0bb573dd3fe745a0d13b7122b3ad8ab4745c0700d0
2022-07-06 13:59:45 +00:00
dan f53487a42d Fix for builds with both SQLITE_DEBUG and SQLITE_OMIT_WINDOWFUNC defined.
FossilOrigin-Name: 77916947ce3f0828e1c50bcb8a6438c951ab9e74d83ec4324e82e90e100b9a98
2022-07-05 19:53:59 +00:00
dan c578e4f0eb Fix a problem in the csv extension that was triggered when the very first field in the csv file is zero bytes in size.
FossilOrigin-Name: b12ddabf07b5e06bcee8dda3f990af3a131ab52b8fa969ba061b26d38458f31d
2022-07-05 17:49:04 +00:00
drh 48cae1320b Fix an assert() in btree to be more precise, as the previous form of that
asser might fail due to recent performance optimizations.

FossilOrigin-Name: 4774938134d0105423720bdc1b4e82164a1e28d993c5cd81c1b03f1f0878427e
2022-07-05 10:40:30 +00:00
drh 3325a5c497 Size reduction and performance optimization in sqlite3WalFindFrame().
FossilOrigin-Name: 1a8c2e54375ee2cf73773b798fed0ae07b42f5e068fddc513c093de5c1f46615
2022-07-04 15:14:25 +00:00
drh 16316f1b68 In the getNormalPage() routine of pager.c, consolidate pgno error checking
into a single spot for small size reduction and performance increase.

FossilOrigin-Name: a1c090e08139f99d30aa89db0756dc59fe8990ce15b3db4d4b726cc6acdab46f
2022-07-04 09:41:44 +00:00
drh 0e4ab0db7e Enhance the REGEXP extension so that the end-of-input indicate ("$") is
allowed to occur on one branch of an OR ("|").
[forum:/forumpost/0107d5d40dd273e2|Forum post 0107d5d40dd273e2], second
issue.

FossilOrigin-Name: 3c04d21e6c632feb3bea8d1fa76bedcbfe254b0dc59865633d158a3f1bddefba
2022-07-03 18:12:43 +00:00
drh f28727f61e Fix the initial-prefix optimization for the REGEXP extension such that it
works even if the prefix contains characters that require a 3-byte UTF8
encoding.  This should fix the problem reported by
[forum:/forumpost/96692f8ba5|forum post 96692f8ba5].

FossilOrigin-Name: c94595a6e15490b432f099fefbe2429fa19287f7bdc86332cba0fd1e08f65bd6
2022-07-03 14:25:47 +00:00
drh a57ac0a827 Improved comment on sqlite3VdbeSwap(). No changes to code.
FossilOrigin-Name: 6a8e4fb72a9e3dea9e5752c3d54fddba8878b355bd43f3c879f042f247a6610f
2022-07-03 11:16:03 +00:00
drh 20e9cbe0ff Remove debugging code that has always been commented out. Cosmetic change.
FossilOrigin-Name: b54064170c51a2c905f98737c224e19c06dc728342ca2224e32ebeb47f803bcd
2022-07-03 11:12:59 +00:00
dan 2c1b1ddc55 Do not attempt the OP_Count optimization on queries with HAVING clauses.
FossilOrigin-Name: 566b7842ee286ed8620a87b06de65bc173ad0d208df4157292238fcb059cb484
2022-07-01 21:03:19 +00:00
drh 064390b2b1 Performance improvement in resolveP2Values(). Save over 1 million CPU
cycles by omitting the loop termination condition and exiting when the
OP_Init (always the first opcode in any VDBE program) is encountered.

FossilOrigin-Name: bb179140c8abfd9d05d62380daba027bdeabbbafc720b7e36b99a36af806017e
2022-07-01 19:42:12 +00:00
drh 2b294b543f Small performance improvement to sqlite3_finalize().
FossilOrigin-Name: 8a6913b66cc5af354497044ccb849eb80d00d0799362475f1537a6999196895e
2022-06-30 22:46:28 +00:00
dan d85e4ac347 Add missing "finish_test" command to the end of test script merge1.test.
FossilOrigin-Name: 13cb3f1e63ed1e906f820655645a4966f0cae140ac442177b6685637dcfd365a
2022-06-30 11:01:48 +00:00
dan e26f592ea5 Have the sqlite_stmt table buffer all data for the current scan within the xFilter method.
FossilOrigin-Name: 84a91c255e3d77728820561f16bdd9a87b7ff42b5430a9e13f404dfc3365c716
2022-06-29 15:16:48 +00:00
dan 00a7db72ce Merge in fix for wal2 recovery.
FossilOrigin-Name: 41d4f14bc657f361976c51e1e0d38d2dd2440127fd230f80a1eb8f2a03bc9325
2022-06-27 21:43:33 +00:00
dan 86b7bb29da Fix a problem causing false corruption reports following recovery of a *-wal file that contains one or more transactions and a *-wal2 file that contains a valid header but no valid transactions.
FossilOrigin-Name: f6eafb65a43c650b065abe4a59e329c977cab1856c72dc9162046576d7bbfc8a
2022-06-27 21:42:45 +00:00
drh b0b734d145 Fix a missing dependency for json.lo in Makefile.in
FossilOrigin-Name: 65930a5c069e7274b945ce1aed0abb0edba3d4ab4e63916cc38c11cdef998926
2022-06-27 11:28:25 +00:00
stephan bc7180cdb3 wasm: corrected the isInt32() check to account for negative values.
FossilOrigin-Name: 7223f4bb588b6c759754ef26cbefbb172e767eaa80989b8f9ef104d4e8b9d856
2022-06-25 21:41:26 +00:00
drh 54f1fc4f94 Put an ALWAYS() on an unreachable branch.
FossilOrigin-Name: 58caa50a410b7eb0e68658ea1e606d75ea85cdae04e864270c932246ba990b5e
2022-06-25 20:32:29 +00:00
drh fe9833482f Allow flattening of a subquery that is the right operand of a LEFT JOIN
in an aggregate query as long as there is no GROUP BY clause.  (The GROUP BY
clause will interfere with the operation of the TK_IF_NULL_ROW expression
nodes.)

FossilOrigin-Name: 2cf373b10c9bc4cbc5fe63d0a6948011df7bbc2f40dc025c9349f875da782b88
2022-06-25 19:43:44 +00:00
drh 62f1b48638 Increase the version number to 3.40.0 so as to begin the next
development cycle.

FossilOrigin-Name: 4542e32302067f24bf71c77620e09663f47c3ddce703c21ffa82d09532ae1d51
2022-06-25 19:30:07 +00:00
stephan f79d277999 wasm: added bindings for sqlite3_extended_result_codes(), sqlite3_open_v2(), and the SQLITE_OPEN_... flags.
FossilOrigin-Name: ac876ab9913332c9a5e3045824bf92a2501707aecfb13906d5c280302d9878a9
2022-06-25 19:07:58 +00:00
drh 3691e56210 Fix documentation type. [forum:/forumpost/8d900996ed|forum post 8d900996ed].
FossilOrigin-Name: 869061f18d2f2f500451c87ab62d3ca71a5321d5246b2e2c7bf960e48c6b5250
2022-06-25 18:55:08 +00:00
stephan 0702c63bc0 wasm: lots of doc additions and refactoring. Refactored the WASM memory heap usage to hopefully eventually account for a runtime-growable heap. Differentiate between supported TypedArray types for input SQL strings vs binding/fetching blobs. Might (untested) have implemented the ability to bind UtfNNArray values as blobs, where NN is one of 16 or 32.
FossilOrigin-Name: e10d57dfbaa672a3a4cbfd9a9209552c3bde15cc75af838690ca412fd182066a
2022-06-25 18:18:45 +00:00
drh acf11ef443 Merge version 3.39.0 into the begin-concurrent-pnu-wal2 branch.
FossilOrigin-Name: 09cfef38daa791c2ba114fcf960436322c5acde312234b2c5105c3d95b6c1337
2022-06-25 17:08:25 +00:00
drh e81feadb41 Merge version 3.39.0 into the wal2 branch.
FossilOrigin-Name: ad3a7005e704711fb9de9e96883db3aed82c4f132a66cb74daf67c2527d79f73
2022-06-25 16:53:36 +00:00
stephan fa17e10592 wasm: added utility C code to generate a JSON-format "enum" of the numerous SQLITE_xyz constants so that we do not risk those getting out of sync in the JS code. Renamed initSqlite3Module to sqlite3InitModule. Cleanups in the TypedArray handling.
FossilOrigin-Name: 778062e3b415dca5104eee398950741b6dbb9d4bdf7c998eef18371a42669946
2022-06-25 10:30:24 +00:00
stephan 1c67ec5692 wasm: eliminated the need for Stmt objects to keep ahold of memory allocated for bound strings and blobs. Added alternate string/blob bind impls which are hypothetically more efficient but not yet proven to be so.
FossilOrigin-Name: da1d3151a440567f34a2f6c0b2bfc2e9fab81c256cc361c9ce7b46f2c23a2aa8
2022-06-25 07:42:24 +00:00
stephan f170b0868d wasm: eliminated the dependency on the deprecated emcc-provided allocate() function. Adjacent cleanups in blob binding.
FossilOrigin-Name: 140618b212e3aa9ff2df20f22af846a1c4c5ffaeefd84330446f61362b39a8f1
2022-06-25 06:46:22 +00:00
stephan 766ba7917e wasm: corrected the propagation of text/blob values via UDFs. DB.exec()'s sql may now be an array of strings which get concatenated together before passing it on to sqlite3_prepare_v2(). DB.exec()'s callback now applies to the first statement which has result columns instead of only the first statement. Fixed a precedence but which caused isInt32() to report false positives.
FossilOrigin-Name: 37a8fecb56793fbd3763a2240a0bd62b639c811934d3af2ef0e5ff579073d632
2022-06-25 03:53:43 +00:00
stephan af260308b3 Minor wasm doc tweaks.
FossilOrigin-Name: 42dc500819bfc1308a9542aa2cae4f6dfd98a29237c59cec82e0e6f9e0bf3779
2022-06-25 02:54:20 +00:00
stephan d45ac3947d wasm binding: consolidated the two sqlite3_prepare_v2() bindings behind a single dispathcer. Various internal cleanups and refactoring. Branched because trunk is in pencils-down mode for pending 3.39 release.
FossilOrigin-Name: ab3e50dab4d71557ab5d179bbd6caf7fb61ab7c51dffc8e4714441c189ce3e5c
2022-06-25 02:37:57 +00:00
drh ed84fbf009 Merge the latest changes into the begin-concurrent-pnu-wal2 branch.
FossilOrigin-Name: d3fa692179fb2e704e22f1fa0a64152bd0d6d6b531e359fa038eecf909373d97
2022-06-24 15:26:30 +00:00
drh 6c45111957 Merge 3.39.0 release candidate 1 into the wal2 branch.
FossilOrigin-Name: 5a5e6e368f5cc8fe7cf29696a07942101f5c114084d4117acf277bb3ad8555ac
2022-06-24 15:15:39 +00:00
drh 521e0b6cb6 New test cases for query flattening with LEFT JOIN.
FossilOrigin-Name: 27f68e47320c751e3663507500c1c44f0b7f885f89c678fce6a35b1bc372dd64
2022-06-23 22:43:11 +00:00
drh 3a6e0c3f04 Reintroduce flattener constraint (3c), but this time make it apply only if
the outer query holds a GROUP BY, not if the outer query is an aggregate.

FossilOrigin-Name: 641dfb9182a6cbadb3c452f5420f896791b7844b794f693443bcd38dca14da35
2022-06-23 20:56:45 +00:00
drh d198183465 Add back the ability to flatten a LEFT JOIN subquery - previously removed
due to ticket [cad1ab4cb7b0fc344].

FossilOrigin-Name: f8fe936ad4f7678f9b26ba6fab41c5df9f4938634cdb3286b0f2dcd1357f919a
2022-06-23 15:15:03 +00:00
drh db7a64354a Merge the latest trunk enhancements into the begin-concurrent-pnu-wal2 branch.
FossilOrigin-Name: 82e7e836011f52c09196aa6af9519cd9f584881d41dcbcdb23969e863c6e4fb0
2022-06-16 13:56:11 +00:00
drh 029eb1bf15 Merge the latest trunk enhancements into the wal2 branch.
FossilOrigin-Name: c8ad869938b06378f49c02655c00ee4f3315e1275d15e69d4ff61d6f60230fe8
2022-06-16 13:44:49 +00:00
drh fbc308314b Merge the latest trunk enhancements into the begin-concurrent-pnu-wal2 branch.
FossilOrigin-Name: 4d65f45b31c239d2a400376b30c486eb53113b0e7b594f2b7592df6027507e20
2022-05-28 14:51:49 +00:00
drh 2a27e5e4cb Merge the latest trunk enhancements into the wal2 branch.
FossilOrigin-Name: 934656f13dabc41ccf307b10dca7377c758b8a3b93eca57c072745c2786d6b3c
2022-05-28 14:44:19 +00:00
drh 5eedf42dbf Merge the latest trunk enhancements into theh begin-concurrent-pnu-wal2 branch.
FossilOrigin-Name: d28af1f8a5da1810d41766074560fc3a1ec8fdc22b1efa59056eae4e5c6b66a4
2022-05-10 12:45:41 +00:00
drh 6702d6dadc Merge recent trunk enhancements into the wal2 branch.
FossilOrigin-Name: 33d77fea4084c5aba9203dfeddb820424f102dcb8347dc59e32b922bdb241382
2022-05-10 12:39:00 +00:00
dan 63da55fb89 Ensure that fts5 closes the read-only blob handle used to read the fts index at the end of each xUpdate operation. This doesn't matter for stock SQLite, as the handle will be closed in the subsequent xRollback or xSync call, but prevents "BEGIN CONCURRENT" transactions from being committed on this branch.
FossilOrigin-Name: 093b60846cbc27175b67132a109d9d98bbc35f00faa9b4677c0aa32f0bd40776
2022-05-09 14:22:54 +00:00
drh d7f2abf418 Merge the latest trunk changes into the begin-concurrent-pnu-wal2 branch.
FossilOrigin-Name: 0071d9da18c639e3865b4d216420415b40ad7fa98cd4288c4e2b98c2c59144a9
2022-03-12 15:11:53 +00:00
drh 6c75609f48 Merge the lates trunk enhancements into the wal2 branch.
FossilOrigin-Name: bafaefc6ab4fa59c0a3859cd16a9a29d3d157953fd9002c96e868fc7a54fa18c
2022-03-12 15:04:21 +00:00
dan 6471011339 Add multi-threaded test for the features on this branch. Fix a misuse of xShmLock().
FossilOrigin-Name: 1ec05168c9fe9c09586da795e9db841647c30825f32a2f2c811186fac57760e1
2022-03-07 18:42:02 +00:00
drh 60845d8a38 Merge version 3.38.0 into the begin-concurrent-pnu-wal2 branch.
FossilOrigin-Name: 8387e4f3af552322ed412e21948437bffc4cc237c5c5a80b558b8d395de4999d
2022-02-22 19:56:17 +00:00
drh 981dcb6236 Merge version 3.38.0 into the wal2 branch.
FossilOrigin-Name: de24c8ee41007bc6d25d25856b64d8bdc4581666bca53eb031530b0e498fb0fe
2022-02-22 19:46:57 +00:00
drh 9a5d28c55f Merge the latest trunk enhancements into the begin-concurrent-pnu-wal2 branch.
FossilOrigin-Name: 1336d2f23fcee5bc969cd5ca9e3be1458915e79365e21566ca2ba438f6df260b
2022-02-15 20:47:54 +00:00
drh 3f17a238bb Merge recent trunk enhancements into the wal2 branch.
FossilOrigin-Name: 7dba818893fb898e4556bb7ed9cb0dc5667b19a3d7969c7d35c902d2bf5450b1
2022-02-15 20:37:36 +00:00
drh 5794413d8c Merge recent trunk changes into the begin-concurrent-pnu-wal2 branch.
FossilOrigin-Name: cb045aba7c7f73ca4ad33f77a1ad0c3196eb4db0b2792a1bbbe079593c4f8f56
2022-02-04 18:02:39 +00:00
drh 5666f09523 Merge trunk enhancements into the wal2 branch.
FossilOrigin-Name: d2de02f372a07b10e21f56fc49755545e7c8c889a9527bf3818e5e65cb33bb80
2022-02-04 17:52:05 +00:00
drh 6ee0c5e255 Merge the latest trunk changes into begin-concurrent-pnu-wal2
FossilOrigin-Name: 3d949775e3029c6bf7cb06946ce52e2dfab451a3f7e685d654807cb5ac750b02
2022-01-25 16:05:35 +00:00
drh 8f1396ae8b Merge the latest trunk changes into the wal2 branch.
FossilOrigin-Name: 84f737abd1c05d66627b601dbd9be42c83d9c8e7a6c7fb4945dd85e7765569d9
2022-01-25 15:57:50 +00:00
drh 77dc2a7ff9 Merge the latest trunk enhancements into the begin-concurrent-pnu-wal2 branch.
FossilOrigin-Name: 45fa7efecb8d28dd68b844c9f74b751aab6e7ad7d03f03e8a5881f0e09fb86a2
2022-01-17 19:46:10 +00:00
drh aed427074b Merge the latest trunk enhancements into the wal2 branch.
FossilOrigin-Name: 82f031b41dbf041cdbdfb2f3ccc8d118b22d9e523e54a32e882a67535e9973c5
2022-01-17 19:36:10 +00:00
dan 910bbc4892 Merge checkpoint-on-close fix from wal2 branch.
FossilOrigin-Name: a92eca6c9ca45208df5764e50dec2b257244176f5893b4bee7f9475c4e8d8c8b
2021-12-15 13:42:40 +00:00
dan 348093c719 Fix a problem causing one wal file to be deleted without being checkpointed on close in cases where the last connection to close the db does so with an out of date wal-index header in memory.
FossilOrigin-Name: b0a70a2356c44d65c54c6d9bdf05972071462e72c28d6c6e593147ffa3d27ef2
2021-12-15 12:51:02 +00:00
drh dbe42de14f Merge version 3.37.0 into the begin-concurrent-pnu-wal2 branch.
FossilOrigin-Name: 2171996a02ad2ec571570749ccee421bc0859a690c8393c5edda6cf164d2c8db
2021-11-27 15:11:55 +00:00
drh b90edad19d Merge version 3.37.0 into the wal2 branch.
FossilOrigin-Name: 3425af625bac6c89950a50c2a22c4b7f66b0080811fe16ec8879158420641d5c
2021-11-27 15:00:54 +00:00
drh 012813f3f9 Merge trunk enhancements into the begin-concurrent-pnu-wal2 branch.
FossilOrigin-Name: 8a9fad0f71135b0218ccddeabb7db0c41cfac08ec772f8b3b1df7751485bc30a
2021-11-22 17:01:50 +00:00
drh b321a9b0fa Merge trunk enhancements into the wal2 branch.
FossilOrigin-Name: 135133b7a65c652e95b0642bc2358425af72e46b1b6b64d256547b5e5e8f7e41
2021-11-22 16:47:29 +00:00
drh 3f15be6b72 Merge all recent trunk enhancements into the begin-concurrent-pnu-wal2 branch.
FossilOrigin-Name: 07bc13395de3e45fec2cd571f1c9d0776e198768a4a0dd00479347ca8058a003
2021-11-09 15:18:21 +00:00
drh 234f384386 Merge the latest trunk enhancements into the wal2 branch.
FossilOrigin-Name: 0894f59569378c41207b6cef43dbff9a4873582709d954c809c18f42d918aa03
2021-11-09 13:52:20 +00:00
dan 7271df982d Merge further changes from trunk into this branch.
FossilOrigin-Name: 700876b79c47017cfbf42b8710ec0b50d7a987864db7599593817b962763bc71
2021-11-06 16:46:27 +00:00
dan f54bc90ca3 Merge change [6979efbf07d93e7a] from trunk to this branch.
FossilOrigin-Name: 2bb2448d6042b8c1597aab53b2c1c1aa0cdf9b36f15ef5c44730558f213297da
2021-11-06 16:36:30 +00:00
dan fa4d43fb64 Merge changes from trunk into this branch.
FossilOrigin-Name: 52667bce485354ee4fee87f19015845baef12adf2674127f8c6f1bac1ccf3b7d
2021-11-06 16:10:10 +00:00
drh c34c456d82 Merge trunk enhancements into the begin-concurrent-pnu-wal2 branch.
FossilOrigin-Name: 1958e6facaaca8e695ae3d7e79acdba0025d6221653397ea45f9b8daa56c8d9b
2021-10-21 14:24:24 +00:00
drh c77906eedd Merge recent trunk enhancements into the wal2 branch.
FossilOrigin-Name: 7e2bc836f6aedfd69588f5723f8797f11ee6437d3b63ffc43c88d40e3baadb1c
2021-10-21 14:01:05 +00:00
drh 23cd2fc7cf Merge recent trunk changes into the begin-concurrent-pnu-wal2 branch.
FossilOrigin-Name: bce02eaa0c379d9b90bfaa766fd24d7dd2f009f840ee18cc4f2c7d7b6f7aa368
2021-10-04 12:02:17 +00:00
drh e1c2be1134 Merge recent trunk enhancements into the wal2 branch.
FossilOrigin-Name: 127173e0baba8839d5922a2537dc42e6bb64ed1499b0700bf3e95d42c2f260b3
2021-10-04 11:44:28 +00:00
dan 51f0c762f0 Fix a case where the system error was not being set correctly.
FossilOrigin-Name: 13a524b351a0812214b223aa4e0a9d778ae7a75cf9875da1147a09c5cb76b9fe
2021-09-11 14:33:05 +00:00
dan 9023444fea If an EXCEPTION_IN_PAGE_ERROR exception is caught, make the underlying OS error code available via sqlite3_system_errno().
FossilOrigin-Name: fdb20e9ee48465b94aa6ac3c5e263ecaa7c3b10f4a193e79f965b7c35944b08b
2021-09-10 21:28:56 +00:00
dan ee7c667a36 Change the structuree exception handler in wal.c to catch EXCEPTION_IN_PAGE_ERROR instead of EXCEPTION_ACCESS_VIOLATION.
FossilOrigin-Name: 0c6ab539cfbc908550fa993a587e85d644b6335853ff9781caae860c461c045d
2021-08-26 21:12:07 +00:00
dan 101a6b87c7 Add test cases to ensure that there is at least one test that causes an exception to be thrown from each SEH_INJECT_FAULT macro.
FossilOrigin-Name: e480146f49004c818ad93464289761d8a7246b502bd079f8140d0a6f684ebe4e
2021-08-21 15:49:15 +00:00
drh 67ae593ffe Use VVA_ONLY() instead of TESTONLY() in SEH macros so that the build
works for coverage testing.

FossilOrigin-Name: 9e9d5b566da72adf780685ea27da7d76a7274e904a533438255af92875c94503
2021-08-20 18:18:12 +00:00
dan 5b6c3c8f7d Add missing SEH_INJECT_FAULT macros.
FossilOrigin-Name: 27e9bdb3a90a5826f45d32e0378e9afa90a16ac762a75236f38218f41bc45ded
2021-08-20 17:23:19 +00:00
dan c6effdbfba Add some assert() statements and fix small issues with code on this branch.
FossilOrigin-Name: bd9b373b177a71f5eaeb7ce8dd1dc755b34951ee1376c24652418bb329bfcb45
2021-08-20 16:19:19 +00:00
dan b15c63cbb9 Fix a problem on this branch causing checkpoints that do not proceed because they can't get the CHECKPOINT lock to return incorrect output values.
FossilOrigin-Name: 5a98820c3d20766813bb4b0da23247ac90f380c07827ff13b61d5b606d8311aa
2021-08-20 11:50:12 +00:00
drh aecd7f9d75 Enable this branch to run both with and without the -DSQLITE_USE_SEH option.
FossilOrigin-Name: 840e0a14ebc4a85d4ede0223287d4c1e4465e9f20c3c43bfc74775dddec1fa8f
2021-08-19 23:10:57 +00:00
dan 8aa22ffe98 If SQLITE_USE_SEH is defined, handle structured-exceptions thrown by MSVC builds if the *-shm file mapping is accessed after it becomes invalid for some reason.
FossilOrigin-Name: 5c5fa47076c5da25ca92248df3a4c11b90a0225a07f05d7220fa95d9bbb36f43
2021-08-19 21:01:15 +00:00
drh c8ff95147d Merge recent trunk enhancements into the begin-concurrent-pnu-wal2 branch.
FossilOrigin-Name: 47a9b75981e7e2d2eeb14f83f994c51d2c70d2f5537c02f6028896dd7b4d62ed
2021-08-09 18:35:52 +00:00
drh 2303f0859c Merge recent trunk enhancements into the wal2 branch.
FossilOrigin-Name: 84dac820a0b1f2af73f9563a137b2c57c4c13debcc864063c39027dc8de95ca6
2021-08-09 18:26:05 +00:00
dan 938ede2078 Merge latest trunk changes into this branch.
FossilOrigin-Name: 67ed7422f8173f95598a6b32447b9200ec369cadae71942c3e87c41d492cfc28
2021-08-09 11:03:15 +00:00
drh 8c8b9357e4 Merge recent trunk changes into the begin-concurrent-pnu-wal2 branch.
FossilOrigin-Name: 5b6f6514bd3dcfb45c0ac9a2c4f806fa2f2e8bd7ee62f766bdc218246b4d9e22
2021-08-03 16:44:57 +00:00
drh 0f1e95e616 Merge recent trunk enhancements into the wal2 branch.
FossilOrigin-Name: f615d7baa889030b230e0d4f018dc9f8a6fe95cd3a671d7e2b941ae3240fa6ca
2021-08-02 20:29:08 +00:00
drh d59c189930 Merge recent trunk changes into branch-concurrent-pnu-wal2 via that wal2 branch.
FossilOrigin-Name: 2ff7e5624924d95b280024f67237b8be2afa1593e3ade1da6e9e68c83a21d2ea
2021-06-17 17:44:48 +00:00
drh ebc672832f Merge the latest trunk enhancements into the wal2 branch.
FossilOrigin-Name: 41363fb970273cb5ae2bddf53f3536de047d7d22736483089fd89493c0c701ca
2021-06-17 16:53:56 +00:00
dan b1ce26122f Update this branch with latest changes from wal2.
FossilOrigin-Name: 0ab899012cef8f32e1a4c9a47c54b73576f7fe3757e97353ccbc7fc87a4a1ea9
2021-06-14 14:25:22 +00:00
dan 67f562233f Update this branch with latest changes from trunk.
FossilOrigin-Name: b38c2b8bc4237ee2167cb5e3ec63d08f10a8718654b2210a4bc04cca6d2adfdb
2021-06-14 14:12:57 +00:00
dan ef319309ea Merge latest changes from wal2 into this branch.
FossilOrigin-Name: 4efefb5938c1ec83ee367ae5cc4bba906729b4e8699fa81adc6ca3618c73eb03
2021-05-21 17:31:04 +00:00
drh 80189c5bfe Merge the latest trunk enhancements into the wal2 branch.
FossilOrigin-Name: 95cc7783457b39bbe0866295648a86e7ee5e1660048ab133f1ccf300498ef1bf
2021-05-21 16:42:05 +00:00
drh b2dcfcad79 Merge up all the 3.35.3 changes into the begin-concurrent-pnu-wal2 branch.
FossilOrigin-Name: 4d425e641d3c45692904018aa8fcff678836b45509e640e67fa2d1ade625a746
2021-03-26 14:04:34 +00:00
drh 17e08c6149 Merge the 3.35.3 enhancements into the wal2 branch.
FossilOrigin-Name: a1b8d9d2131830510df291832565311803aa3e047fe07ba979fc69bb0ff329b0
2021-03-26 13:58:26 +00:00
drh 43cc438ef0 Merge the 3.35.0 release into the begin-concurrent-pnu-wal2 branch.
FossilOrigin-Name: 939b9e75307de5067a11351dd4010046a5400e8126c5c2bafea4fbea97e00f86
2021-03-12 17:18:13 +00:00
drh e809deee13 Merge the 3.35.0 release into the wal2 branch.
FossilOrigin-Name: 03142e5f760ef46c7837fef1b21f54baf64f4b9d99d1c9e3f4d8d6c8d1ab875f
2021-03-12 17:08:43 +00:00
dan 7c4df3349d Merge latest wal2 changes into this branch.
FossilOrigin-Name: 88c3c173e04cb01a07a323242872f5d1472d45c726a90ea90386b08e9fd82f57
2021-03-11 20:07:07 +00:00
dan 226fda81af Merge latest trunk changes into this branch.
FossilOrigin-Name: 30f912d0b39ff6fcf9a555affb5990c4bce8726fbd27062cd0c95851724f91d4
2021-03-11 19:42:05 +00:00
drh 1641e35ac4 Merge in recent enhancements.
FossilOrigin-Name: b0fc8ca14ec8ed3a42bf647b112d5fba640f2adc948485da795fc802b74b3106
2021-03-03 19:41:15 +00:00
drh c25b39e41d Merge recent trunk enhancements into the wal2 branch.
FossilOrigin-Name: 0ae81039083dbeb67cd4da9f56cdcb31fd747ba258179a46781a12b2872e9f49
2021-03-03 19:32:30 +00:00
dan 74154a7904 Merge latest changes from wal2 branch.
FossilOrigin-Name: b451bb724ea73367a27c51fcf6e48d02d774c7c5b5b8ec0bd3726a6e4689e591
2020-11-26 18:41:08 +00:00
dan 1a2a4f12dc Merge latest trunk changes into this branch.
FossilOrigin-Name: cb1f6f18e74aeff522c332cb6c8f4c202bba8fba4ff7869198e4501ffa4a0387
2020-11-26 18:08:59 +00:00
dan 50c9522c24 Declare a variable "const" in order to avoid compiler warnings.
FossilOrigin-Name: 4f5481bf291c39e228dd9c67f504dfe70879617b7266245d6236cd1d2d99274c
2020-11-20 11:17:48 +00:00
dan 9bf3da81a0 Fix things so that sqlite3_database_file_object() works with wal2 filenames.
FossilOrigin-Name: c2426ae8a80d61e1ede2d065c10145ba31d4ac615803f48b7903f5f7daaa6f89
2020-11-19 16:08:08 +00:00
drh 1d80c33599 Merge all the latest changes into the begin-concurrent-pnu-wal2 branch.
FossilOrigin-Name: ebefcb8607c91d19ee68935b01fc9c237af41df8b866a9372f7940d89ba44c9c
2020-08-11 13:22:22 +00:00
dan 174f4fa241 Merge latest trunk changes into this branch.
FossilOrigin-Name: 91262e665755a1430f3b667867b9c4fa0cc3aa0388c21e8ba436485beefa9f4a
2020-08-11 11:34:00 +00:00
dan 57f7daf312 Merge latest wal2 changes into this branch.
FossilOrigin-Name: ebd39665e5787fa2c35941ab6c6019f95f0c01229076e3859a3349b516183985
2020-07-31 17:08:40 +00:00
dan aae84d814c Merge latest trunk changes into this branch.
FossilOrigin-Name: 0c0d0a77bc8fa68ee584ca6d7a0408dcd00734a9b6b69d127ee8477c230a749c
2020-07-31 16:01:33 +00:00
dan d05cc8f8b7 Allow BEGIN CONCURRENT transactions to contain schema modifications. However, this causes them to conflict with all other transactions.
FossilOrigin-Name: 2160ede1d12591275c3d82026be7775fd6c890435ab6ca83600029e96ca381a8
2020-07-16 20:57:35 +00:00
dan 4ffaa7c5de Passive checkpoints do not use the busy-handler. So, in order to minimize visible changes for legacy applications, do not enable blocking locks for passive checkpoints.
FossilOrigin-Name: 9c2b4bdd03716bf492ba85198717f3084ebf187bdb068893bd1ff8662362df89
2020-06-29 19:58:26 +00:00
dan 5289c51050 Change things so that if SQLITE_ENABLE_SETLK_TIMEOUT is defined as 2 instead of 1, all blocking locks are taken for a single millisecond and the default busy-handler invoked as normal.
FossilOrigin-Name: ac381e6eb3c9284e65f7aad66d21bee1bca9ef4123684ccaf069b8a6d157a56d
2020-06-29 18:22:21 +00:00
drh 9ee441a3a5 Merge version 3.32.1 into the begin-concurrent-pnu-wal2 branch.
FossilOrigin-Name: 0f9464c509cb99906e2e9df3b55087bebb89d678af1736b431035d6048fface5
2020-05-25 18:18:42 +00:00
drh ea8c9a43aa Merge version 3.32.1 into the wal2 branch.
FossilOrigin-Name: 5c1837572586902313702c46057ef5dc84030683ff3467c6f3f25903b6ab22d1
2020-05-25 18:09:15 +00:00
drh 70311ddd1b Merge version 3.32.0 into the begin-concurrent-pnu-wal2 branch.
FossilOrigin-Name: 3cb296065aba04d01cbaa0b3b17b033cd20d8e84782466e8452bdffa8cee693e
2020-05-22 18:35:37 +00:00
drh ca97a18493 Merge version 3.32.0 into the wal2 branch.
FossilOrigin-Name: 1cb46a7431797978a37e1c6ed77f6473eb44f13d15cd37267d4f9555ac825f53
2020-05-22 18:28:59 +00:00
dan 9019e39d0d Merge latest trunk changes into this branch.
FossilOrigin-Name: 5dc355ccf6ecf0af61e73efbf65bc5e1e192f748568344afbf8bca014ef98df9
2020-05-18 19:55:43 +00:00
dan 69105498b9 Merge latest trunk changes into this branch.
FossilOrigin-Name: 16b1eca922d6e10a0ebce1deb559c6c7dd74f8655e3456bad8384c6f4637e6d6
2020-05-18 17:18:28 +00:00
drh bf92e44f43 Merge recent trunk changes into begin-concurrent-pnu-wal2.
FossilOrigin-Name: 6c11b67bd2c2844566027d59bf5a342d8b2bebd591cd06922e4609e79aa46b82
2020-04-09 19:00:09 +00:00
drh bd74861e91 Merge recent trunk enhancements into the wal2 branch.
FossilOrigin-Name: 6fb870625cb7007fe5055609da35f6af87f3b375b7a43fd4e842d40dfef9169f
2020-04-09 18:46:15 +00:00
dan 526161d3dc Merge latest trunk/wal2 changes, including the SQLITE_ENABLE_CORRUPT_PGNO patch, with this branch.
FossilOrigin-Name: 76668b55894a9c75b2ed1f404ecd3c93ef22586bc20a86ac1ab23f20828474cc
2020-02-04 20:22:32 +00:00
dan 6d671d4c09 Merge latest trunk changes into this branch.
FossilOrigin-Name: 6ad4d6b7ff2b34385039490f1dc8b020254e06fb70b9bff69d453d0c0affc2a2
2020-02-04 20:11:22 +00:00
drh 74b950f145 Merge version 3.31.0
FossilOrigin-Name: 9c3f46384dc73a79f1b04cd5eed8e6a279fcb4387d1dbc20b912ee5aad180140
2020-01-22 21:18:33 +00:00
drh 8caab4aebe Merge version 3.31.0
FossilOrigin-Name: 3bb267deb150c09bc424320d1de4bbaf519310e0e0c4af6a8ac5b6f8d59170b2
2020-01-22 21:12:27 +00:00
drh 7ee2126844 Merge in the latest trunk changes via the begin-concurrent-pnu branch.
FossilOrigin-Name: 350627f3b12a94723b7d52cae8a53d07917ff037c4be9312ef7e4fd2e845b9cc
2020-01-15 14:21:02 +00:00
drh b3168a0056 Merge recent enhancements and fixes from trunk.
FossilOrigin-Name: 35eae71a4dd4bd4e2d4a8f12c5e4a0cdb93dadee353b92e67a70a79b29587984
2020-01-15 14:11:34 +00:00
drh b9bb4e7245 Bring the begin-concurrent-pnu-wal2 branch up-to-date with 3.30.0.
FossilOrigin-Name: dd09f7ce970e8b139d105d005a4c99ae0766c1cba6e0794d46bd6879570c33bc
2019-10-04 16:24:47 +00:00
drh 2c34d3c61e Bring the wal2 branch up to date with version 3.30.0
FossilOrigin-Name: 16e1dced8b56ef422e5b747dd26accba5bf9f2df69b24b10363ef288890e21ee
2019-10-04 16:15:45 +00:00
drh c91f60a056 Merge in the 3.30.0 beta 1 enhancements.
FossilOrigin-Name: ebf8872626e71f5fbaefb76e95d218dfa123d15134762c810d6a5f442325154e
2019-09-30 16:49:59 +00:00
drh d59a566707 Merge 3.30.0 beta 1 changes from trunk.
FossilOrigin-Name: 918bd97d2946c0a403030fcf0eba596a742ada94b122bf0ac4b808097171056b
2019-09-30 16:44:00 +00:00
drh 80c72a78bb Bring the begin-concurrent-pnu-wal2 branch up to date with all the latest
enhancements on trunk.

FossilOrigin-Name: 15593bf3ad9650ce6a6ef8591240eba1be11974d3827cb86ebf0ecec3acda321
2019-09-16 12:29:47 +00:00
drh 254bcdd02d Merge recent trunk enhancements into the wal2 branch.
FossilOrigin-Name: da8bc974126729f1558f3c18118a32efcc5bdd9611533c34090f7e9e5176b4e6
2019-09-16 12:23:30 +00:00
drh 69a35f9232 Merge the 3.29.0 updates into the begin-concurrent-pnu-wal2 branch.
FossilOrigin-Name: 627b428fc8ca0f233cb3ac82a43fc81a22cf81ea958e6b7f69d93a0e53e05d09
2019-07-10 18:40:29 +00:00
drh e31602c62d Merge the 3.29.0 release into the wal2 branch.
FossilOrigin-Name: 8baa402282a231d42f419ac220a32b532766e52180e4a70da874d383d8703121
2019-07-10 18:30:00 +00:00
drh 2d3b3fcb66 Merge enhancments from wal2, which also merges trunk enhancements.
FossilOrigin-Name: 08dcb4593d32b240b585b3fb72fec835c8ad94d9e9c00253e70bbb1b4fc28c94
2019-07-08 21:25:24 +00:00
drh 1e469f79b1 Merge all enhancements from trunk into the wal2 branch.
FossilOrigin-Name: dcbcbc2a6905588a63b9c926473a7097ca0a03803c5bd3f416f268ae2aa3dc87
2019-07-08 21:19:34 +00:00
drh 25b9a24ad7 Merge the latest trunk and wal2 enhancements.
FossilOrigin-Name: 3deaa6e23b2e7f90f91f48435a9a7bfa124b488cd8498d57d8c71a9419cee5db
2019-05-14 22:07:36 +00:00
drh 6fdd8fc5f0 Merge the latest trunk enhancements into the wal2 branch.
FossilOrigin-Name: 23ec34e4f61775198a2b6f8b683fa7db2c57efb1add32c12d5051558952c394c
2019-05-14 22:02:03 +00:00
drh 655af9d7b6 Merge all recent enhancements.
FossilOrigin-Name: 1cd536ecc8f21898475d4641fd67da15a8007af78d59a1f418143b348aceeaad
2019-04-15 15:28:58 +00:00
drh e5fd180648 Merge the latest trunk enhancements.
FossilOrigin-Name: 8950f1195a5e5a67202192857fda50c359a446d20264eab328209239ae0a2d95
2019-04-15 15:17:40 +00:00
drh 76f79406ff Bring this branch up to date with all the latest enhancements.
FossilOrigin-Name: 2e964aafc2773bf6003c9e1d879d6b7467651d5a4592e8371fdc7f393d6485a9
2019-03-26 12:29:24 +00:00
drh b83e6e9420 Merge enhancements from trunk.
FossilOrigin-Name: a0e2e90206fe09cdacc01fce93740fd515c88ed06b03c78c48076b4cb177349c
2019-03-26 12:22:30 +00:00
drh b0120de573 Merge the latest enhancements and fixes from trunk.
FossilOrigin-Name: 543ecb30d67bdf7639ea5d57d091e8c0981d4b5042bb6f9cedd1264bbcbef0de
2019-03-08 16:06:26 +00:00
drh d9df7cb876 Merge the latest enhancements and fixes from trunk.
FossilOrigin-Name: bf20d4ffcb7b27b519502e2986cd0eb1d34d5fd273752fb27de9cdab75772acd
2019-03-08 15:57:31 +00:00
drh 281ee2f754 Add the bgckpt.c extension to Makefile.in and Makefile.msc.
FossilOrigin-Name: 3712d625d625f962c627c74222467ad4858be46feda570449f9cb640549c9ddd
2019-03-08 15:39:24 +00:00
dan d65b8528a5 Merge documentation changes from branch begin-concurrent-wal2 into this branch.
FossilOrigin-Name: cf8a0c71cf1032bd4271cf889304def0fab82a76eac255d734652008fff6c9b6
2019-01-11 15:26:13 +00:00
dan d55e8f5ff0 Merge documentation changes from branch "begin-concurrent" into this branch.
FossilOrigin-Name: 41e742bd0f94894b096b30474fb548f58951756d1ea47d0afb717a54802888f4
2019-01-11 15:22:27 +00:00
dan 77428e6055 Add new documentation file begin_concurrent.md.
FossilOrigin-Name: fdbf97e611ed23f86dc0df9d53421fb04226af0d1c7171d55c11acd0dbe935f0
2019-01-11 15:06:39 +00:00
dan 22672ad742 Merge latest begin-concurrent-wal2 changes (documentation only).
FossilOrigin-Name: d0ab56a3be29d112c792efef8ae7ceacc43fa5773fe06bdc0e45bf9147f84bb4
2019-01-11 15:00:05 +00:00
dan 4853110e61 Merge latest wal2 changes (documentation only) into this branch.
FossilOrigin-Name: 820ba1cc7e374409f27d0f1e62ed17c9670ce0e1f6e1d39a02fa8d48319745e8
2019-01-11 14:59:24 +00:00
dan f552a23740 Add new file doc/wal2.md to this branch.
FossilOrigin-Name: b495dce153f7f886f3dba09593f29ce2375718bf6508f2cfffd1af8071a995ae
2019-01-11 14:58:03 +00:00
dan 3dfff8dd8b Fix the experimental sqlite3_wal_info() API so that it works with wal2 mode.
FossilOrigin-Name: 079daedbe440e712dc944aeef831ef6531d44096da406d52ce0d85a8a49618c5
2019-01-04 19:39:04 +00:00
dan 052416f01f Add the "-external" option to the tserver_test.tcl program..
FossilOrigin-Name: ae7c5ed15d5d549326bf570c8f03b49e8e601e7389bf8d702d0d7db0c0570ee8
2019-01-04 19:25:22 +00:00
dan a45691267f Update the "tserver" program to report the amount of time spent in COMMIT
commands.

FossilOrigin-Name: cf494b412c941704ff73b0d100c55eb9bd268ee156db88b3e60512d501689e8e
2019-01-04 17:12:28 +00:00
dan 5a19f287dc Merge changes from begin-concurrent-wal2. Also modify the unix-excl shm-locks
on this branch so that it is not an error to attempt to release an EXCLUSIVE
lock that is not held.

FossilOrigin-Name: 86685679b4cff94704fe4b52364f25f579d98ee725407cfe3bd2bbf7ecd651a8
2019-01-02 19:46:46 +00:00
dan b93ce6d4a6 Reinstate assert() statements in os_unix.c that were removed to allow
wal-mode SHARED locks to be taken over more than one locking slot
(this branch no longer does that, so the assert() statements can go
back in).

FossilOrigin-Name: fcb7348ba190e48112a681f704313a733f7a377456e7c1d6845ba34c270d86b6
2019-01-02 17:08:06 +00:00
dan 3eeda95d36 Reinstate assert() statements in os_unix.c that were removed to allow wal-mode
SHARED locks to be taken over more than one locking slot (this branch no
longer does that, so the assert() statements can go back in).

FossilOrigin-Name: 8445fb6d5dae98a0129514a10a15f08412bac5a1d8114b12e592c744ea2bc4c9
2019-01-02 17:00:00 +00:00
dan 2cdce9c105 Merge latest wal2 changes into this branch.
FossilOrigin-Name: c31eeb3c87f260fa192638b3610a4d3b84b2c26cfaefc97ed63910495403f8c9
2019-01-02 16:30:10 +00:00
dan 1f3762607d Merge latest trunk changes into this branch.
FossilOrigin-Name: 87ef68f9174b5e21fa2fb4f2fa23e0ecd6a0ea854422c8aa74cd7306a1ed8427
2019-01-02 16:15:02 +00:00
dan 2da2a2698b Merge latest trunk changes into this branch.
FossilOrigin-Name: 5bf212f1a754fde90af0af82dd76f636754829c7372e19df1fd4fa579f0926aa
2019-01-02 16:08:14 +00:00
dan d080e84c81 Merge latest begin-concurrent changes with this branch.
FossilOrigin-Name: 1625887c0240eb1cb37e41d79e789c4ff311352c5708e6a2d3ed1b223ff382e3
2018-12-29 20:47:45 +00:00
dan ff9e9b27d5 Fix a problem causing a corrupt pager-cache if an OOM or IO error was
encountered while committing a concurrent transacation.

FossilOrigin-Name: 48ca30f9d7817d87a5e9a069fdc51b1a34e00585f8a35771895dd743c7bfe07c
2018-12-29 20:42:12 +00:00
dan 71fad44b38 Increase coverage provided by permutation "coverage-wal" on this branch.
FossilOrigin-Name: f664f940a7eb6938b1ee171143a691e2f497aec921f979be63ef844d38053f56
2018-12-29 16:34:22 +00:00
dan d445ed094a Add new test file wal2rollback.test to this branch.
FossilOrigin-Name: 85a376fc6c77117b96814df800b3d68a441d69602ffc2eb8cd7dba29f25d69aa
2018-12-28 16:20:53 +00:00
dan 30db882a87 Merge latest wal2 changes with this branch.
FossilOrigin-Name: ea96001e801fbf228d1fb63b9d3c73fd9da54994d3cfe383ea76edfa6da83b8d
2018-12-27 17:11:59 +00:00
dan ce909d1aae Increase test coverage of wal.c provided by permutation "coverage-wal" on this
branch.

FossilOrigin-Name: 2f7f893a702728745445f6ef5f914c1157058a8fbdfd1a58cfb8906e5566729d
2018-12-27 16:49:33 +00:00
dan 0f297b6e50 Improve testing of the wal.c changes on this branch.
FossilOrigin-Name: 63483e22c775183e01924b5cb3d0f3655b63c3cdd06faf26cacd4d0913c0055c
2018-12-26 20:42:32 +00:00
dan 2d111c8bbb Merge latest trunk changes with this branch.
FossilOrigin-Name: 404f9d99d325ddf82fde6d957860d30b2fb67f232c0878a3cf6df9596541f16e
2018-12-26 19:10:59 +00:00
dan 665f3d209b Enhance the wal2 header comment in wal.c to explain how the wal-hook is
invoked in wal2 mode.

FossilOrigin-Name: 118aa7e32a94ad971a955ab60db5bfc5b6241f93734a41ba83ab72ea71bc9aaa
2018-12-22 15:50:00 +00:00
dan 25117ddbfc Add the "mutexfree-shmlock" patch to this branch.
FossilOrigin-Name: c3191fc8fcb0775f87f55cb8124e0c36caa54993c2a5fe770d437e4f6ee72d77
2018-12-18 19:46:50 +00:00
dan 49967a4d9a Fix a test script problem in wal2savepoint.test.
FossilOrigin-Name: e388d02940e8c17e8e3f460643d988c8f523382df343f4fe766c2da17878e98d
2018-12-18 18:16:00 +00:00
dan 0ba73a6e4d Merge the wal2 feature into this branch.
FossilOrigin-Name: a2c618c0a45e1b1d271a4a981ee1e3b368dfcce94eda5c2d7d08f794784046d9
2018-12-18 18:01:17 +00:00
dan db6fe08914 Merge latest begin-concurrent changes into this branch.
FossilOrigin-Name: a93ca38b432e0c9fb2e00499cfc88d09f59f27f908f377d8ae99b6717e548a60
2018-12-18 17:47:53 +00:00
dan e7f92d27ad Merge latest trunk changes into this branch.
FossilOrigin-Name: b3a163b46cad19a2e6a90cdd5b4c2e94d66e80b7ad2a0c9857580fe74748a604
2018-12-18 17:32:08 +00:00
dan b031157128 Merge latest trunk changes into this branch.
FossilOrigin-Name: 123cbb3312917ad5b3c32556547c5b7e8ba4e2d2def8651ff80f9fc1bdc1875c
2018-12-18 17:20:01 +00:00
dan 8b832e24d4 Add the ".integrity_check" command to tserver.
FossilOrigin-Name: fa46fa3bfcd4b6d1d219db4179ce69d0edb0786a521acaa034fdb74b312274ff
2018-12-18 16:24:21 +00:00
dan 8348abc3a2 Add wal2 related tests to this branch.
FossilOrigin-Name: 5645822039b9f36aeb986645106cac2d95b09964689deb767cadaa6ea1d1867f
2018-12-17 18:26:48 +00:00
dan 1ac9aaf3fc Merge changes from wal2 branch.
FossilOrigin-Name: 85f49f0d7344611ac4157421273660c648f73ca62afa4faa8707475f21c67f7a
2018-12-17 15:22:44 +00:00
dan c5140b39ce Add extra test case to wal2recover.test.
FossilOrigin-Name: 1a5aa73435b11fff0567672dc921d34aad57adf0381ad430e1389f39a6f025b7
2018-12-17 15:22:24 +00:00
dan 6d537ca2d8 Merge latest wal2 changes into this branch.
FossilOrigin-Name: 06bb80eeb84f57b1e8109a35f5b14992a2e23f2bf5a8921fa29087f96cb44d10
2018-12-15 20:59:14 +00:00
dan 93543215e0 Further test cases for wal2 mode.
FossilOrigin-Name: 9cb5f8dab685f5ea36ad142cfa588dee82e87f3a89e8dcf84e0ee124bb29bc7f
2018-12-15 20:20:13 +00:00
dan e698c1f79e Add further tests for wal2 mode.
FossilOrigin-Name: 54e628f902a5508badc8941ceda1bec12fe8f58969c91c670a37888ef2681880
2018-12-14 19:53:39 +00:00
dan 597bf19266 Add tests cases for recovery in wal2 mode.
FossilOrigin-Name: 34f56f8a4239781071edf6317723d0a7333249551c2c1ce7ed39fc116d53d780
2018-12-13 16:26:43 +00:00
dan eeb778d506 Add new test file wal2big.test.
FossilOrigin-Name: e3e50bcdab8c91e003942d84430b3e580e034141236d19dda0e8af4ecf0e085b
2018-12-12 20:39:38 +00:00
dan 2c4328ed87 Add tests to ensure that each of the 4 wal read-locks does what it is supposed to.
FossilOrigin-Name: 4d5779f31d4931edb8bb8952d8886625ead5e51f0c308e5763e519427f6609e1
2018-12-12 19:04:19 +00:00
dan f17e510ee5 Change the way wal2 locks work to ensure a reader only ever has to lock a
single slot.

FossilOrigin-Name: 18b2c23ac53d985ccc5798ea2d92fb75644b857c373fb490e0d04d5d0194a3d5
2018-12-11 17:56:23 +00:00
dan ca72570a11 Merge latest trunk changes into this branch.
FossilOrigin-Name: d8dd98a39ea061dea264a7d81153f7b1be2f81b554b30d0ce289897c802209bd
2018-12-11 13:44:59 +00:00
dan 107c5de8b2 Make SQLITE_MFS_NSHARD a compile time setting.
FossilOrigin-Name: b9a741519055a8560ec1d0ab7d0c15ff8db53e740011825506cd2ede11c956f0
2018-12-10 16:52:31 +00:00
dan 1d5430f85e Add extra tests to shmlock.test (direct testing of xShmLock methods).
FossilOrigin-Name: d2c785f94cc6b704ce29147399e2c473fd0f4ff377f1379bd29178bf6d127c1a
2018-12-10 15:51:05 +00:00
dan c967a11569 Rework the SQLITE_MUTEXFREE_SHMLOCK code to reduce contention.
FossilOrigin-Name: d9157dd176a2d18c6e02a2a0c7e16cef2da43bf44be9765e0363f34aebad23e9
2018-12-10 15:24:29 +00:00
dan 5b5cda2027 Avoid a mutex in-and-out in unixShmBarrier() on this branch. Use
__sync_synchronize() instead.

FossilOrigin-Name: a8c5fd86ce1d9b5815f82106f2e44797b743bbb3c44aee5f835ce8d278caf8f7
2018-12-10 09:45:38 +00:00
dan b0b27ab5c6 Add multi-threaded performance test program "tserver" to this branch. Fix bugs
in the begin-concurrent/wal2 integration revealed by the same.

FossilOrigin-Name: 7bd3b35661d7d0e51113b9e4b15a0ab7f8e26edeafb43941ef5b44bb94df5109
2018-12-07 20:25:14 +00:00
dan 24fb9ffbc1 Experiment with using an atomic CPU primitive instead of a mutex for
intra-process locking with the unix-excl VFS.

FossilOrigin-Name: 8f4cb9dd3396bcaaf85dcdb4e3ae3c96f28a4d71d72665c4abf7c221370be626
2018-12-06 18:58:58 +00:00
dan d597b966d9 Add test file wal2snapshot.test that should have been added as part of an
earlier commit.

FossilOrigin-Name: f6baa7e1163ed5f61375b0554337030fec23e8a9f60c6412e1b5d626961e93f9
2018-12-06 16:54:44 +00:00
drh ef180caded Merge fixes from trunk.
FossilOrigin-Name: f2083ee410b7504e8478f7373e76ded913e3a6a92cb7345b8c1ac27554f6edf8
2018-12-06 03:13:02 +00:00
drh 9c1eb40d69 Merge bug fixes from trunk.
FossilOrigin-Name: 1e13aaa29fb6324b60b3758bdab0491fdef9727e0de603d3da3e1885b52c5caa
2018-12-06 02:08:05 +00:00
drh 9ee5641ae0 Merge ALTER TABLE and sqlite3_normalized_sql() bug fixes from trunk.
FossilOrigin-Name: 3793e5d5d60f909480c83bc2a6473d27e414d69b0c0e3289413c9bfd2f15bb08
2018-12-06 00:08:48 +00:00
drh 254fa054ec Merge ALTER TABLE and sqlite3_normalized_sql() fixes from trunk.
FossilOrigin-Name: acf10b3f8d6675ddf787f7db55a9ff0ec5729adb6fa70b48b13d2cd71862cffd
2018-12-06 00:05:18 +00:00
dan 7b812b6b16 Add test cases to wal2concurrent.test.
FossilOrigin-Name: ef9e7a87a4ce4fe9596aed2f880f2b9fc3526c00c48f2868c625a4c24be65abe
2018-12-05 18:25:23 +00:00
dan ff46e3f460 Fix a test script problem on this branch.
FossilOrigin-Name: 692ddc280850ea2ff5d2fa3743f85fb123b4e4ebc452aae9c58b8d80d22b75b1
2018-12-05 17:36:20 +00:00
dan c30d437376 Fix a test script problem on this branch.
FossilOrigin-Name: 285d1c5904dd607457c6b5ff7be3d3191a90ca8b1fb0837327663c9910f978ac
2018-12-05 17:31:24 +00:00
dan 079ae546fe Fix a problem causing "PRAGMA journal_mode" to report the wrong journal mode
(wal instead of wal2) under some circumstances.

FossilOrigin-Name: bf309107dfc4d2abd68b9339c68aeaaffd278096bb2a5912e4a0b0c124fd6345
2018-12-05 17:20:47 +00:00
dan e05f922583 Fix a problem causing "PRAGMA journal_mode" to report the wrong journal mode
(wal instead of wal2) under some circumstances.

FossilOrigin-Name: 1d8d4f689653ce80157740e339f7f1b42479bf90d82b176d8202d0a49f428398
2018-12-05 17:14:03 +00:00
dan 5330a3c6c9 Fixes for snapshots API on this branch. Also ensure that the snapshots API
cannot be used with wal2 mode databases (for now anyhow).

FossilOrigin-Name: d8c2d55fa4ec95c6506201edcd1cb2ef108a233bd87a6154f3593909977f7d3f
2018-12-05 16:45:38 +00:00
dan 6456b3929c Fixes for snapshots API on this branch. Also ensure that the snapshots API
cannot be used with wal2 mode databases (for now anyhow).

FossilOrigin-Name: 19c61ab79458936ff4dfca46cf4d1fb1ab16d7bdb5024f502eb4339ec4eef32c
2018-12-05 16:31:02 +00:00
drh a693b000df Merge enhancements from trunk, especially the sqlite3_normalized_sql()
enhancement.

FossilOrigin-Name: 342c9538d9c6a993ac0acaa6f74ad58886bcef7bb53783d053f9b24c131aec5d
2018-12-05 13:49:23 +00:00
drh f8d7432aa3 Merge enhancements from trunk, especially the enhanced
sqlite3_normalized_sql() interface.

FossilOrigin-Name: 47b73f6bfee8c5e41c408fb70dff0e4596c0a3eb0aeba40ad232a6cb1fe75532
2018-12-05 13:44:19 +00:00
dan 7a13e692be First attempt at making features work together. Only the most minimal testing
so far.

FossilOrigin-Name: fd707001f0afb1cf32cfeeda3ec7b5622eb49ddedf8fec1a7aa4c8841c77c37a
2018-12-04 19:41:07 +00:00
dan 50232dd821 Fix a problem with SQLITE_ENABLE_EXPENSIVE_ASSERT builds on this branch.
FossilOrigin-Name: ddb4a6fbf8619db058b5eb8fcd687084ed4b65a6f69810357e324158257a911f
2018-12-04 13:51:43 +00:00
dan 4d8517e500 Merge the wal2 and begin-concurrent code. Both features work, but not at the
same time.

FossilOrigin-Name: b7281a1caa574870a071bea3e96b1d8210c28c17f9094449b3ce1a42b311e6a1
2018-12-03 20:49:34 +00:00
dan 834c48c279 Minor change to wal.c on this branch to make it more similar to trunk.
FossilOrigin-Name: 6a7af3ead5949c461430c1fa92798dc2bbbc58c8cd504005c5afa38993f0be82
2018-12-03 20:38:15 +00:00
dan 51883dfc8b Cherrypick a couple of fixes from begin-concurrent-pnu into this branch. The
differences between the two branches are now that this one does not have
"PRAGMA noop_update" or the mutex-free PRNG.

FossilOrigin-Name: a56506b9387a067ef259504d127694ad20223f4b08781d1676ff7f5fdd9443d8
2018-12-03 19:29:37 +00:00
drh 15e8543ecd Merge in all changes for release 3.26.0.
FossilOrigin-Name: 85fd92c71fd5b29bd45bb1717a5be761600fc23bed12eaff9eed708701a0fdf7
2018-12-03 18:24:31 +00:00
drh a63585eb8e Bring up to date with version 3.26.0.
FossilOrigin-Name: f0ddb358cc68e5ec6d9e758893ab3da058a3b2e705124a7449279c992e672a4a
2018-12-03 18:15:52 +00:00
dan 816fc35956 Increase a timeout in test file walprotocol2.test. To account for unix builds
without HAVE_USLEEP.

FossilOrigin-Name: 480be916c840e9e28010c22cf29a8396502c9e6387f31f750260c6290f58e9a1
2018-12-03 18:13:46 +00:00
dan 8eed108df1 Sync this branch with the latest trunk.
FossilOrigin-Name: 7a44fa5a350a3f19b8e9f5196d22535788885f8c0e849572202bf64a055ddc2d
2018-12-01 20:14:06 +00:00
dan 9b5c67f784 Merge the mutex-free PRNG change into this branch.
FossilOrigin-Name: 81e626f4a44be75425cf916ec61b6b36df0907ebac4adbf6786f87ad4f3a0674
2018-11-30 16:26:39 +00:00
dan 8e90c85bfb Merge latest begin-concurrent changes into this branch.
FossilOrigin-Name: 76608f750ab13c0a165def9672759fee43cf4e9895df3bfa21765e08358b07a0
2018-11-26 07:34:33 +00:00
dan dcb3992c66 Merge latest trunk changes into this branch.
FossilOrigin-Name: 28a615a2e0f48b0fee3eaf7841ff902e069fa6c221df6ad9a57b8709c88561fb
2018-11-26 07:21:06 +00:00
dan 0131a914d3 Merge latest begin-concurrent changes into this branch.
FossilOrigin-Name: 6f3dd9809fdef7d6195f1f93428a662d5a8c01dba9815daa22d1b94059a2eb43
2018-09-28 21:11:12 +00:00
dan 2dd93c8838 Merge latest trunk into this branch.
FossilOrigin-Name: 86750c927cb5ba36acad33d5279629699c42b598e70fdc4887b40a1a16771ff6
2018-09-28 20:58:10 +00:00
dan c1b7fdd43e Merge latest begin-concurrent changes with this branch.
FossilOrigin-Name: d33527d2231db9eda53124d3e11582a6b2eefa994eb7d9435fb349d3d8522ed7
2018-07-12 19:39:45 +00:00
dan fccd5d43a9 Merge latest trunk changes into this branch.
FossilOrigin-Name: 6a00a34e198408d69d638a25763f64b8a5beec61a60affb4d344426a2e232d72
2018-07-12 19:28:43 +00:00
dan ec6e332f01 Merge begin-concurrent changes into this branch.
FossilOrigin-Name: af17432eb135895f9a55115c0eb364745715bb2dc14da01cbb2dc498e61c1853
2018-07-10 15:48:39 +00:00
dan eb30704cc9 Merge latest trunk changes into this branch.
FossilOrigin-Name: e9a3e8642e7acc8e63b35662ffae3a5a0da8d4f1ecefcb1db3110be19cbdab2b
2018-07-10 15:45:54 +00:00
drh de26c16ea1 Merge the 3.24.0 changes plus a few subsequent enhancements.
FossilOrigin-Name: be7004a971a174ae5c862cb3fa29f3586882d9ed1630d3e50e686d7310ccbc91
2018-06-06 17:12:46 +00:00
drh 0c1fa5c8c6 Merge changes from trunk, including all the 3.24.0 changes plus some
later enhancements.

FossilOrigin-Name: d7299bfeb1eb031d6106b46139715047fe759d27b34b617ca43f2cd377b8188e
2018-06-06 17:03:53 +00:00
dan 6ccceeb3bd Merge latest trunk changes into this branch.
FossilOrigin-Name: 72f39efa9b1b97a54fe35d005b48f7e8b57d6285ba42f36487f796e09c710958
2018-05-15 11:55:15 +00:00
dan 4921c159bd Merge latest trunk changes with this branch.
FossilOrigin-Name: ae86cf60b6648a7ec789e233c9b0cc826efbbb0f301140b4848dc84d691ccd4f
2018-05-15 11:45:47 +00:00
dan 1d735ad3bb Instead of just the flags byte, include the first 8 bytes of the relevant page
in an on-commit conflict log message.

FossilOrigin-Name: 52e443eb5523963a6b09be66ab1c7281930d0155bf3df13eee0ec9066dbc7f0b
2018-05-15 11:33:42 +00:00
dan b7ee5667c2 Instead of just the flags byte, include the first 8 bytes of the relevant page
in an on-commit conflict log message.

FossilOrigin-Name: e7dc03e7439f224e4bec458f27fe364f2fb1ac637328657a578ada693ae22842
2018-05-15 11:28:36 +00:00
dan a233f09330 Include the value of the "flags" byte of the relevant page in the log message
emitted when a BEGIN CONCURRENT commit conflict is detected.

FossilOrigin-Name: de19abb950e4186c491c39112a52062871ac44fc2bdc5b611e9fdfa611352716
2018-05-15 09:03:57 +00:00
dan bfbe2b8c80 Include the value of the "flags" byte of the relevant page in the log message
emitted when a BEGIN CONCURRENT commit conflict is detected.

FossilOrigin-Name: fbfa547177ac7d57dcb0775d761a7f92fc1f2263cd83e2843cd109139a8c5121
2018-05-15 08:51:05 +00:00
dan 886fac36ba Merge latest changes from trunk. Including the "ORDER BY ... LIMIT" optimization.
FossilOrigin-Name: 83a4c9f0885768ed3cbe556aed9f4709a1d7a1cd52f2e3632ade28091febee47
2018-04-26 18:01:48 +00:00
dan 4af5f425f2 Merge latest changes from trunk. Including the "ORDER BY ... LIMIT"
optimization.

FossilOrigin-Name: d8ae7ba083551f05b963f13c5164ff60257c7b615cd48d675027cb6289b463da
2018-04-26 17:54:11 +00:00
dan 8ba5d0d4f7 Update this branch with latest changes from trunk.
FossilOrigin-Name: 2d8d13e4a169cbfa957c4e978b559817a13927a85e30fe275129bf7e3fb54920
2018-04-24 19:22:49 +00:00
dan 51abf049c7 Merge latest trunk changes into this branch.
FossilOrigin-Name: b27bd799ea66748b89223044c536813d56d0aa513f077499cbf8ed9c974d7b01
2018-04-24 19:21:38 +00:00
drh ed432167dc Merge all recent trunk enchancements, and especially the autoincrement
write reduction fix.

FossilOrigin-Name: 257900494f9aac7e4181b65d0eddc9cf2e9a52dc794ad68284f085d1c402addb
2018-03-20 14:00:25 +00:00
drh cdbf6d01a4 Remove debugging puts from concurrrent3.test.
FossilOrigin-Name: 13b4975681f249831a22562d1c26958f841ea77ca779858b0bee735dd30b710e
2018-03-20 13:54:22 +00:00
drh 3a5fc00808 Merge all recent enhancements from trunk.
FossilOrigin-Name: b0c2f760a637ee973f4dcc27308eec44950e6d0a9c5ab5c828c1210c1f868efa
2018-03-20 13:52:42 +00:00
dan eb816a7547 Merge latest changes from begin-concurrent into this branch.
FossilOrigin-Name: 8ade94ba6712827c672ddad8436f8b97cc520ced122e7d0043ad3c3ba94f1ed6
2018-03-02 18:26:50 +00:00
dan 908803f5e0 Update this branch with recent checkpoint related changes from trunk.
FossilOrigin-Name: fb6b7938601505186c0b1f0df6c45630039027f85ff394d290bc2c86b16a3a07
2018-03-02 17:59:37 +00:00
dan 699bd8121e Merge latest trunk changes into this branch.
FossilOrigin-Name: 36801effa9ec67b551f58972e21794466420f10cd0420701fcd87695e6cd11ee
2018-03-02 17:40:23 +00:00
dan c7a82772dc Add extra code to log details when corruption is detected in the pointer-map
structure maintained by the b-tree layer in begin-concurrent transactions.

FossilOrigin-Name: 570233716032f258b878d52c4d5a47e07292d66fa84e3a85c0388ec15efee625
2018-02-20 21:00:45 +00:00
dan 5d9c916150 Fix problem causing free-list corruption when merging free-lists for two
concurrent transactions that have both used page X as an in-memory free-list
trunk page, where X lies past the end of the initial database images.

FossilOrigin-Name: dc0fc2aa7cbefeb5f0ba8c992fd3e9adcfb5a4d61e2321c1bd93f4d36ba9aafc
2018-01-04 18:36:39 +00:00
dan 55fba4f05c Fix a spurious SQLITE_CORRUPT error that could occur within a COMMIT of a
concurrent transaction.

FossilOrigin-Name: 50c8952c92b9f0c61935fb0df04ed1426d9e266a812071b7bf5b0215c5552757
2018-01-02 19:57:26 +00:00
dan 02bc6f8fb9 Merge latest begin-concurrent changes into this branch.
FossilOrigin-Name: 3fde0b4d05c249b9d2a54dd721185202c353cee23c0351b634cac349dc0a7b14
2017-12-12 18:17:09 +00:00
drh cb0267185c Abort on an invalid paramater to sqlite3BitvecSet().
FossilOrigin-Name: 163c870950f386f6b0bb1ff9b3886cf95ba0deed414cae75baf87621ed3528c2
2017-12-11 14:02:10 +00:00
dan 165ba4a2a5 Merge changes from trunk. This fixes the SQLITE_ENABLE_UPDATE_DELETE_LIMIT functionality so that it works with views and WITHOUT ROWID tables.
FossilOrigin-Name: d90e5f346bcf7adab26ca8dad9dfbd0fbb86604a15f2fe827f11b3faab036750
2017-11-14 20:06:15 +00:00
dan 8706485d1a Merge latest trunk changes into this branch.
FossilOrigin-Name: 7f217edab4575554f657d38e2a1bc6b3f577998fdbecb04eb200aeb8b8406581
2017-11-06 20:02:08 +00:00
dan c0f69fcec7 Allow "BEGIN CONCURRENT" transactions to modify the temp schema.
FossilOrigin-Name: 0fb6d91cea347384fc081ce4c79582b365801dd4f56f5bf2ed40922bbfca0344
2017-11-06 10:04:45 +00:00
drh 4d13729861 Fix the SQLITE_NoopUpdate #define so that it occurs under the correct
conditions.

FossilOrigin-Name: bdf791f9f7f7ab4b1871e1d3a0d5edcad46fc67d7336ae7f3f7b20e69801be8e
2017-11-01 19:30:41 +00:00
drh 56391f2769 The "PRAGMA noop_update" command now requires SQLITE_ENABLE_NOOP_UPDATE
and no longer requires SQLITE_DEBUG.

FossilOrigin-Name: 81baf67c4493468e4feb2f4990bf82d59804ce4f3149252c0e1e8c43f90d6bc1
2017-11-01 18:48:54 +00:00
dan 6cbf8e173f Add new extension "bgckpt" in ext/misc/bgckpt.c. For experimenting with
running wal2 mode checkpoints in a background thread.

FossilOrigin-Name: 63955442304052f5adddd05ccaeebe87ddc5e25af695702793518f015b4f0a87
2017-10-10 20:11:10 +00:00
dan 2df2b3d05f Merge latest trunk changes with this branch.
FossilOrigin-Name: d218d815f89cb1368fdb5e3f774b7adaaf02560a367ba0f3e54987e08dd6241a
2017-10-09 19:50:09 +00:00
dan 40927fd61e Add a header comment to wal.c describing the differences between wal and wal2
mode.

FossilOrigin-Name: 9c80cd202f1c966929b279e18f19c663912686fcf92f03b85a02b9c7e55a0fc6
2017-10-09 19:49:08 +00:00
dan 50182fa846 Ignore the *-wal2 file if the *-wal file is zero bytes in size.
FossilOrigin-Name: f7360fad51f224f347bb7d263eb89056b27461c278309e00e575a0e8898c9f40
2017-10-07 19:55:37 +00:00
dan 5d122d308f Check in test file wal2simple.test.
FossilOrigin-Name: 8932b2f1d7e6a26221ea3dea01000832b2d1eb17ac0b70ef6028f9286ae450a3
2017-10-07 13:37:04 +00:00
dan 288e98fb4b Merge latest trunk changes into this branch.
FossilOrigin-Name: 7e43517861d4ecfa86766a16a2c721377b75da78771d5ba18870dcb9626a8dce
2017-10-06 14:25:25 +00:00
dan bfcf8d6d67 Fix a bug in recovering wal2 mode databases introduced by the previous commit.
FossilOrigin-Name: 9e1502e1b650217efc361732a3dfe686caa2e6352d040c73865f1faf09bf4591
2017-10-06 14:08:39 +00:00
dan 24f1b25c7b Fix frame overwriting in wal2 mode.
FossilOrigin-Name: a4b02bc9388226da21b3837a20c6c7eb0d13854dde62b7136e04f4978528dc71
2017-10-06 13:43:42 +00:00
dan 3400e78f38 Fix test case failures on this branch.
FossilOrigin-Name: 16decc13af908087fb8aa34eeccf43e8da1b8f2e4b808028986d1ef08134c72c
2017-10-05 18:14:46 +00:00
dan 055cc1e431 Add experimental mode that uses two wal files. Activated using "PRAGMA
journal_mode = wal2".

FossilOrigin-Name: e2fc5c814cf6862d536aacb9eca66ecd31ba7e3e3033fa4c5564d533f4a18dfc
2017-10-04 20:57:14 +00:00
dan 7393c7424e Cherrypick [ec37ad6d08] into this branch. With this patch, if
SQLITE_SHARED_MAPPING is defined at build-time SQLite will use a single memory
mapping for multiple connections to the same database file within a single
process.

FossilOrigin-Name: c7a5880d6d898299b4c9414b7702cfa450aa5f7bf4ec8f417b94d2a7b6558264
2017-09-22 11:09:09 +00:00
dan 3eba5d00b6 Merge latest trunk changes into this branch.
FossilOrigin-Name: 307b802e8627c93a51e4c54851a4fab33db5061bb80e3d327ce53b127d6d511b
2017-09-22 10:49:03 +00:00
drh 03d405f046 Add the highly-experimental "PRAGMA noop_update=TRUE" command.
FossilOrigin-Name: afe45271b9c0cd379cf0beb94657e2396068c4a18f84003c4c48297760fd83ee
2017-09-12 20:09:31 +00:00
drh b25057c291 Merge recent enhancements from trunk.
FossilOrigin-Name: d53108e76307fb24ef1d8f75fe82c180f5588ad8e65eeceace92f4cecadfa452
2017-08-28 17:19:35 +00:00
dan 9c54156a9c Test BEGIN CONCURRENT transactions that consist entirely of read-only
statements.

FossilOrigin-Name: c3fe1f4b7e8dcadcb516622719d000b808effe3ad497244ba44f57d52dc2cc08
2017-08-24 10:10:28 +00:00
dan d828d0debc Add a more rigorous test case for the bug fixed by the previous commit on this
branch.

FossilOrigin-Name: 4256072399f44f48ed0856aa8112226af6feaf8676923612bde6cea239ebf920
2017-08-12 14:06:15 +00:00
dan 6aab028db0 Fix a problem allowing a conflicting transaction to be committed in the case
where more than one 32KB shared-memory page has been written to since the
transaction was started.

FossilOrigin-Name: 38dd9b50fe260d853cbc2551bc1bb60ddf5752f0456e0da3afe4cbf728c891d8
2017-08-11 21:16:23 +00:00
drh 9f16a96d4b Add another PAGERTRACE() macro to show when pages are added to Pager.pAllRead.
No impact on production builds.

FossilOrigin-Name: 11054cf5e8c24ef9ca869d558a0ca6750b56103c3b3793dc4afcef75192ea943
2017-08-11 20:22:30 +00:00
drh 741802e825 Add a PAGERTRACE() macro to log when frames are written into the WAL file.
This is for analysis only and is a no-op for production builds.

FossilOrigin-Name: 39f39e3d9a083eebe2f8dd5812d20c5b07cc17607b04b573a52fde6d00666f36
2017-08-11 19:31:53 +00:00
drh fd80ce4b10 Merge the latest enhancements from trunk.
FossilOrigin-Name: 47e716952d4a5d893b75726a2c52202cb0bc8ce1f75131e920a3ac2e202a507e
2017-08-11 19:16:55 +00:00
drh 9ee9b0f4c3 Merge recent enhancements from trunk.
FossilOrigin-Name: aafe1fec803ddf207bbcb3c669da0a9639411375fc04a52066dde4eb99c3c11b
2017-08-01 13:54:55 +00:00
drh c5856def1e Minor fix to the concurrent.test module so that it works on windows.
FossilOrigin-Name: 58553d61d199477f88ea9b45055ddaf821eca2aff1bf67c7d81fa80b23c2575a
2017-07-20 19:28:49 +00:00
drh 0a2afca904 Fix compiler warnings.
FossilOrigin-Name: 44c760d150b494ddd88382360cfcc4734fec936ed620f88b9df33ea7215c2fc2
2017-07-20 19:08:35 +00:00
drh 1af0a4e1df Fix a faulty #ifdef on the previous check-in
FossilOrigin-Name: 7355e74239fc20f3d948867902675069e6a4efd103bf734f1b24a132f6d2edd1
2017-07-20 18:56:21 +00:00
drh 7365bcd72c Get the build working with SQLITE_OMIT_CONCURRENT.
FossilOrigin-Name: a29401e924e6f82ca1c3eec98c6c0b2f1d5b9d7fba9af41a78516f21a7bab3a2
2017-07-20 18:28:33 +00:00
drh 1c6276eac6 Merge all the latest trunk enhancements.
FossilOrigin-Name: 213c61cb82d4ee51cc638fc5bb2b54cd51b963c40d088b2f01caf77032ea32fb
2017-07-20 17:47:12 +00:00
dan 827c9b785e Remove sqlite3_log() and abort() calls added to this branch to debug the
pointer-map problem ([fda22108]).

FossilOrigin-Name: 79544fc2856f30cac8b0962d00698974e8918562f09769a68264d17e1e1176fe
2017-06-10 17:23:20 +00:00
dan 4956bd5f9a Update the recent auto-vacuum fix so that it works for the in-memory
pointer-map structure used by this branch.

FossilOrigin-Name: 8e311a6dba202e8733830d8f31b8f0ce11eaefb3a0ab5e5e95ac0d2e5136043b
2017-06-08 16:23:55 +00:00
dan 63624a0ab8 Merge 3.19.3 changes with this branch.
FossilOrigin-Name: e2d38d51a9cf1c3dfef742507ec76e3d35853bd09b0d09bf2d404c4b036a184d
2017-06-08 16:10:41 +00:00
dan dc1bf28ad4 Merge changes for version 3.19.0 with this branch.
FossilOrigin-Name: 0f59bb94d5e8e16a47c53ef0d6138a35c87230ad384b58008e8e7127780446d8
2017-06-08 15:54:02 +00:00
drh d6396e9b16 Version 3.19.3
FossilOrigin-Name: 0ee482a1e0eae22e08edc8978c9733a96603d4509645f348ebf55b579e89636b
2017-06-08 14:26:16 +00:00
drh d4d01a689d Fix an issue with OPT_FEATURE_FLAGS in configure.ac.
FossilOrigin-Name: 97b5c4a53d0af3792c5ae5ceafdb02245f841c2bd4d79bcfd0ffdf6c5ff27c9e
2017-06-08 11:32:19 +00:00
drh a096fcc73c Increase the version number to 3.19.3.
FossilOrigin-Name: 903fff53b3f02cbc011c4cd47d841f45a447746fdf3bcb01202e27c3f2b57589
2017-06-08 11:27:05 +00:00
drh 50179f91f3 Ensure pointer map entries are always added when a row that does use overflow
pages replaces one that does not in an auto-vacuum database. Fix for
[fda22108].

FossilOrigin-Name: 9478106ca963e3ae5cfe59da40301c5a0a07494d03e656b5eb10ab45e441b870
2017-06-08 11:26:13 +00:00
dan 35dcd0e892 Extend even further the logging designed to find problems in the pointer-map
structure. Call abort() to dump a core as soon as such a problem is seen.

FossilOrigin-Name: f131677dcb4937e0dd62626afa91756aa28079e92acd6e9e127f6f676aa334f9
2017-06-02 09:31:27 +00:00
dan 7f002db7e3 Generate extra log messages in response to irregularites in the pointer-map
used by "BEGIN CONCURRENT" transactions.

FossilOrigin-Name: f7e3e2bc88f110d9282ce5d2fa58580c585faeb57cb707253f05001e5f4bd91b
2017-05-31 17:06:13 +00:00
dan 995b2457e2 Instead of a root page number, log the object (table or index) name if a page
level locking conflict is detected.

FossilOrigin-Name: 9ad846e57bd427adc7c29768cabca18905f7f978168e0642a5917d894fda8bfd
2017-05-29 19:23:56 +00:00
dan 7fff2e1cb9 Enhance the log messages emitted when a page conflict is detected.
FossilOrigin-Name: 92618492b048867af38922825f3d094eaaa2dd919b1ed2f7372483cc53f892bf
2017-05-29 14:27:37 +00:00
dan e3b047b3d7 Adjust the bitvec related sqlite3_log messages added by [9527089b].
FossilOrigin-Name: a7e0e7a4835314a04e065839a7f88b074eda795da2b2da5741b2afc096421a32
2017-05-26 18:18:51 +00:00
dan 7037787d30 Add extra test cases for deferred page allocation.
FossilOrigin-Name: 9df0195780306a26709e87daabf9d426856aebf2082060b98818f2d48c7472ec
2017-05-26 16:51:15 +00:00
dan 606f718480 Fix a problem with deferred page allocation in transactions that revert page
allocations by savepoint rollbacks.

FossilOrigin-Name: a4a3bbe64690856403642352f36e664a5a7fba686463a63446c6ada99df4e89f
2017-05-26 16:15:05 +00:00
dan e3c3be8511 Fix a problem with the deferred page allocation on this branch that could
occur when the database file is just slightly smaller than the PENDING_BYTE
page offset.

FossilOrigin-Name: 47a7dd92355ffd74645cf8da8186d5799c05a95907877a09c28de41135deeede
2017-05-25 21:02:00 +00:00
dan aa59505ae8 Add experimental new API sqlite3_wal_info().
FossilOrigin-Name: 5b9d498f6e9de6ee2ab86370c02c28a2a8b83d717b96d23b1fc52107677e45a2
2017-05-23 19:23:45 +00:00
dan 55b36d5c07 Invoke sqlite3_log() in response to irregularities surrounding the
Pager.pAllRead bit-vector.

FossilOrigin-Name: 9527089b7aa3695cd577f31b263b4777e9bd62dbbc1bd3af892c570e52e8c3a1
2017-05-19 19:57:15 +00:00
dan 6dbb452b38 Fix a problem on this branch causing some page-level read-locks to be omitted.
FossilOrigin-Name: 0eed152162b4721f7aaba8b480426476978a869e9578f100fca7b1d32942fe1a
2017-05-18 20:47:31 +00:00
drh 2ecf1ec8c9 Merge the latest changes from trunk.
FossilOrigin-Name: 14ea84003600ada6f4605e47028cb72fe621a14bcce3637ad48b1512dcce41cd
2017-05-15 17:34:23 +00:00
drh 9a89254e7a Merge changes from the 3.18.0 release.
FossilOrigin-Name: 785c37d9dbdba9f3deedecbdd72f6761428ebfb58e9b9393b9ca4feab3f66f02
2017-03-30 20:35:30 +00:00
drh fe329a7370 Version 3.18.0 Release Candidate
FossilOrigin-Name: 424a0d380332858ee55bdebc4af3789f74e70a2b3ba1cf29d84b9b4bcf3e2e37
2017-03-28 18:48:43 +00:00
drh 4d55d35df6 Merge all recent enhancements from trunk.
FossilOrigin-Name: fd5676fe7f55b3d4fa15ca119af7c064c6f9d053affdfd5e748785e300affbeb
2017-03-24 15:09:47 +00:00
dan 246e1d8faf Merge the "changebatch" functionality into this branch.
FossilOrigin-Name: 50fb1eb368f14e4ddce2da875601c227f9f937fd
2017-01-09 07:00:41 +00:00
dan 0195d8ef5d Disable the optimization from [8cb8516d] (omit TableLock instructions for
non-sharable databases) on this branch. This branch uses the TableLock
instructions to ensure that the db schema is not written from within a BEGIN
CONCURRENT transaction.

FossilOrigin-Name: c8ca3e0a8d82dbb077551c2d952cb2043f78333b
2017-01-09 06:53:00 +00:00
dan e32c7c2b64 Upgrade this branch to 3.16 plus the various fixes that appeared after its
release.

FossilOrigin-Name: d0e212d08f82dfb5e42a156b3e2bb03dd8e21258
2017-01-09 06:33:50 +00:00
dan 0ba9725489 Merge latest trunk changes into this branch.
FossilOrigin-Name: 391344d88a284f92c59b8d96a315b69292641de0
2016-12-07 07:46:50 +00:00
dan ff8e42e2ca Add the sqlite3changebatch_db() API.
FossilOrigin-Name: bee44ebc532f37e3fe61c18878e0d3db06805190
2016-08-24 19:14:59 +00:00
dan a8dee8df64 Add further tests for changebatch. And a fix to prevent a changeset from conflicting with itself.
FossilOrigin-Name: 506d6ff4b64c72d4ca65f0d15e1fdf8a132556bc
2016-08-23 19:02:21 +00:00
drh 7d931b985a Fix harmless compiler warnings in changebatch.
FossilOrigin-Name: a721a738184d914fcde3f5684099984a9373dff3
2016-08-23 18:09:37 +00:00
dan 8bbf544747 Add a new method to sessions - sqlite3sessions_fullchangeset() - to return a changeset that always contains values for all old.* fields. Update changebatch to use these values to more reliably detect multi-column UNIQUE constraint violations.
FossilOrigin-Name: efa761b2f509844b9212dd20bf0d082c6338e83f
2016-08-23 17:02:28 +00:00
dan a7d16a5cd1 Add a couple of extra tests to changebatch1.test.
FossilOrigin-Name: 207d970b7956c38af42c389b91a741a68b2c4eec
2016-08-22 21:01:39 +00:00
dan c18d304066 Add an experimental module to detect conflicts between sessions changesets.
FossilOrigin-Name: 0c9fd6b723041955b5182caa430312e5124fdc83
2016-08-22 20:49:06 +00:00
drh 3d4ad18cbe Merge version 3.14 plus some subsequent patches (including the page-cache
performance patch) from trunk.

FossilOrigin-Name: d9f8918c5b7b6c8540b3f433142e1b4aa4433885
2016-08-11 19:12:25 +00:00
dan 21ac1de75b Fix a typo in test program bc_test1.c.
FossilOrigin-Name: 2c61b7ab1808d5576a21fea84c3f414db51abfa5
2016-07-06 08:32:53 +00:00
dan 96b9dc9b36 Minor tweaks to the bc_test1 test program.
FossilOrigin-Name: d0d0bab4e92402b6af98366be6e8955588613a51
2016-05-30 05:45:32 +00:00
dan adeb47e57b Add further instrumentation to the bc_test1.c test app.
FossilOrigin-Name: 5528de4a53c19557798b6169e1d738f1a301e131
2016-05-26 20:52:15 +00:00
dan 12891cc492 Use pthreads mutexes and conditions to synchronize threads in bc_test1.
FossilOrigin-Name: f33aa76f074d8686a5a5c0edecabb71cb259c48d
2016-05-21 18:50:56 +00:00
dan 820fc4933b Add options to bc_test1.c to make it more flexible.
FossilOrigin-Name: ec6ef5f2c2dc18e1a19c205f365f4071f0870b68
2016-05-07 18:02:53 +00:00
dan b00083429a Add test/bc_test1.c, for testing the degree of concurrency provided by this branch under various conditions.
FossilOrigin-Name: 128c7eaed5479b615d75ebce1d781ea661e0fb2d
2016-05-06 21:04:47 +00:00
drh b0b18f6dca Merge the latest enhancements from trunk.
FossilOrigin-Name: 91e5c07eaf884d3598df8eb54f0910a80df48397
2016-04-29 16:01:18 +00:00
drh 76cb74da4d Merge all recent enhancements from trunk.
FossilOrigin-Name: 1f709fbf931e4d75848fc90532e2bc67ccd47cd4
2016-04-20 12:16:05 +00:00
drh db454d72b6 Merge 3.12.0 changes.
FossilOrigin-Name: d7381efff47c0a2f307478f196d03df7534f19e3
2016-03-29 10:36:07 +00:00
drh 99b14894eb Merge all recent changes from trunk.
FossilOrigin-Name: 8ee7d346c1d9beedd02bf11d0a45810985a45480
2016-03-21 15:54:46 +00:00
drh 03206c433c Merge the latest enhancements from trunk.
FossilOrigin-Name: a7978ab6d26b865f55c6b3e86ba3a24777a3c1f0
2016-03-16 19:34:45 +00:00
drh 93f88ebbb1 Merge version 3.11.0
FossilOrigin-Name: c393ddc71a041a711f8eaa6fbd75371309df5602
2016-02-15 17:44:45 +00:00
drh faf010d77a Merge PRAGMA synchronous=EXTRA and OOM performance enhancements from trunk.
FossilOrigin-Name: 332c8036ca8a97087ced210e322c8a91be2a9036
2016-02-05 14:34:51 +00:00
drh 7c9ecc62c5 Merge recent enhancements from trunk.
FossilOrigin-Name: 347f6a80c2aa6dde9b58d28069156db22cd21b6d
2016-02-02 02:53:49 +00:00
drh 04b9cf1e7e Merge recent enhancements from trunk.
FossilOrigin-Name: 5520f600eade720f465575ab1661dcddaba5adcf
2016-01-20 12:18:31 +00:00
dan f687ba597f Merge the latest enhancements and fixes from trunk.
FossilOrigin-Name: fccc5f20c360fa17c3692a9f1fc1d16612632a7f
2016-01-14 15:46:31 +00:00
drh 76ff0699bb Merge changes for version 3.10.0.
FossilOrigin-Name: e1d2ffc06d47b550ca9447a71443dea08b98d1ad
2016-01-06 15:49:19 +00:00
drh 607d4c2184 Merge recent trunk enhancements.
FossilOrigin-Name: 6bea792c3d71be34ccb9c14a0df8a5244d2d3678
2016-01-01 14:14:55 +00:00
drh 8c9ea487d0 Merge recent enhancements from trunk.
FossilOrigin-Name: cb22efaf50d83d9a73fdf8d986e6ea2fc6500cfb
2015-12-16 19:55:57 +00:00
drh 1d9497a0d0 Merge recent enhancements from trunk.
FossilOrigin-Name: 9130661a786e4c158f15103be57467a5cc03875a
2015-12-02 20:53:14 +00:00
drh 31441fb50b Merge all the latest enhancements and fixes from trunk.
FossilOrigin-Name: 41c8b8e39bc0483cfbc0b4bfcc8ef8b2737a70a9
2015-11-20 13:49:01 +00:00
drh f12cc2a68c Merge the sqlite3_db_cacheflush() enhancements and other changes from trunk.
FossilOrigin-Name: f2cde4cfc58cc372f59ae274bf0c2f7cf6e7ddf9
2015-10-30 17:17:12 +00:00
drh cd9491ca2f Merge the 3.9.1 updates from trunk.
FossilOrigin-Name: bf866e6c0d95d48744c86ff0c0be9b2653629a18
2015-10-16 20:55:12 +00:00
drh e8a1ea6d8b Merge in the 3.9.0 changes from trunk.
FossilOrigin-Name: 5c3a2a6ed64f4d0e10238720bc2e4ae3af3eded3
2015-10-15 07:44:58 +00:00
drh 0faad42d59 Merge all recent trunk enhancements and fixes into the begin-concurrent branch.
FossilOrigin-Name: c63c1e15f8cc8981aa4dbec82d510ed70cd09797
2015-09-24 15:17:54 +00:00
drh 411547ec6c Merge enhancements from trunk.
FossilOrigin-Name: fc4d1de8aeb39a4c0ea9275a0bd2447535f1a955
2015-09-15 19:16:47 +00:00
drh 9a5a469c7a Merge parser enhancements and other improvements and bug fixes from trunk.
FossilOrigin-Name: 9cf3e51bcce1268dbb22cc8fa77160db3cb72746
2015-09-07 20:22:22 +00:00
drh 573ebef187 Merge the latest trunk changes, and especially the fix for allowing
strings as identifiers in CREATE INDEX statements.

FossilOrigin-Name: a9b84885aa572b7f92e5aafa246af328d13e3e6e
2015-09-04 17:22:03 +00:00
drh 144cd3df05 Remove the EXCLUSIVE and CONCURRENT tokens from the tokenizer. Let the
BEGIN statement be followed by an ID, but throw a syntax error if the ID
is anything other than EXCLUSIVE or CONCURRENT.

FossilOrigin-Name: c0bf92eca4d3d4666e8a0476ef30fa8123de1cb0
2015-09-04 16:39:16 +00:00
drh 3561ec2ede Merge performance enhancements from trunk. This branch now runs (slightly)
faster than the 3.8.11.1 release, though still slightly slower than trunk.

FossilOrigin-Name: c490bfb150c66763226a35e30ba289abbf29906d
2015-09-03 20:52:44 +00:00
drh 01be463eeb Changes from ENABLE_CONCURRENT (default off) to OMIT_CONCURRENT (default on).
This is not a clear-cut decision and might be changed back.

FossilOrigin-Name: f8ae9bfd05abc35293ad6bc62ab1bdbe357d964e
2015-09-03 15:17:12 +00:00
drh a62b6bd93c Merge trunk optimizations.
FossilOrigin-Name: 71e7299e8d501618b10f8c1f78572789b1b6853a
2015-09-03 14:04:05 +00:00
drh f56af5ee2d Merge the latest trunk enhancements.
FossilOrigin-Name: 3dea047465fa8e3031046a33016b6ed760336ddc
2015-09-01 18:01:16 +00:00
drh 3f531da564 Fixes so that it builds without warnings both with and without
SQLITE_ENABLE_CONCURRENT.

FossilOrigin-Name: 5ed2a445a164f0f0c2669c6681ea76618e639961
2015-09-01 17:48:54 +00:00
dan 7b8996c878 Merge latest trunk changes with this branch.
FossilOrigin-Name: 57bc0194f41dbcd2c343e665e7af475cd4dd7328
2015-08-28 09:27:51 +00:00
dan c8a9d15887 Add header comments for new methods in pager.c.
FossilOrigin-Name: 437c7e219d3240767a28f73487bc26c3be3044b3
2015-08-27 19:57:16 +00:00
dan 6b3e51bd33 Add test cases for concurrent transactions and long-lived SELECT statements.
FossilOrigin-Name: fd4798cb7af263409c20d3cf81236b830bd68570
2015-08-27 19:22:56 +00:00
dan 987f821f79 Fix a problem whereby concurrent transactions would not consider pages read by the transaction before the first write statement.
FossilOrigin-Name: fc17f73170a27c2fe511ed6b6d488535c4e35bae
2015-08-27 17:42:38 +00:00
dan f6cf5ea790 Fix an assert() in pager.c that could fail in a concurrent transaction.
FossilOrigin-Name: 69394ddaa2bc9d26477b4359c676c598b733ac9f
2015-08-26 18:54:45 +00:00
dan de36c76a23 Fix a problem allowing some conflicting transactions to be committed.
FossilOrigin-Name: a0566382d564ca17fd13475a44fed8f714742d97
2015-08-26 18:02:20 +00:00
dan fef3410f7f Add miscellaneous test cases for concurrent transactions.
FossilOrigin-Name: 779b1d0e17bc54062b2b09cdbf94e9e2f4bae4f7
2015-08-25 19:10:29 +00:00
dan 57888f7300 If "PRAGMA integrity_check" is run while the database is being written by a CONCURRENT transaction, do not consider unreferenced pages to be an error. They may be part of the free-page list, which is not visible at the b-tree layer when running a CONCURRENT transaction.
FossilOrigin-Name: f32b57b49311693eb0c0c9f6f14859e7b1fa93d8
2015-08-25 17:16:33 +00:00
dan 4073b26a20 Test that if a corrupt wal-index header is encountered when attempting to commit a concurrent transaction, SQLITE_BUSY_SNAPSHOT is returned to the caller.
FossilOrigin-Name: c746e0bd20cb136eed2b691f326657d266e2f1ed
2015-08-25 16:01:04 +00:00
dan ac0a42233a Fix a segfault that could occur following an OOM condition in the concurrent transaction code.
FossilOrigin-Name: 231b5880228cf01efe3981bc8be3150d79b422e5
2015-08-25 14:37:39 +00:00
dan 3c40ed4281 Merge latest trunk changes with this branch.
FossilOrigin-Name: 3e7d6dd62dfa63d7def00bd00ac055a606a0c80d
2015-08-25 11:16:02 +00:00
mistachkin 0cd4f69c0c Remove duplicated line of code.
FossilOrigin-Name: 47280f2a2b7cc83bf11ab86284204f565c278c55
2015-08-24 22:06:02 +00:00
dan bf3cf57e15 Change "BEGIN UNLOCKED" to "BEGIN CONCURRENT".
FossilOrigin-Name: ba1ab858e2997c88dd7eee6e6893a8616d85c665
2015-08-24 19:56:04 +00:00
dan f5e89dba9d Fix handling of attempts to modify the database schema, application_id or user_version within an UNLOCKED transaction.
FossilOrigin-Name: 5b9f272113d21fd606903509d6f830fe60fac039
2015-08-24 19:08:10 +00:00
dan 0408529b48 Fix compilation without SQLITE_ENABLE_UNLOCKED. Also other code organization issues.
FossilOrigin-Name: 041135575417201bbcf0544cc69dcb7369c7fb34
2015-08-24 16:00:08 +00:00
dan 5cf03728a6 Consolidate two blocks of similar code in btreeFixUnlocked().
FossilOrigin-Name: 701302b4bd62ca7aefe643eac096a0ee672a62fa
2015-08-24 10:05:03 +00:00
dan 8b994f29a5 Merge trunk changes with this branch.
FossilOrigin-Name: 876810c28b3ad573ae46050ec699ef7eea4e313d
2015-08-24 06:44:17 +00:00
dan 654a965367 Fix another problem involving unlocked transactions and wal-file restarts.
FossilOrigin-Name: 4460764ea8fc948fe02f0a09476857839b3aa1ae
2015-08-24 06:43:25 +00:00
dan 0c52b373a0 Fix a problem to do with detecting unlocked transaction conflicts if another client restarts the wal while the transaction is running.
FossilOrigin-Name: e3968b256282d8c0a87c26667b1ba02faf7a5a17
2015-08-22 20:32:39 +00:00
dan f5cebf71fe Add further tests for deferred page allocation. And fixes for the same.
FossilOrigin-Name: ed0a31be726e60115a5dd73d4ed580201b400ab7
2015-08-22 17:28:55 +00:00
dan c299a11c02 Merge further trunk changes.
FossilOrigin-Name: c2327a3b8e5d604ab948b1e9f6cfc401429e51db
2015-08-22 07:56:49 +00:00
dan 8c5847a257 Merge trunk changes into this branch.
FossilOrigin-Name: 9021f7369f4b32db6126b3dc0ac75d11039625be
2015-08-22 07:55:08 +00:00
dan b87b25f219 Fix many minor issues in the unlocked transaction code.
FossilOrigin-Name: 53aaeea6c98f82f3b55d4b3ab139ee41b727243c
2015-08-21 20:11:23 +00:00
dan 572a21c8e9 When committing an UNLOCKED transaction, try to move pages allocated at the end of the file to free slots within the file (like an incremental-vacuum operation does).
FossilOrigin-Name: 069679162d8d50e9731831e658aa58f280dbb3e7
2015-08-21 18:55:22 +00:00
dan 70af25d03d Fix a problem with UNLOCKED transactions that free pages allocated within the same transaction.
FossilOrigin-Name: 227bb8a1815c4dc6084970f06b0a6bfccdff3fd2
2015-08-21 17:57:16 +00:00
dan cec0beebb3 Merge trunk changes with this branch.
FossilOrigin-Name: deaf3b18569025d8d168da29fae76142cfffe9e5
2015-08-21 16:22:04 +00:00
dan 64b310ed1a Add extra tests and a fix for rollbacks of UNLOCKED transactions.
FossilOrigin-Name: 82cd837e72ed4cf5821be717369694be29a2997e
2015-08-21 14:21:22 +00:00
dan 699bdf056b Fix a problem causing corruption when an UNLOCKED transaction is rolled back.
FossilOrigin-Name: 7c36147846484c96023939864417b5624f3bc5f8
2015-08-20 20:25:56 +00:00
dan 7b3d71e9cc When committing an unlocked transaction, relocate newly allocated database pages within the file to avoid conflicting with committed transactions. There are lots of things still to fix in this code.
FossilOrigin-Name: 3bbc31d515ba9fc920c5cbc7059d3eb1ba9c7f8e
2015-08-19 20:27:05 +00:00
dan 1a9cde3ba9 Handle writes to auto-vacuum databases within UNLOCKED transactions in the same way as for non-UNLOCKED transactions.
FossilOrigin-Name: de1ea450db33b140b11af5b801ea6a15875e774e
2015-08-15 18:16:46 +00:00
dan 773d2d6c09 Only allow UNLOCKED transactions to commit if none of the pages read by the transaction have been modified since it was opened.
FossilOrigin-Name: 0b9718426e44df092850c5d095ce1b84a1e567cf
2015-07-29 12:14:28 +00:00
dan 37d36205f3 Add some test cases and fix some small problems with BEGIN UNLOCKED transactions.
FossilOrigin-Name: 6da0e962ad2aa5e52c1f1b5c3dbf77a2cb16ac2d
2015-07-28 16:46:49 +00:00
dan 3d39434c36 Add an experimental "BEGIN UNLOCKED" command.
FossilOrigin-Name: 8079421604dbd40d03471dad6d12115119b554c2
2015-07-27 19:31:45 +00:00
1119 changed files with 160202 additions and 76552 deletions
+351 -275
View File
@@ -35,7 +35,7 @@ TCC += -I${TOP}/ext/fts3 -I${TOP}/ext/async -I${TOP}/ext/session
TCC += -I${TOP}/ext/userauth
# Define this for the autoconf-based build, so that the code knows it can
# include the generated config.h
# include the generated sqlite_cfg.h
#
TCC += -D_HAVE_SQLITE_CONFIG_H -DBUILD_sqlite
@@ -57,6 +57,7 @@ LIBTCL = @TCL_LIB_SPEC@
#
READLINE_FLAGS = -DHAVE_READLINE=@TARGET_HAVE_READLINE@ @TARGET_READLINE_INC@
READLINE_FLAGS += -DHAVE_EDITLINE=@TARGET_HAVE_EDITLINE@
READLINE_FLAGS += -DHAVE_LINENOISE=@TARGET_HAVE_LINENOISE@
# The library that programs using readline() must link against.
#
@@ -82,9 +83,13 @@ TEMP_STORE = -DSQLITE_TEMP_STORE=@TEMP_STORE@
# based on configuration. (-DSQLITE_OMIT*, -DSQLITE_ENABLE*).
# The same set of OMIT and ENABLE flags should be passed to the
# LEMON parser generator and the mkkeywordhash tool as well.
OPT_FEATURE_FLAGS = @OPT_FEATURE_FLAGS@
#
# Add OPTIONS=... on the command line to append additional options
# to the OPT_FEATURE_FLAGS.
#
OPT_FEATURE_FLAGS = @OPT_FEATURE_FLAGS@ $(OPTIONS)
TCC += $(OPT_FEATURE_FLAGS)
TCC += $(OPT_FEATURE_FLAGS)
# Add in any optional parameters specified on the make commane line
# ie. make "OPTS=-DSQLITE_ENABLE_FOO=1 -DSQLITE_OMIT_FOO=1".
@@ -116,14 +121,15 @@ HAVE_TCL = @HAVE_TCL@
#
TCLSH_CMD = @TCLSH_CMD@
# Additional options when running tests using testrunner.tcl
# This is usually either blank, or else --status
#
TSTRNNR_OPTS = @TSTRNNR_OPTS@
# Where do we want to install the tcl plugin
#
TCLLIBDIR = @TCLLIBDIR@
# The suffix used on shared libraries. Ex: ".dll", ".so", ".dylib"
#
SHLIB_SUFFIX = @TCL_SHLIB_SUFFIX@
# If gcov support was enabled by the configure script, add the appropriate
# flags here. It's not always as easy as just having the user add the right
# CFLAGS / LDFLAGS, because libtool wants to use CFLAGS when linking, which
@@ -184,7 +190,7 @@ LIBOBJS0 = alter.lo analyze.lo attach.lo auth.lo \
main.lo malloc.lo mem0.lo mem1.lo mem2.lo mem3.lo mem5.lo \
memdb.lo memjournal.lo \
mutex.lo mutex_noop.lo mutex_unix.lo mutex_w32.lo \
notify.lo opcodes.lo os.lo os_unix.lo os_win.lo \
notify.lo opcodes.lo os.lo os_kv.lo os_unix.lo os_win.lo \
pager.lo parse.lo pcache.lo pcache1.lo pragma.lo prepare.lo printf.lo \
random.lo resolve.lo rowset.lo rtree.lo \
sqlite3session.lo select.lo sqlite3rbu.lo status.lo stmt.lo \
@@ -257,6 +263,7 @@ SRC = \
$(TOP)/src/os.h \
$(TOP)/src/os_common.h \
$(TOP)/src/os_setup.h \
$(TOP)/src/os_kv.c \
$(TOP)/src/os_unix.c \
$(TOP)/src/os_win.c \
$(TOP)/src/os_win.h \
@@ -314,24 +321,6 @@ SRC = \
# Source code for extensions
#
SRC += \
$(TOP)/ext/fts1/fts1.c \
$(TOP)/ext/fts1/fts1.h \
$(TOP)/ext/fts1/fts1_hash.c \
$(TOP)/ext/fts1/fts1_hash.h \
$(TOP)/ext/fts1/fts1_porter.c \
$(TOP)/ext/fts1/fts1_tokenizer.h \
$(TOP)/ext/fts1/fts1_tokenizer1.c
SRC += \
$(TOP)/ext/fts2/fts2.c \
$(TOP)/ext/fts2/fts2.h \
$(TOP)/ext/fts2/fts2_hash.c \
$(TOP)/ext/fts2/fts2_hash.h \
$(TOP)/ext/fts2/fts2_icu.c \
$(TOP)/ext/fts2/fts2_porter.c \
$(TOP)/ext/fts2/fts2_tokenizer.h \
$(TOP)/ext/fts2/fts2_tokenizer.c \
$(TOP)/ext/fts2/fts2_tokenizer1.c
SRC += \
$(TOP)/ext/fts3/fts3.c \
$(TOP)/ext/fts3/fts3.h \
@@ -377,7 +366,7 @@ SRC += \
opcodes.h \
parse.c \
parse.h \
config.h \
sqlite_cfg.h \
shell.c \
sqlite3.h
@@ -390,7 +379,6 @@ TESTSRC = \
$(TOP)/src/test4.c \
$(TOP)/src/test5.c \
$(TOP)/src/test6.c \
$(TOP)/src/test7.c \
$(TOP)/src/test8.c \
$(TOP)/src/test9.c \
$(TOP)/src/test_autoext.c \
@@ -419,7 +407,6 @@ TESTSRC = \
$(TOP)/src/test_quota.c \
$(TOP)/src/test_rtree.c \
$(TOP)/src/test_schema.c \
$(TOP)/src/test_server.c \
$(TOP)/src/test_superlock.c \
$(TOP)/src/test_syscall.c \
$(TOP)/src/test_tclsh.c \
@@ -433,7 +420,13 @@ TESTSRC = \
$(TOP)/ext/fts3/fts3_term.c \
$(TOP)/ext/fts3/fts3_test.c \
$(TOP)/ext/session/test_session.c \
$(TOP)/ext/rbu/test_rbu.c
$(TOP)/ext/session/sqlite3changebatch.c \
$(TOP)/ext/recover/sqlite3recover.c \
$(TOP)/ext/recover/dbdata.c \
$(TOP)/ext/recover/test_recover.c \
$(TOP)/ext/intck/test_intck.c \
$(TOP)/ext/intck/sqlite3intck.c \
$(TOP)/ext/rbu/test_rbu.c
# Statically linked extensions
#
@@ -441,7 +434,9 @@ TESTSRC += \
$(TOP)/ext/expert/sqlite3expert.c \
$(TOP)/ext/expert/test_expert.c \
$(TOP)/ext/misc/amatch.c \
$(TOP)/ext/misc/bgckpt.c \
$(TOP)/ext/misc/appendvfs.c \
$(TOP)/ext/misc/basexx.c \
$(TOP)/ext/misc/carray.c \
$(TOP)/ext/misc/cksumvfs.c \
$(TOP)/ext/misc/closure.c \
@@ -461,10 +456,12 @@ TESTSRC += \
$(TOP)/ext/misc/percentile.c \
$(TOP)/ext/misc/prefixes.c \
$(TOP)/ext/misc/qpvtab.c \
$(TOP)/ext/misc/randomjson.c \
$(TOP)/ext/misc/regexp.c \
$(TOP)/ext/misc/remember.c \
$(TOP)/ext/misc/series.c \
$(TOP)/ext/misc/spellfix.c \
$(TOP)/ext/misc/stmtrand.c \
$(TOP)/ext/misc/totype.c \
$(TOP)/ext/misc/unionvtab.c \
$(TOP)/ext/misc/wholenumber.c \
@@ -492,6 +489,7 @@ TESTSRC2 = \
$(TOP)/src/main.c \
$(TOP)/src/mem5.c \
$(TOP)/src/os.c \
$(TOP)/src/os_kv.c \
$(TOP)/src/os_unix.c \
$(TOP)/src/os_win.c \
$(TOP)/src/pager.c \
@@ -503,6 +501,7 @@ TESTSRC2 = \
$(TOP)/src/pcache1.c \
$(TOP)/src/select.c \
$(TOP)/src/tokenize.c \
$(TOP)/src/treeview.c \
$(TOP)/src/utf.c \
$(TOP)/src/util.c \
$(TOP)/src/vdbeapi.c \
@@ -554,18 +553,10 @@ HDR = \
$(TOP)/src/vdbeInt.h \
$(TOP)/src/vxworks.h \
$(TOP)/src/whereInt.h \
config.h
sqlite_cfg.h
# Header files used by extensions
#
EXTHDR += \
$(TOP)/ext/fts1/fts1.h \
$(TOP)/ext/fts1/fts1_hash.h \
$(TOP)/ext/fts1/fts1_tokenizer.h
EXTHDR += \
$(TOP)/ext/fts2/fts2.h \
$(TOP)/ext/fts2/fts2_hash.h \
$(TOP)/ext/fts2/fts2_tokenizer.h
EXTHDR += \
$(TOP)/ext/fts3/fts3.h \
$(TOP)/ext/fts3/fts3Int.h \
@@ -609,7 +600,8 @@ TESTOPTS = --verbose=file --output=test-out.txt
# Extra compiler options for various shell tools
#
SHELL_OPT = -DSQLITE_ENABLE_FTS4
SHELL_OPT += -DSQLITE_DQS=0
SHELL_OPT += -DSQLITE_ENABLE_FTS4
#SHELL_OPT += -DSQLITE_ENABLE_FTS5
SHELL_OPT += -DSQLITE_ENABLE_RTREE
SHELL_OPT += -DSQLITE_ENABLE_EXPLAIN_COMMENTS
@@ -619,24 +611,68 @@ SHELL_OPT += -DSQLITE_ENABLE_DBPAGE_VTAB
SHELL_OPT += -DSQLITE_ENABLE_DBSTAT_VTAB
SHELL_OPT += -DSQLITE_ENABLE_BYTECODE_VTAB
SHELL_OPT += -DSQLITE_ENABLE_OFFSET_SQL_FUNC
FUZZERSHELL_OPT =
FUZZCHECK_OPT = -DSQLITE_ENABLE_MEMSYS5 -DSQLITE_OSS_FUZZ
FUZZCHECK_OPT += -DSQLITE_MAX_MEMORY=50000000
FUZZCHECK_OPT += -DSQLITE_PRINTF_PRECISION_LIMIT=1000
FUZZCHECK_OPT += -DSQLITE_ENABLE_FTS4
FUZZCHECK_OPT += -DSQLITE_ENABLE_FTS3_PARENTHESIS
FUZZCHECK_OPT += -DSQLITE_ENABLE_FTS5
FUZZCHECK_OPT += -DSQLITE_ENABLE_RTREE
FUZZCHECK_OPT += -DSQLITE_ENABLE_GEOPOLY
FUZZCHECK_OPT += -DSQLITE_ENABLE_DBSTAT_VTAB
FUZZCHECK_OPT += -DSQLITE_ENABLE_BYTECODE_VTAB
FUZZCHECK_SRC = $(TOP)/test/fuzzcheck.c $(TOP)/test/ossfuzz.c $(TOP)/test/fuzzinvariants.c
SHELL_OPT += -DSQLITE_STRICT_SUBTYPE=1
FUZZERSHELL_OPT =
FUZZCHECK_OPT += -I$(TOP)/test
FUZZCHECK_OPT += -I$(TOP)/ext/recover
FUZZCHECK_OPT += \
-DSQLITE_OSS_FUZZ \
-DSQLITE_ENABLE_BYTECODE_VTAB \
-DSQLITE_ENABLE_DBPAGE_VTAB \
-DSQLITE_ENABLE_DBSTAT_VTAB \
-DSQLITE_ENABLE_BYTECODE_VTAB \
-DSQLITE_ENABLE_DESERIALIZE \
-DSQLITE_ENABLE_EXPLAIN_COMMENTS \
-DSQLITE_ENABLE_FTS3_PARENTHESIS \
-DSQLITE_ENABLE_FTS4 \
-DSQLITE_ENABLE_FTS5 \
-DSQLITE_ENABLE_GEOPOLY \
-DSQLITE_ENABLE_MATH_FUNCTIONS \
-DSQLITE_ENABLE_MEMSYS5 \
-DSQLITE_ENABLE_NORMALIZE \
-DSQLITE_ENABLE_OFFSET_SQL_FUNC \
-DSQLITE_ENABLE_PREUPDATE_HOOK \
-DSQLITE_ENABLE_RTREE \
-DSQLITE_ENABLE_SESSION \
-DSQLITE_ENABLE_STMTVTAB \
-DSQLITE_ENABLE_UNKNOWN_SQL_FUNCTION \
-DSQLITE_ENABLE_STAT4 \
-DSQLITE_ENABLE_STMT_SCANSTATUS \
-DSQLITE_MAX_MEMORY=50000000 \
-DSQLITE_MAX_MMAP_SIZE=0 \
-DSQLITE_OMIT_LOAD_EXTENSION \
-DSQLITE_PRINTF_PRECISION_LIMIT=1000 \
-DSQLITE_PRIVATE="" \
-DSQLITE_STRICT_SUBTYPE=1 \
-DSQLITE_STATIC_RANDOMJSON
FUZZCHECK_SRC += $(TOP)/test/fuzzcheck.c
FUZZCHECK_SRC += $(TOP)/test/ossfuzz.c
FUZZCHECK_SRC += $(TOP)/test/fuzzinvariants.c
FUZZCHECK_SRC += $(TOP)/ext/recover/dbdata.c
FUZZCHECK_SRC += $(TOP)/ext/recover/sqlite3recover.c
FUZZCHECK_SRC += $(TOP)/test/vt02.c
FUZZCHECK_SRC += $(TOP)/ext/misc/percentile.c
FUZZCHECK_SRC += $(TOP)/ext/misc/randomjson.c
DBFUZZ_OPT =
ST_OPT = -DSQLITE_OS_KV_OPTIONAL
# In wasi-sdk builds, disable the CLI shell build in the "all" target.
SQLITE3_SHELL_TARGET_ = sqlite3$(TEXE)
SQLITE3_SHELL_TARGET_1 =
SQLITE3_SHELL_TARGET = $(SQLITE3_SHELL_TARGET_@HAVE_WASI_SDK@)
# Use $(libtclsqlite3.la_$(HAVE_TCL)) to resolve to either
# libtclsqlite3.la or an empty value.
libtclsqlite3.la_0 =
libtclsqlite3.la_1 = libtclsqlite3.la
# This is the default Makefile target. The objects listed here
# are what get build when you type just "make" with no arguments.
#
all: sqlite3.h libsqlite3.la sqlite3$(TEXE) $(HAVE_TCL:1=libtclsqlite3.la)
all: sqlite3.h libsqlite3.la $(SQLITE3_SHELL_TARGET) \
$(libtclsqlite3.la_$(HAVE_TCL))
Makefile: $(TOP)/Makefile.in
./config.status
@@ -660,12 +696,26 @@ sqlite3$(TEXE): shell.c sqlite3.c
shell.c sqlite3.c \
$(LIBREADLINE) $(TLIBS) -rpath "$(libdir)"
sqldiff$(TEXE): $(TOP)/tool/sqldiff.c sqlite3.lo sqlite3.h
$(LTLINK) -o $@ $(TOP)/tool/sqldiff.c sqlite3.lo $(TLIBS)
sqldiff$(TEXE): $(TOP)/tool/sqldiff.c $(TOP)/ext/misc/sqlite3_stdio.h sqlite3.lo sqlite3.h
$(LTLINK) -I$(TOP)/ext/misc -o $@ $(TOP)/tool/sqldiff.c sqlite3.lo $(TLIBS)
dbhash$(TEXE): $(TOP)/tool/dbhash.c sqlite3.lo sqlite3.h
$(LTLINK) -o $@ $(TOP)/tool/dbhash.c sqlite3.lo $(TLIBS)
RSYNC_SRC = \
$(TOP)/tool/sqlite3-rsync.c \
sqlite3.c
RSYNC_OPT = \
-DSQLITE_ENABLE_DBPAGE_VTAB \
-USQLITE_THREADSAFE \
-DSQLITE_THREADSAFE=0 \
-DSQLITE_OMIT_LOAD_EXTENSION \
-DSQLITE_OMIT_DEPRECATED
sqlite3-rsync$(TEXE): $(RSYNC_SRC)
$(TCC) -o $@ $(RSYNC_OPT) $(RSYNC_SRC) $(TLIBS)
scrub$(TEXE): $(TOP)/ext/misc/scrub.c sqlite3.lo
$(LTLINK) -o $@ -I. -DSCRUB_STANDALONE \
$(TOP)/ext/misc/scrub.c sqlite3.lo $(TLIBS)
@@ -676,13 +726,40 @@ srcck1$(BEXE): $(TOP)/tool/srcck1.c
sourcetest: srcck1$(BEXE) sqlite3.c
./srcck1 sqlite3.c
src-verify: $(TOP)/tool/src-verify.c
$(BCC) -o src-verify$(BEXE) $(TOP)/tool/src-verify.c
verify-source: ./src-verify
./src-verify $(TOP)
fuzzershell$(TEXE): $(TOP)/tool/fuzzershell.c sqlite3.c sqlite3.h
$(LTLINK) -o $@ $(FUZZERSHELL_OPT) \
$(TOP)/tool/fuzzershell.c sqlite3.c $(TLIBS)
fuzzcheck$(TEXE): $(FUZZCHECK_SRC) sqlite3.c sqlite3.h
fuzzcheck$(TEXE): $(FUZZCHECK_SRC) sqlite3.c sqlite3.h $(FUZZCHECK_DEP)
$(LTLINK) -o $@ $(FUZZCHECK_OPT) $(FUZZCHECK_SRC) sqlite3.c $(TLIBS)
fuzzcheck-asan$(TEXE): $(FUZZCHECK_SRC) sqlite3.c sqlite3.h $(FUZZCHECK_DEP)
$(LTLINK) -o $@ -fsanitize=address $(FUZZCHECK_OPT) $(FUZZCHECK_SRC) sqlite3.c $(TLIBS)
fuzzcheck-ubsan$(TEXE): $(FUZZCHECK_SRC) sqlite3.c sqlite3.h $(FUZZCHECK_DEP)
$(LTLINK) -o $@ -fsanitize=undefined $(FUZZCHECK_OPT) $(FUZZCHECK_SRC) sqlite3.c $(TLIBS)
# Usage: FUZZDB=filename make run-fuzzcheck
#
# Where filename is a fuzzcheck database, this target builds and runs
# fuzzcheck, fuzzcheck-asan, and fuzzcheck-ubsan on that database.
#
# FUZZDB can be a glob pattern of two or more databases. Example:
#
# FUZZDB=test/fuzzdata*.db make run-fuzzcheck
#
run-fuzzcheck: fuzzcheck$(TEXE) fuzzcheck-asan$(TEXE) fuzzcheck-ubsan$(TEXE)
@if test "$(FUZZDB)" = ""; then echo 'ERROR: No FUZZDB specified. Rerun with FUZZDB=filename'; exit 1; fi
./fuzzcheck$(TEXE) --spinner $(FUZZDB)
./fuzzcheck-asan$(TEXE) --spinner $(FUZZDB)
./fuzzcheck-ubsan$(TEXE) --spinner $(FUZZDB)
ossshell$(TEXE): $(TOP)/test/ossfuzz.c $(TOP)/test/ossshell.c sqlite3.c sqlite3.h
$(LTLINK) -o $@ $(FUZZCHECK_OPT) $(TOP)/test/ossshell.c \
$(TOP)/test/ossfuzz.c sqlite3.c $(TLIBS)
@@ -742,13 +819,26 @@ mptest: mptester$(TEXE)
$(MPTEST2) --journalmode DELETE
has_tclsh84:
sh $(TOP)/tool/cktclsh.sh 8.4 $(TCLSH_CMD)
touch has_tclsh84
has_tclsh85:
sh $(TOP)/tool/cktclsh.sh 8.5 $(TCLSH_CMD)
touch has_tclsh85
has_tclconfig:
@ if test x"$(HAVE_TCL)" != "x1"; then echo 'ERROR: Requires access to "tclConfig.sh" which "configure" was not able to locate'; exit 1; fi
touch has_tclconfig
# This target creates a directory named "tsrc" and fills it with
# copies of all of the C source code and header files needed to
# build on the target system. Some of the C source code and header
# files are automatically generated. This target takes care of
# all that automatic generation.
#
.target_source: $(SRC) $(TOP)/tool/vdbe-compress.tcl fts5.c
.target_source: $(SRC) $(TOP)/tool/vdbe-compress.tcl has_tclsh84 fts5.c
rm -rf tsrc
mkdir tsrc
cp -f $(SRC) tsrc
@@ -758,11 +848,20 @@ mptest: mptester$(TEXE)
cp fts5.c fts5.h tsrc
touch .target_source
sqlite3.c: .target_source $(TOP)/tool/mksqlite3c.tcl
$(TCLSH_CMD) $(TOP)/tool/mksqlite3c.tcl $(AMALGAMATION_LINE_MACROS)
sqlite3.c: .target_source $(TOP)/tool/mksqlite3c.tcl src-verify has_tclsh84
$(TCLSH_CMD) $(TOP)/tool/mksqlite3c.tcl $(AMALGAMATION_LINE_MACROS) $(EXTRA_SRC)
cp tsrc/sqlite3ext.h .
cp $(TOP)/ext/session/sqlite3session.h .
sqlite3r.h: sqlite3.h has_tclsh84
$(TCLSH_CMD) $(TOP)/tool/mksqlite3h.tcl $(TOP) --enable-recover >sqlite3r.h
sqlite3r.c: sqlite3.c sqlite3r.h has_tclsh84
cp $(TOP)/ext/recover/sqlite3recover.c tsrc/
cp $(TOP)/ext/recover/sqlite3recover.h tsrc/
cp $(TOP)/ext/recover/dbdata.c tsrc/
$(TCLSH_CMD) $(TOP)/tool/mksqlite3c.tcl --enable-recover $(AMALGAMATION_LINE_MACROS) $(EXTRA_SRC)
sqlite3ext.h: .target_source
cp tsrc/sqlite3ext.h .
@@ -772,7 +871,7 @@ tclsqlite3.c: sqlite3.c
echo '#endif /* USE_SYSTEM_SQLITE */' >>tclsqlite3.c
cat $(TOP)/src/tclsqlite.c >>tclsqlite3.c
sqlite3-all.c: sqlite3.c $(TOP)/tool/split-sqlite3c.tcl
sqlite3-all.c: sqlite3.c $(TOP)/tool/split-sqlite3c.tcl has_tclsh84
$(TCLSH_CMD) $(TOP)/tool/split-sqlite3c.tcl
# Rule to build the amalgamation
@@ -874,7 +973,7 @@ hash.lo: $(TOP)/src/hash.c $(HDR)
insert.lo: $(TOP)/src/insert.c $(HDR)
$(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/insert.c
json.lo: $(TOP)/src/json.c
json.lo: $(TOP)/src/json.c $(HDR)
$(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/json.c
legacy.lo: $(TOP)/src/legacy.c $(HDR)
@@ -937,6 +1036,9 @@ pcache1.lo: $(TOP)/src/pcache1.c $(HDR) $(TOP)/src/pcache.h
os.lo: $(TOP)/src/os.c $(HDR)
$(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/os.c
os_kv.lo: $(TOP)/src/os_kv.c $(HDR)
$(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/os_kv.c
os_unix.lo: $(TOP)/src/os_unix.c $(HDR)
$(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/os_unix.c
@@ -1051,16 +1153,16 @@ tclsqlite-shell.lo: $(TOP)/src/tclsqlite.c $(HDR)
tclsqlite-stubs.lo: $(TOP)/src/tclsqlite.c $(HDR)
$(LTCOMPILE) -DUSE_TCL_STUBS=1 -o $@ -c $(TOP)/src/tclsqlite.c
tclsqlite3$(TEXE): tclsqlite-shell.lo libsqlite3.la
tclsqlite3$(TEXE): has_tclconfig tclsqlite-shell.lo libsqlite3.la
$(LTLINK) -o $@ tclsqlite-shell.lo \
libsqlite3.la $(LIBTCL)
# Rules to build opcodes.c and opcodes.h
#
opcodes.c: opcodes.h $(TOP)/tool/mkopcodec.tcl
opcodes.c: opcodes.h $(TOP)/tool/mkopcodec.tcl has_tclsh84
$(TCLSH_CMD) $(TOP)/tool/mkopcodec.tcl opcodes.h >opcodes.c
opcodes.h: parse.h $(TOP)/src/vdbe.c $(TOP)/tool/mkopcodeh.tcl
opcodes.h: parse.h $(TOP)/src/vdbe.c $(TOP)/tool/mkopcodeh.tcl has_tclsh84
cat parse.h $(TOP)/src/vdbe.c | $(TCLSH_CMD) $(TOP)/tool/mkopcodeh.tcl >opcodes.h
# Rules to build parse.c and parse.h - the outputs of lemon.
@@ -1071,10 +1173,10 @@ parse.c: $(TOP)/src/parse.y lemon$(BEXE)
cp $(TOP)/src/parse.y .
./lemon$(BEXE) $(OPT_FEATURE_FLAGS) $(OPTS) -S parse.y
sqlite3.h: $(TOP)/src/sqlite.h.in $(TOP)/manifest mksourceid$(BEXE) $(TOP)/VERSION
sqlite3.h: $(TOP)/src/sqlite.h.in $(TOP)/manifest mksourceid$(BEXE) $(TOP)/VERSION has_tclsh84
$(TCLSH_CMD) $(TOP)/tool/mksqlite3h.tcl $(TOP) >sqlite3.h
sqlite3rc.h: $(TOP)/src/sqlite3.rc $(TOP)/VERSION
sqlite3rc.h: $(TOP)/src/sqlite3.rc $(TOP)/VERSION has_tclsh84
echo '#ifndef SQLITE_RESOURCE_VERSION' >$@
echo -n '#define SQLITE_RESOURCE_VERSION ' >>$@
cat $(TOP)/VERSION | $(TCLSH_CMD) $(TOP)/tool/replace.tcl exact . , >>$@
@@ -1084,26 +1186,40 @@ keywordhash.h: $(TOP)/tool/mkkeywordhash.c
$(BCC) -o mkkeywordhash$(BEXE) $(OPT_FEATURE_FLAGS) $(OPTS) $(TOP)/tool/mkkeywordhash.c
./mkkeywordhash$(BEXE) >keywordhash.h
# Source files that go into making shell.c
SHELL_SRC = \
$(TOP)/src/shell.c.in \
$(TOP)/ext/misc/appendvfs.c \
$(TOP)/ext/misc/completion.c \
$(TOP)/ext/misc/decimal.c \
$(TOP)/ext/misc/fileio.c \
$(TOP)/ext/misc/ieee754.c \
$(TOP)/ext/misc/regexp.c \
$(TOP)/ext/misc/series.c \
$(TOP)/ext/misc/shathree.c \
$(TOP)/ext/misc/sqlar.c \
$(TOP)/ext/misc/uint.c \
$(TOP)/ext/expert/sqlite3expert.c \
$(TOP)/ext/expert/sqlite3expert.h \
$(TOP)/ext/misc/zipfile.c \
$(TOP)/ext/misc/memtrace.c \
$(TOP)/src/test_windirent.c
# Source and header files that shell.c depends on
SHELL_DEP = \
$(TOP)/src/shell.c.in \
$(TOP)/ext/expert/sqlite3expert.c \
$(TOP)/ext/expert/sqlite3expert.h \
$(TOP)/ext/intck/sqlite3intck.c \
$(TOP)/ext/intck/sqlite3intck.h \
$(TOP)/ext/misc/appendvfs.c \
$(TOP)/ext/misc/base64.c \
$(TOP)/ext/misc/base85.c \
$(TOP)/ext/misc/completion.c \
$(TOP)/ext/misc/decimal.c \
$(TOP)/ext/misc/fileio.c \
$(TOP)/ext/misc/ieee754.c \
$(TOP)/ext/misc/memtrace.c \
$(TOP)/ext/misc/pcachetrace.c \
$(TOP)/ext/misc/percentile.c \
$(TOP)/ext/misc/regexp.c \
$(TOP)/ext/misc/series.c \
$(TOP)/ext/misc/sha1.c \
$(TOP)/ext/misc/shathree.c \
$(TOP)/ext/misc/sqlar.c \
$(TOP)/ext/misc/sqlite3_stdio.c \
$(TOP)/ext/misc/sqlite3_stdio.h \
$(TOP)/ext/misc/uint.c \
$(TOP)/ext/misc/vfstrace.c \
$(TOP)/ext/misc/zipfile.c \
$(TOP)/ext/recover/dbdata.c \
$(TOP)/ext/recover/sqlite3recover.c \
$(TOP)/ext/recover/sqlite3recover.h \
$(TOP)/src/test_windirent.c \
$(TOP)/src/test_windirent.h
shell.c: $(SHELL_SRC) $(TOP)/tool/mkshellc.tcl
shell.c: $(SHELL_DEP) $(TOP)/tool/mkshellc.tcl has_tclsh84
$(TCLSH_CMD) $(TOP)/tool/mkshellc.tcl >shell.c
@@ -1114,24 +1230,6 @@ shell.c: $(SHELL_SRC) $(TOP)/tool/mkshellc.tcl
icu.lo: $(TOP)/ext/icu/icu.c $(HDR) $(EXTHDR)
$(LTCOMPILE) -DSQLITE_CORE -c $(TOP)/ext/icu/icu.c
fts2.lo: $(TOP)/ext/fts2/fts2.c $(HDR) $(EXTHDR)
$(LTCOMPILE) -DSQLITE_CORE -c $(TOP)/ext/fts2/fts2.c
fts2_hash.lo: $(TOP)/ext/fts2/fts2_hash.c $(HDR) $(EXTHDR)
$(LTCOMPILE) -DSQLITE_CORE -c $(TOP)/ext/fts2/fts2_hash.c
fts2_icu.lo: $(TOP)/ext/fts2/fts2_icu.c $(HDR) $(EXTHDR)
$(LTCOMPILE) -DSQLITE_CORE -c $(TOP)/ext/fts2/fts2_icu.c
fts2_porter.lo: $(TOP)/ext/fts2/fts2_porter.c $(HDR) $(EXTHDR)
$(LTCOMPILE) -DSQLITE_CORE -c $(TOP)/ext/fts2/fts2_porter.c
fts2_tokenizer.lo: $(TOP)/ext/fts2/fts2_tokenizer.c $(HDR) $(EXTHDR)
$(LTCOMPILE) -DSQLITE_CORE -c $(TOP)/ext/fts2/fts2_tokenizer.c
fts2_tokenizer1.lo: $(TOP)/ext/fts2/fts2_tokenizer1.c $(HDR) $(EXTHDR)
$(LTCOMPILE) -DSQLITE_CORE -c $(TOP)/ext/fts2/fts2_tokenizer1.c
fts3.lo: $(TOP)/ext/fts3/fts3.c $(HDR) $(EXTHDR)
$(LTCOMPILE) -DSQLITE_CORE -c $(TOP)/ext/fts3/fts3.c
@@ -1209,7 +1307,7 @@ fts5parse.c: $(TOP)/ext/fts5/fts5parse.y lemon$(BEXE)
fts5parse.h: fts5parse.c
fts5.c: $(FTS5_SRC)
fts5.c: $(FTS5_SRC) has_tclsh84
$(TCLSH_CMD) $(TOP)/ext/fts5/tool/mkfts5c.tcl
cp $(TOP)/ext/fts5/fts5.h .
@@ -1237,19 +1335,21 @@ TESTFIXTURE_FLAGS += -DSQLITE_ENABLE_STMTVTAB
TESTFIXTURE_FLAGS += -DSQLITE_ENABLE_DBPAGE_VTAB
TESTFIXTURE_FLAGS += -DSQLITE_ENABLE_BYTECODE_VTAB
TESTFIXTURE_FLAGS += -DSQLITE_CKSUMVFS_STATIC
TESTFIXTURE_FLAGS += -DSQLITE_STATIC_RANDOMJSON
TESTFIXTURE_FLAGS += -DSQLITE_STRICT_SUBTYPE=1
TESTFIXTURE_SRC0 = $(TESTSRC2) libsqlite3.la
TESTFIXTURE_SRC1 = sqlite3.c
TESTFIXTURE_SRC = $(TESTSRC) $(TOP)/src/tclsqlite.c
TESTFIXTURE_SRC += $(TESTFIXTURE_SRC$(USE_AMALGAMATION))
testfixture$(TEXE): $(TESTFIXTURE_SRC)
testfixture$(TEXE): has_tclconfig has_tclsh85 $(TESTFIXTURE_SRC)
$(LTLINK) -DSQLITE_NO_SYNC=1 $(TEMP_STORE) $(TESTFIXTURE_FLAGS) \
-o $@ $(TESTFIXTURE_SRC) $(LIBTCL) $(TLIBS)
coretestprogs: $(TESTPROGS)
coretestprogs: testfixture$(BEXE) sqlite3$(BEXE)
testprogs: coretestprogs srcck1$(BEXE) fuzzcheck$(TEXE) sessionfuzz$(TEXE)
testprogs: $(TESTPROGS) srcck1$(BEXE) fuzzcheck$(TEXE) sessionfuzz$(TEXE)
# A very detailed test running most or all test cases
fulltest: alltest fuzztest
@@ -1267,11 +1367,17 @@ fulltestonly: $(TESTPROGS) fuzztest
./testfixture$(TEXE) $(TOP)/test/full.test
# Fuzz testing
fuzztest: fuzzcheck$(TEXE) $(FUZZDATA) sessionfuzz$(TEXE) $(TOP)/test/sessionfuzz-data1.db
#
# WARNING: When the "fuzztest" target is run by the testrunner.tcl script,
# it does not actually run this code. Instead, it schedules equivalent
# commands. Therefore, if this target is updated, then code in
# testrunner_data.tcl (search for "trd_fuzztest_data") must also be updated.
#
fuzztest: fuzzcheck$(TEXE) $(FUZZDATA) sessionfuzz$(TEXE)
./fuzzcheck$(TEXE) $(FUZZDATA)
./sessionfuzz$(TEXE) run $(TOP)/test/sessionfuzz-data1.db
valgrindfuzz: fuzzcheck$(TEXT) $(FUZZDATA) sessionfuzz$(TEXE) $(TOP)/test/sessionfuzz-data1.db
valgrindfuzz: fuzzcheck$(TEXT) $(FUZZDATA) sessionfuzz$(TEXE)
valgrind ./fuzzcheck$(TEXE) --cell-size-check --limit-mem 10M $(FUZZDATA)
valgrind ./sessionfuzz$(TEXE) run $(TOP)/test/sessionfuzz-data1.db
@@ -1280,15 +1386,50 @@ valgrindfuzz: fuzzcheck$(TEXT) $(FUZZDATA) sessionfuzz$(TEXE) $(TOP)/test/sessio
tcltest: ./testfixture$(TEXE)
./testfixture$(TEXE) $(TOP)/test/veryquick.test $(TESTOPTS)
# Runs all the same tests cases as the "tcltest" target but uses
# the testrunner.tcl script to run them in multiple cores
# concurrently.
testrunner: testfixture$(TEXE)
./testfixture$(TEXE) $(TOP)/test/testrunner.tcl
# This is the testing target preferred by the core SQLite developers.
# It runs tests under a standard configuration, regardless of how
# ./configure was run. The devs run "make devtest" prior to each
# check-in, at a minimum. Probably other tests too, but at least this
# one.
#
devtest: srctree-check sourcetest
$(TCLSH_CMD) $(TOP)/test/testrunner.tcl mdevtest $(TSTRNNR_OPTS)
mdevtest: srctree-check has_tclsh85
$(TCLSH_CMD) $(TOP)/test/testrunner.tcl mdevtest $(TSTRNNR_OPTS)
sdevtest: has_tclsh85
$(TCLSH_CMD) $(TOP)/test/testrunner.tcl sdevtest $(TSTRNNR_OPTS)
# Validate that various generated files in the source tree
# are up-to-date.
#
srctree-check: $(TOP)/tool/srctree-check.tcl
$(TCLSH_CMD) $(TOP)/tool/srctree-check.tcl
# Testing for a release
#
releasetest: srctree-check has_tclsh85 verify-source
$(TCLSH_CMD) $(TOP)/test/testrunner.tcl release $(TSTRNNR_OPTS)
# Minimal testing that runs in less than 3 minutes
#
quicktest: ./testfixture$(TEXE)
./testfixture$(TEXE) $(TOP)/test/extraquick.test $(TESTOPTS)
# This is the common case. Run many tests that do not take too long,
# including fuzzcheck, sqlite3_analyzer, and sqldiff tests.
# Try to run tests on whatever options are specified by the
# ./configure. The developers seldom use this target. Instead
# they use "make devtest" which runs tests on a standard set of
# options regardless of how SQLite is configured. This "test"
# target is provided for legacy only.
#
test: fuzztest sourcetest $(TESTPROGS) tcltest
test: srctree-check fuzztest sourcetest $(TESTPROGS) tcltest
# Run a test using valgrind. This can take a really long time
# because valgrind is so much slower than a native machine.
@@ -1303,19 +1444,19 @@ valgrindtest: $(TESTPROGS) valgrindfuzz
smoketest: $(TESTPROGS) fuzzcheck$(TEXE)
./testfixture$(TEXE) $(TOP)/test/main.test $(TESTOPTS)
shelltest: $(TESTPROGS)
./testfixture$(TEXT) $(TOP)/test/permutations.test shell
shelltest:
$(TCLSH_CMD) $(TOP)/test/testrunner.tcl release shell
sqlite3_analyzer.c: sqlite3.c $(TOP)/src/tclsqlite.c $(TOP)/tool/spaceanal.tcl $(TOP)/tool/mkccode.tcl $(TOP)/tool/sqlite3_analyzer.c.in
sqlite3_analyzer.c: sqlite3.c $(TOP)/src/tclsqlite.c $(TOP)/tool/spaceanal.tcl $(TOP)/tool/mkccode.tcl $(TOP)/tool/sqlite3_analyzer.c.in has_tclsh85
$(TCLSH_CMD) $(TOP)/tool/mkccode.tcl $(TOP)/tool/sqlite3_analyzer.c.in >sqlite3_analyzer.c
sqlite3_analyzer$(TEXE): sqlite3_analyzer.c
sqlite3_analyzer$(TEXE): has_tclconfig sqlite3_analyzer.c
$(LTLINK) sqlite3_analyzer.c -o $@ $(LIBTCL) $(TLIBS)
sqltclsh.c: sqlite3.c $(TOP)/src/tclsqlite.c $(TOP)/tool/sqltclsh.tcl $(TOP)/ext/misc/appendvfs.c $(TOP)/tool/mkccode.tcl $(TOP)/tool/sqltclsh.c.in
sqltclsh.c: sqlite3.c $(TOP)/src/tclsqlite.c $(TOP)/tool/sqltclsh.tcl $(TOP)/ext/misc/appendvfs.c $(TOP)/tool/mkccode.tcl $(TOP)/tool/sqltclsh.c.in has_tclsh85
$(TCLSH_CMD) $(TOP)/tool/mkccode.tcl $(TOP)/tool/sqltclsh.c.in >sqltclsh.c
sqltclsh$(TEXE): sqltclsh.c
sqltclsh$(TEXE): has_tclconfig sqltclsh.c
$(LTLINK) sqltclsh.c -o $@ $(LIBTCL) $(TLIBS)
sqlite3_expert$(TEXE): $(TOP)/ext/expert/sqlite3expert.h $(TOP)/ext/expert/sqlite3expert.c $(TOP)/ext/expert/expert.c sqlite3.c
@@ -1331,10 +1472,10 @@ CHECKER_DEPS =\
$(TOP)/ext/misc/btreeinfo.c \
$(TOP)/ext/repair/sqlite3_checker.c.in
sqlite3_checker.c: $(CHECKER_DEPS)
sqlite3_checker.c: $(CHECKER_DEPS) has_tclsh85
$(TCLSH_CMD) $(TOP)/tool/mkccode.tcl $(TOP)/ext/repair/sqlite3_checker.c.in >$@
sqlite3_checker$(TEXE): sqlite3_checker.c
sqlite3_checker$(TEXE): has_tclconfig sqlite3_checker.c
$(LTLINK) sqlite3_checker.c -o $@ $(LIBTCL) $(TLIBS)
dbdump$(TEXE): $(TOP)/ext/misc/dbdump.c sqlite3.lo
@@ -1380,7 +1521,7 @@ LogEst$(TEXE): $(TOP)/tool/logest.c sqlite3.h
wordcount$(TEXE): $(TOP)/test/wordcount.c sqlite3.lo
$(LTLINK) -o $@ $(TOP)/test/wordcount.c sqlite3.lo $(TLIBS)
speedtest1$(TEXE): $(TOP)/test/speedtest1.c sqlite3.c
speedtest1$(TEXE): $(TOP)/test/speedtest1.c sqlite3.c Makefile
$(LTLINK) $(ST_OPT) -o $@ $(TOP)/test/speedtest1.c sqlite3.c $(TLIBS)
startup$(TEXE): $(TOP)/test/startup.c sqlite3.c
@@ -1417,6 +1558,11 @@ amalgamation-tarball: sqlite3.c sqlite3rc.h
snapshot-tarball: sqlite3.c sqlite3rc.h
TOP=$(TOP) sh $(TOP)/tool/mkautoconfamal.sh --snapshot
# Build a ZIP archive containing various command-line tools.
#
tool-zip: testfixture sqlite3 sqldiff sqlite3_analyzer $(TOP)/tool/mktoolzip.tcl
./testfixture $(TOP)/tool/mktoolzip.tcl
# The next two rules are used to support the "threadtest" target. Building
# threadtest runs a few thread-safety tests that are implemented in C. This
# target is invoked by the releasetest.tcl script.
@@ -1437,16 +1583,18 @@ threadtest: threadtest3$(TEXE)
threadtest5: sqlite3.c $(TOP)/test/threadtest5.c
$(LTLINK) $(TOP)/test/threadtest5.c sqlite3.c -o $@ $(TLIBS)
releasetest:
$(TCLSH_CMD) $(TOP)/test/releasetest.tcl
# Standard install and cleanup targets
#
lib_install: libsqlite3.la
$(INSTALL) -d $(DESTDIR)$(libdir)
$(LTINSTALL) libsqlite3.la $(DESTDIR)$(libdir)
install: sqlite3$(TEXE) lib_install sqlite3.h sqlite3.pc ${HAVE_TCL:1=tcl_install}
# Use $(tcl_install_$(HAVE_TCL)) to resolve to either tcl_install or
# an empty value.
tcl_install_0 =
tcl_install_1 = tcl_install
install: sqlite3$(TEXE) lib_install sqlite3.h sqlite3.pc $(tcl_install_$(HAVE_TCL))
$(INSTALL) -d $(DESTDIR)$(bindir)
$(LTINSTALL) sqlite3$(TEXE) $(DESTDIR)$(bindir)
$(INSTALL) -d $(DESTDIR)$(includedir)
@@ -1457,43 +1605,71 @@ install: sqlite3$(TEXE) lib_install sqlite3.h sqlite3.pc ${HAVE_TCL:1=tcl_instal
pkgIndex.tcl:
echo 'package ifneeded sqlite3 $(RELEASE) [list load [file join $$dir libtclsqlite3[info sharedlibextension]] sqlite3]' > $@
tcl_install: lib_install libtclsqlite3.la pkgIndex.tcl
$(INSTALL) -d $(DESTDIR)$(TCLLIBDIR)
$(LTINSTALL) libtclsqlite3.la $(DESTDIR)$(TCLLIBDIR)
rm -f $(DESTDIR)$(TCLLIBDIR)/libtclsqlite3.la $(DESTDIR)$(TCLLIBDIR)/libtclsqlite3.a
$(INSTALL) -m 0644 pkgIndex.tcl $(DESTDIR)$(TCLLIBDIR)
clean:
rm -f *.lo *.la *.o sqlite3$(TEXE) libsqlite3.la
rm -f sqlite3.h opcodes.*
rm -rf .libs .deps
rm -f lemon$(BEXE) lempar.c parse.* sqlite*.tar.gz
rm -f mkkeywordhash$(BEXE) keywordhash.h
rm -f *.da *.bb *.bbg gmon.out
rm -rf tsrc .target_source
rm -f tclsqlite3$(TEXE)
rm -f testfixture$(TEXE) test.db
# Build the SQLite TCL extension in a way that make it compatible
# with whatever version of TCL is running as $TCLSH_CMD, possibly defined
# by --with-tclsh=
#
tclextension: tclsqlite3.c
$(TCLSH_CMD) $(TOP)/tool/buildtclext.tcl --build-only --cc $(CC) $(CFLAGS) $(OPT_FEATURE_FLAGS) $(OPTS)
# Install the SQLite TCL extension in a way that is appropriate for $TCLSH_CMD
# to find it.
#
tclextension-install: tclsqlite3.c
$(TCLSH_CMD) $(TOP)/tool/buildtclext.tcl --cc $(CC) $(CFLAGS) $(OPT_FEATURE_FLAGS) $(OPTS)
# Install the SQLite TCL extension that is used by $TCLSH_CMD
#
tclextension-uninstall:
$(TCLSH_CMD) $(TOP)/tool/buildtclext.tcl --uninstall
# List all installed the SQLite TCL extension that is are accessible
# by $TCLSH_CMD, included prior versions.
#
tclextension-list:
$(TCLSH_CMD) $(TOP)/tool/buildtclext.tcl --info
# Remove build products sufficient so that subsequent makes will recompile
# everything from scratch. Do not remove:
#
# * test results and test logs
# * output from ./configure
#
tidy:
rm -f *.lo *.la *.o *.c *.da *.bb *.bbg gmon.* *.rws sqlite3$(TEXE)
rm -f fts5.h keywordhash.h opcodes.h sqlite3.h sqlite3ext.h sqlite3session.h
rm -rf .libs .deps tsrc .target_source
rm -f lemon$(BEXE) sqlite*.tar.gz
rm -f mkkeywordhash$(BEXE) mksourceid$(BEXE)
rm -f parse.* fts5parse.*
rm -f tclsqlite3$(TEXE) $(TESTPROGS)
rm -f LogEst$(TEXE) fts3view$(TEXE) rollback-test$(TEXE) showdb$(TEXE)
rm -f showjournal$(TEXE) showstat4$(TEXE) showwal$(TEXE) speedtest1$(TEXE)
rm -f wordcount$(TEXE) changeset$(TEXE)
rm -f sqlite3.dll sqlite3.lib sqlite3.exp sqlite3.def
rm -f sqlite3.c
rm -f sqlite3rc.h
rm -f shell.c sqlite3ext.h
rm -f sqlite3_analyzer$(TEXE) sqlite3_analyzer.c
rm -f sqlite-*-output.vsix
rm -f mptester mptester.exe
rm -f rbu rbu.exe
rm -f srcck1 srcck1.exe
rm -f fuzzershell fuzzershell.exe
rm -f fuzzcheck fuzzcheck.exe
rm -f sqldiff sqldiff.exe
rm -f dbhash dbhash.exe
rm -f fts5.* fts5parse.*
rm -f threadtest5
rm -f wordcount$(TEXE) changeset$(TEXE) version-info$(TEXE)
rm -f *.dll *.lib *.exp *.def *.pc *.vsix *.so *.dylib pkgIndex.tcl
rm -f sqlite3_analyzer$(TEXE) sqlite3-rsync$(TEXE)
rm -f mptester$(TEXE) rbu$(TEXE) srcck1$(TEXE)
rm -f fuzzershell$(TEXE) fuzzcheck$(TEXE) sqldiff$(TEXE) dbhash$(TEXE)
rm -f threadtest5$(TEXE)
rm -f src-verify has_tclsh*
# Removes build products and test logs. Retains ./configure outputs.
#
clean: tidy
rm -rf omittest* testrunner* testdir*
# Clean up everything. No exceptions.
#
distclean: clean
rm -f config.h config.log config.status libtool Makefile sqlite3.pc
rm -f sqlite_cfg.h config.log config.status Makefile $(LIBTOOL)
#
# Windows section
@@ -1513,133 +1689,33 @@ sqlite3.dll: $(REAL_LIBOBJ) sqlite3.def
$(TCC) -shared -o $@ sqlite3.def \
-Wl,"--strip-all" $(REAL_LIBOBJ)
#
# Fiddle app
#
fiddle: sqlite3.c shell.c
make -C ext/wasm fiddle emcc_opt=-Os
#
# fiddle/wasm section
# Spell-checking for source comments
# The sources checked are either C sources or C source templates.
# Their comments are extracted and processed through aspell using
# a custom dictionary that contains scads of odd identifiers that
# find their way into the comments.
#
fiddle_dir = ext/fiddle
fiddle_dir_abs = $(TOP)/$(fiddle_dir)
# ^^^ some emcc opts require absolute paths
fiddle_html = $(fiddle_dir)/fiddle.html
fiddle_module_js = $(fiddle_dir)/fiddle-module.js
sqlite3_wasm_js = $(fiddle_dir)/sqlite3.js
sqlite3_wasm = $(fiddle_dir)/sqlite3.wasm
#emcc_opt = -O0
#emcc_opt = -O1
#emcc_opt = -O2
#emcc_opt = -O3
emcc_opt = -Oz
emcc_flags = $(emcc_opt) -sALLOW_TABLE_GROWTH -sSTRICT_JS \
-sENVIRONMENT=web -sMODULARIZE \
-sEXPORTED_RUNTIME_METHODS=@$(fiddle_dir_abs)/EXPORTED_RUNTIME_METHODS \
-sDYNAMIC_EXECUTION=0 \
-I. $(SHELL_OPT)
$(fiddle_module_js): Makefile sqlite3.c shell.c \
$(fiddle_dir)/EXPORTED_RUNTIME_METHODS \
$(fiddle_dir)/EXPORTED_FUNCTIONS.fiddle
emcc -o $@ $(emcc_flags) \
-sEXPORT_NAME=initFiddleModule \
-sEXPORTED_FUNCTIONS=@$(fiddle_dir_abs)/EXPORTED_FUNCTIONS.fiddle \
sqlite3.c shell.c
gzip < $@ > $@.gz
gzip < $(fiddle_dir)/fiddle-module.wasm > $(fiddle_dir)/fiddle-module.wasm.gz
$(sqlite3_wasm_js): Makefile sqlite3.c \
$(fiddle_dir)/sqlite3-api.js \
$(fiddle_dir)/EXPORTED_RUNTIME_METHODS \
$(fiddle_dir)/EXPORTED_FUNCTIONS.sqlite3-api
emcc -o $@ $(emcc_flags) \
-sEXPORT_NAME=initSqlite3Module \
-sEXPORTED_FUNCTIONS=@$(fiddle_dir_abs)/EXPORTED_FUNCTIONS.sqlite3-api \
--post-js=$(fiddle_dir)/sqlite3-api.js \
--no-entry \
sqlite3.c
gzip < $@ > $@.gz
gzip < $(sqlite3_wasm) > $(sqlite3_wasm).gz
gzip < $(fiddle_dir)/sqlite3-api.js > $(fiddle_dir)/sqlite3-api.js.gz
$(fiddle_dir)/fiddle.js.gz: $(fiddle_dir)/fiddle.js
gzip < $< > $@
$(fiddle_dir)/sqlite3-api.js.gz: $(fiddle_dir)/sqlite3-api.js
gzip < $< > $@
# Currently, this target is setup to be "made" in-tree only.
# The output is ephemeral. Redirect it to guide spelling fixups,
# either to correct spelling or add words to tool/custom.txt.
#
./custom.rws: ./tool/custom.txt
@echo 'Updating custom dictionary from tool/custom.txt'
aspell --lang=en create master ./custom.rws < $<
fiddle_generated = $(fiddle_module_js) $(fiddle_module_js).gz \
$(fiddle_dir)/fiddle-module.wasm \
$(fiddle_dir)/fiddle-module.wasm.gz \
$(fiddle_dir)/fiddle.js.gz
sqlite3_wasm_generated = \
$(sqlite3_wasm) $(sqlite3_wasm).gz \
$(sqlite3_wasm_js) $(sqlite3_wasm_js).gz \
$(fiddle_dir)/sqlite3.js.gz \
$(fiddle_dir)/sqlite3-api.js.gz
misspell: ./custom.rws has_tclsh84
$(TCLSH_CMD) ./tool/spellsift.tcl ./src/*.c ./src/*.h ./src/*.in
clean-wasm:
rm -f $(fiddle_generated) $(sqlite3_wasm_generated)
clean: clean-wasm
fiddle: $(fiddle_module_js) $(fiddle_dir)/fiddle.js.gz
sqlite3-wasm: $(sqlite3_wasm_js)
wasm: fiddle sqlite3-wasm
########################################################################
# Explanation of the emcc build flags follows. Full docs for these can
# be found at:
#
# https://github.com/emscripten-core/emscripten/blob/main/src/settings.js
# tool/version-info: a utility for emitting sqlite3 version info
# in various forms.
#
# -sENVIRONMENT=web: elides bootstrap code related to non-web JS
# environments like node.js. Removing this makes the output a tiny
# tick larger but hypothetically makes it more portable to
# non-browser JS environments.
#
# -sMODULARIZE: changes how the generated code is structured to avoid
# declaring a global Module object and instead installing a function
# which loads and initialized the module. The function is named...
#
# -sEXPORT_NAME=jsFunctionName (see -sMODULARIZE)
#
# -sEXPORTED_RUNTIME_METHODS=@/absolute/path/to/file: a file
# containing a list of emscripten-supplied APIs, one per line, which
# must be exported into the generated JS. Must be an absolute path!
#
# -sEXPORTED_FUNCTIONS=@/absolute/path/to/file: a file containing a
# list of C functions, one per line, which must be exported via wasm
# so they're visible to JS. C symbols names in that file must all
# start with an underscore for reasons known only to the emcc
# developers. e.g., _sqlite3_open_v2 and _sqlite3_finalize. Must be
# an absolute path!
#
# -sSTRICT_JS ensures that the emitted JS code includes the 'use
# strict' option. Note that -sSTRICT is more broadly-scoped and
# results in build errors.
#
# -sALLOW_TABLE_GROWTH is required for (at a minimum) the UDF-binding
# feature.
#
# -sDYNAMIC_EXECUTION=0 disables eval() and the Function constructor.
# If the build runs without these, it's preferable to use this flag
# because certain execution environments disallow those constructs.
# This flag is not strictly necessary, however.
#
# -sWASM_BIGINT is UNTESTED but "should" allow the int64-using C APIs
# to work with JS/wasm, insofar as the JS environment supports the
# BigInt type. That support requires an extremely recent browser:
# Safari didn't get that support until late 2020.
#
# --no-entry: for compiling library code with no main(). If this is
# not supplied and the code has a main(), it is called as part of the
# module init process. Note that main() is #if'd out of shell.c
# (renamed) when building in wasm mode.
#
# --pre-js/--post-js=FILE relative or absolute paths to JS files to
# prepend/append to the emcc-generated bootstrapping JS. It's
# easier/faster to develop with separate JS files (reduces rebuilding
# requirements) but certain configurations, namely -sMODULARIZE, may
# require using at least a --pre-js file. They can be used
# individually and need not be paired.
#
# -O0..-O3 and -Oz: optimization levels affect not only C-style
# optimization but whether or not the resulting generated JS code
# gets minified. -O0 compiles _much_ more quickly than -O3 or -Oz,
# and doesn't minimize any JS code, so is recommended for
# development. -O3 or -Oz are recommended for deployment, but
# primarily because -Oz will shrink the wasm file notably. JS-side
# minification makes little difference in terms of overall
# distributable size.
########################################################################
version-info$(TEXE): $(TOP)/tool/version-info.c Makefile sqlite3.h
$(LTLINK) $(ST_OPT) -o $@ $(TOP)/tool/version-info.c
-6
View File
@@ -22,12 +22,6 @@ TOP = ../sqlite
BCC = gcc -g -O0
#BCC = /opt/ancic/bin/c89 -0
#### If the target operating system supports the "usleep()" system
# call, then define the HAVE_USLEEP macro for all C modules.
#
#USLEEP =
USLEEP = -DHAVE_USLEEP=1
#### If you want the SQLite library to be safe for use within a
# multi-threaded program, then define the following macro
# appropriately:
+243 -99
View File
@@ -18,6 +18,13 @@ USE_AMALGAMATION = 1
!ENDIF
# <</mark>>
# Optionally set EXTRA_SRC to a list of C files to append to
# the generated sqlite3.c.
#
!IFNDEF EXTRA_SRC
EXTRA_SRC =
!ENDIF
# Set this non-0 to enable full warnings (-W4, etc) when compiling.
#
!IFNDEF USE_FULLWARN
@@ -52,6 +59,13 @@ MINIMAL_AMALGAMATION = 0
USE_STDCALL = 0
!ENDIF
# Use the USE_SEH=0 option on the nmake command line to omit structured
# exception handling (SEH) support. SEH is on by default.
#
!IFNDEF USE_SEH
USE_SEH = 1
!ENDIF
# Set this non-0 to have the shell executable link against the core dynamic
# link library.
#
@@ -218,6 +232,12 @@ WIN32HEAP = 0
OSTRACE = 0
!ENDIF
# enable address sanitizer using ASAN=1 on the command-line.
#
!IFNDEF ASAN
ASAN = 0
!ENDIF
# Set this to one of the following values to enable various debugging
# features. Each level includes the debugging options from the previous
# levels. Currently, the recognized values for DEBUG are:
@@ -359,8 +379,10 @@ SQLITE_TCL_DEP =
# the Windows platform.
#
!IFNDEF OPT_FEATURE_FLAGS
OPT_FEATURE_FLAGS = $(OPT_XTRA)
!IF $(MINIMAL_AMALGAMATION)==0
OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_FTS3=1
OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_FTS5=1
OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_RTREE=1
OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_GEOPOLY=1
OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_STMTVTAB=1
@@ -371,6 +393,14 @@ OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_BYTECODE_VTAB=1
OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_COLUMN_METADATA=1
!ENDIF
# Additional feature-options above and beyond what are normally used can be
# be added using OPTIONS=.... on the command-line. These values are
# appended to the OPT_FEATURE_FLAGS variable.
#
!IFDEF OPTIONS
OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) $(OPTIONS)
!ENDIF
# Should the session extension be enabled? If so, add compilation options
# to enable it.
#
@@ -389,6 +419,14 @@ OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_MATH_FUNCTIONS
OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_RBU=1
!ENDIF
# Should structured exception handling (SEH) be enabled for WAL mode in
# the core library? It is on by default. Only omit it if the
# USE_SEH=0 option is provided on the nmake command-line.
#
!IF $(USE_SEH)==0
OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_OMIT_SEH=1
!ENDIF
# These are the "extended" SQLite compilation options used when compiling for
# the Windows 10 platform.
#
@@ -877,6 +915,13 @@ RCC = $(RCC) -DSQLITE_WIN32_MALLOC_VALIDATE=1
!ENDIF
!ENDIF
# Address sanitizer if ASAN=1
#
!IF $(ASAN)>0
TCC = $(TCC) /fsanitize=address
!ENDIF
# <<mark>>
# The locations of the Tcl header and library files. Also, the library that
# non-stubs enabled programs using Tcl must link against. These variables
@@ -893,7 +938,7 @@ TCLSUFFIX =
!ENDIF
!IFNDEF TCLDIR
TCLDIR = $(TOP)\compat\tcl
TCLDIR = C:\Tcl
!ENDIF
!IFNDEF TCLINCDIR
@@ -1251,7 +1296,7 @@ LIBOBJS0 = vdbe.lo parse.lo alter.lo analyze.lo attach.lo auth.lo \
main.lo malloc.lo mem0.lo mem1.lo mem2.lo mem3.lo mem5.lo \
memdb.lo memjournal.lo \
mutex.lo mutex_noop.lo mutex_unix.lo mutex_w32.lo \
notify.lo opcodes.lo os.lo os_unix.lo os_win.lo \
notify.lo opcodes.lo os.lo os_kv.lo os_unix.lo os_win.lo \
pager.lo pcache.lo pcache1.lo pragma.lo prepare.lo printf.lo \
random.lo resolve.lo rowset.lo rtree.lo \
sqlite3session.lo select.lo sqlite3rbu.lo status.lo stmt.lo \
@@ -1332,6 +1377,7 @@ SRC00 = \
$(TOP)\src\mutex_w32.c \
$(TOP)\src\notify.c \
$(TOP)\src\os.c \
$(TOP)\src\os_kv.c \
$(TOP)\src\os_unix.c \
$(TOP)\src\os_win.c
@@ -1411,20 +1457,6 @@ SRC05 = \
$(TOP)\src\wal.h \
$(TOP)\src\whereInt.h
# Extension source code files, part 1.
#
SRC06 = \
$(TOP)\ext\fts1\fts1.c \
$(TOP)\ext\fts1\fts1_hash.c \
$(TOP)\ext\fts1\fts1_porter.c \
$(TOP)\ext\fts1\fts1_tokenizer1.c \
$(TOP)\ext\fts2\fts2.c \
$(TOP)\ext\fts2\fts2_hash.c \
$(TOP)\ext\fts2\fts2_icu.c \
$(TOP)\ext\fts2\fts2_porter.c \
$(TOP)\ext\fts2\fts2_tokenizer.c \
$(TOP)\ext\fts2\fts2_tokenizer1.c
# Extension source code files, part 2.
#
SRC07 = \
@@ -1447,16 +1479,6 @@ SRC07 = \
$(TOP)\ext\rbu\sqlite3rbu.c \
$(TOP)\ext\misc\stmt.c
# Extension header files, part 1.
#
SRC08 = \
$(TOP)\ext\fts1\fts1.h \
$(TOP)\ext\fts1\fts1_hash.h \
$(TOP)\ext\fts1\fts1_tokenizer.h \
$(TOP)\ext\fts2\fts2.h \
$(TOP)\ext\fts2\fts2_hash.h \
$(TOP)\ext\fts2\fts2_tokenizer.h
# Extension header files, part 2.
#
SRC09 = \
@@ -1497,7 +1519,7 @@ SRC12 =
# All source code files.
#
SRC = $(SRC00) $(SRC01) $(SRC03) $(SRC04) $(SRC05) $(SRC06) $(SRC07) $(SRC08) $(SRC09) $(SRC10) $(SRC11) $(SRC12)
SRC = $(SRC00) $(SRC01) $(SRC03) $(SRC04) $(SRC05) $(SRC07) $(SRC09) $(SRC10) $(SRC11) $(SRC12)
# Source code to the test files.
#
@@ -1508,7 +1530,6 @@ TESTSRC = \
$(TOP)\src\test4.c \
$(TOP)\src\test5.c \
$(TOP)\src\test6.c \
$(TOP)\src\test7.c \
$(TOP)\src\test8.c \
$(TOP)\src\test9.c \
$(TOP)\src\test_autoext.c \
@@ -1537,7 +1558,6 @@ TESTSRC = \
$(TOP)\src\test_quota.c \
$(TOP)\src\test_rtree.c \
$(TOP)\src\test_schema.c \
$(TOP)\src\test_server.c \
$(TOP)\src\test_superlock.c \
$(TOP)\src\test_syscall.c \
$(TOP)\src\test_tclsh.c \
@@ -1551,7 +1571,7 @@ TESTSRC = \
$(TOP)\ext\fts3\fts3_term.c \
$(TOP)\ext\fts3\fts3_test.c \
$(TOP)\ext\rbu\test_rbu.c \
$(TOP)\ext\session\test_session.c
$(TOP)\ext\session\test_session.c
# Statically linked extensions.
#
@@ -1560,6 +1580,8 @@ TESTEXT = \
$(TOP)\ext\expert\test_expert.c \
$(TOP)\ext\misc\amatch.c \
$(TOP)\ext\misc\appendvfs.c \
$(TOP)\ext\misc\basexx.c \
$(TOP)\ext\misc\bgckpt.c \
$(TOP)\ext\misc\carray.c \
$(TOP)\ext\misc\cksumvfs.c \
$(TOP)\ext\misc\closure.c \
@@ -1579,15 +1601,21 @@ TESTEXT = \
$(TOP)\ext\misc\percentile.c \
$(TOP)\ext\misc\prefixes.c \
$(TOP)\ext\misc\qpvtab.c \
$(TOP)\ext\misc\randomjson.c \
$(TOP)\ext\misc\regexp.c \
$(TOP)\ext\misc\remember.c \
$(TOP)\ext\misc\series.c \
$(TOP)\ext\misc\spellfix.c \
$(TOP)\ext\misc\stmtrand.c \
$(TOP)\ext\misc\totype.c \
$(TOP)\ext\misc\unionvtab.c \
$(TOP)\ext\misc\wholenumber.c \
$(TOP)\ext\rtree\test_rtreedoc.c \
fts5.c
$(TOP)\ext\recover\sqlite3recover.c \
$(TOP)\ext\recover\test_recover.c \
$(TOP)\ext\intck\test_intck.c \
$(TOP)\ext\intck\sqlite3intck.c \
$(TOP)\ext\recover\dbdata.c
# If use of zlib is enabled, add the "zipfile.c" source file.
#
@@ -1601,10 +1629,10 @@ TESTEXT = $(TESTEXT) $(TOP)\ext\misc\zipfile.c
TESTSRC2 = \
$(SRC00) \
$(SRC01) \
$(SRC06) \
$(SRC07) \
$(SRC10) \
$(TOP)\ext\async\sqlite3async.c
$(TOP)\ext\async\sqlite3async.c \
fts5.c
# Header files used by all library source files.
#
@@ -1636,14 +1664,6 @@ HDR = \
# Header files used by extensions
#
EXTHDR = $(EXTHDR) \
$(TOP)\ext\fts1\fts1.h \
$(TOP)\ext\fts1\fts1_hash.h \
$(TOP)\ext\fts1\fts1_tokenizer.h
EXTHDR = $(EXTHDR) \
$(TOP)\ext\fts2\fts2.h \
$(TOP)\ext\fts2\fts2_hash.h \
$(TOP)\ext\fts2\fts2_tokenizer.h
EXTHDR = $(EXTHDR) \
$(TOP)\ext\fts3\fts3.h \
$(TOP)\ext\fts3\fts3Int.h \
@@ -1687,9 +1707,13 @@ FUZZDATA = \
# when the shell is not being dynamically linked.
#
!IF $(DYNAMIC_SHELL)==0 && $(FOR_WIN10)==0
SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_DQS=0
SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_FTS4=1
SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_EXPLAIN_COMMENTS=1
SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_OFFSET_SQL_FUNC=1
SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_UNKNOWN_SQL_FUNCTION=1
SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_STMT_SCANSTATUS=1
SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_STRICT_SUBTYPE=1
!ENDIF
# <<mark>>
@@ -1697,15 +1721,56 @@ SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_OFFSET_SQL_FUNC=1
#
MPTESTER_COMPILE_OPTS = -DSQLITE_ENABLE_FTS5
FUZZERSHELL_COMPILE_OPTS =
FUZZCHECK_OPTS = -DSQLITE_ENABLE_MEMSYS5 -DSQLITE_OSS_FUZZ -DSQLITE_MAX_MEMORY=50000000 -DSQLITE_PRINTF_PRECISION_LIMIT=1000
FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -I$(TOP)\test -I$(TOP)\ext\recover
FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_MEMSYS5
FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_OSS_FUZZ
FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_BYTECODE_VTAB
FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_DBPAGE_VTAB
FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_DBSTAT_VTAB
FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_BYTECODE_VTAB
FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_DESERIALIZE
FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_EXPLAIN_COMMENTS
FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_FTS3_PARENTHESIS
FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_FTS4
FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_FTS5
FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_GEOPOLY
FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_MATH_FUNCTIONS
FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_MEMSYS5
FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_NORMALIZE
FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_OFFSET_SQL_FUNC
FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_PREUPDATE_HOOK
FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_RTREE
FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_SESSION
FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_STMTVTAB
FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_UNKNOWN_SQL_FUNCTION
FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_STAT4
FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_STMT_SCANSTATUS
FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_MAX_MEMORY=50000000
FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_MAX_MMAP_SIZE=0
FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_OMIT_LOAD_EXTENSION
FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_PRINTF_PRECISION_LIMIT=1000
FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_PRIVATE=""
FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_STRICT_SUBTYPE=1
FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_STATIC_RANDOMJSON
FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_MAX_MEMORY=50000000
FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_PRINTF_PRECISION_LIMIT=1000
FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_OMIT_LOAD_EXTENSION
FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_FTS4
FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_FTS5
FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_RTREE
FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_GEOPOLY
FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_DBSTAT_VTAB
FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_BYTECODE_VTAB
FUZZCHECK_SRC = $(FUZZCHECK_SRC) $(TOP)\test\fuzzcheck.c
FUZZCHECK_SRC = $(FUZZCHECK_SRC) $(TOP)\test\ossfuzz.c
FUZZCHECK_SRC = $(FUZZCHECK_SRC) $(TOP)\test\fuzzinvariants.c
FUZZCHECK_SRC = $(FUZZCHECK_SRC) $(TOP)\test\vt02.c
FUZZCHECK_SRC = $(FUZZCHECK_SRC) $(TOP)\ext\recover\dbdata.c
FUZZCHECK_SRC = $(FUZZCHECK_SRC) $(TOP)\ext\recover\sqlite3recover.c
FUZZCHECK_SRC = $(FUZZCHECK_SRC) $(TOP)\ext\misc\percentile.c
FUZZCHECK_SRC = $(FUZZCHECK_SRC) $(TOP)\ext\misc\randomjson.c
FUZZCHECK_SRC = $(TOP)\test\fuzzcheck.c $(TOP)\test\ossfuzz.c $(TOP)\test\fuzzinvariants.c
OSSSHELL_SRC = $(TOP)\test\ossshell.c $(TOP)\test\ossfuzz.c
DBFUZZ_COMPILE_OPTS = -DSQLITE_THREADSAFE=0 -DSQLITE_OMIT_LOAD_EXTENSION
KV_COMPILE_OPTS = -DSQLITE_THREADSAFE=0 -DSQLITE_DIRECT_OVERFLOW_READ
@@ -1766,6 +1831,19 @@ pkgIndex.tcl: $(TOP)\VERSION
$(SQLITE3TCLDLL): libtclsqlite3.lib $(LIBRESOBJS) tclsqlite3.def pkgIndex.tcl
$(LD) $(LDFLAGS) $(LTLINKOPTS) $(LTLIBPATHS) /DLL /DEF:tclsqlite3.def /OUT:$@ libtclsqlite3.lib $(LIBRESOBJS) $(LTLIBS) $(TLIBS)
tclextension: $(SQLITE3TCLDLL)
tclextension-install: $(SQLITE3TCLDLL)
$(TCLSH_CMD) $(TOP)\tool\buildtclext.tcl --install-only
tclextension-uninstall:
$(TCLSH_CMD) $(TOP)\tool\buildtclext.tcl --uninstall
tclextension-list:
$(TCLSH_CMD) $(TOP)\tool\buildtclext.tcl --info
# <</mark>>
$(SQLITE3DLL): $(LIBOBJ) $(LIBRESOBJS) $(CORE_LINK_DEP)
@@ -1784,12 +1862,25 @@ $(SQLITE3EXE): shell.c $(SHELL_CORE_DEP) $(LIBRESOBJS) $(SHELL_CORE_SRC) $(SQLIT
/link $(SQLITE3EXEPDB) $(LDFLAGS) $(LTLINKOPTS) $(SHELL_LINK_OPTS) $(LTLIBPATHS) $(LIBRESOBJS) $(LIBREADLINE) $(LTLIBS) $(TLIBS)
# <<mark>>
sqldiff.exe: $(TOP)\tool\sqldiff.c $(SQLITE3C) $(SQLITE3H)
$(LTLINK) $(NO_WARN) $(TOP)\tool\sqldiff.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS)
sqldiff.exe: $(TOP)\tool\sqldiff.c $(TOP)\ext\misc\sqlite3_stdio.h $(TOP)\ext\misc\sqlite3_stdio.c $(SQLITE3C) $(SQLITE3H) $(LIBRESOBJS)
$(LTLINK) $(NO_WARN) -I$(TOP)\ext\misc $(TOP)\tool\sqldiff.c $(TOP)\ext\misc\sqlite3_stdio.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS) $(LIBRESOBJS)
dbhash.exe: $(TOP)\tool\dbhash.c $(SQLITE3C) $(SQLITE3H)
$(LTLINK) $(NO_WARN) $(TOP)\tool\dbhash.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS)
RSYNC_SRC = \
$(TOP)\tool\sqlite3-rsync.c \
$(SQLITE3C)
RSYNC_OPT = \
-DSQLITE_ENABLE_DBPAGE_VTAB \
-DSQLITE_THREADSAFE=0 \
-DSQLITE_OMIT_LOAD_EXTENSION \
-DSQLITE_OMIT_DEPRECATED
sqlite3-rsync.exe: $(RSYNC_SRC) $(LIBRESOBJS)
$(LTLINK) $(RSYNC_OPT) $(NO_WARN) $(RSYNC_SRC) /link $(LDFLAGS) $(LTLINKOPTS) $(LIBRESOBJS)
scrub.exe: $(TOP)\ext\misc\scrub.c $(SQLITE3C) $(SQLITE3H)
$(LTLINK) $(NO_WARN) -DSCRUB_STANDALONE=1 $(TOP)\ext\misc\scrub.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS)
@@ -1799,6 +1890,12 @@ srcck1.exe: $(TOP)\tool\srcck1.c
sourcetest: srcck1.exe $(SQLITE3C)
srcck1.exe $(SQLITE3C)
src-verify.exe: $(TOP)\tool\src-verify.c
$(LTLINK) $(NO_WARN) $(TOP)\tool\src-verify.c
verify-source: src-verify.exe
src-verify.exe $(TOP)
fuzzershell.exe: $(TOP)\tool\fuzzershell.c $(SQLITE3C) $(SQLITE3H)
$(LTLINK) $(NO_WARN) $(FUZZERSHELL_COMPILE_OPTS) $(TOP)\tool\fuzzershell.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS)
@@ -1806,7 +1903,14 @@ dbfuzz.exe: $(TOP)\test\dbfuzz.c $(SQLITE3C) $(SQLITE3H)
$(LTLINK) $(NO_WARN) $(DBFUZZ_COMPILE_OPTS) $(TOP)\test\dbfuzz.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS)
fuzzcheck.exe: $(FUZZCHECK_SRC) $(SQLITE3C) $(SQLITE3H)
$(LTLINK) $(NO_WARN) $(FUZZCHECK_OPTS) $(FUZZCHECK_SRC) $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS)
$(LTLINK) /F 8388608 $(NO_WARN) $(FUZZCHECK_OPTS) $(FUZZCHECK_SRC) $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS)
fuzzcheck-asan.exe: $(FUZZCHECK_SRC) $(SQLITE3C) $(SQLITE3H)
$(LTLINK) $(NO_WARN) /fsanitize=address $(FUZZCHECK_OPTS) $(FUZZCHECK_SRC) $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS)
run-fuzzcheck: fuzzcheck.exe fuzzcheck-asan.exe
fuzzcheck --spinner $(FUZZDB)
fuzzcheck-asan --spinner $(FUZZDB)
ossshell.exe: $(OSSSHELL_SRC) $(SQLITE3C) $(SQLITE3H)
$(LTLINK) $(NO_WARN) $(FUZZCHECK_OPTS) $(OSSSHELL_SRC) $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS)
@@ -1845,9 +1949,7 @@ mptest: mptester.exe
for %i in ($(SRC03)) do copy /Y %i tsrc
for %i in ($(SRC04)) do copy /Y %i tsrc
for %i in ($(SRC05)) do copy /Y %i tsrc
for %i in ($(SRC06)) do copy /Y %i tsrc
for %i in ($(SRC07)) do copy /Y %i tsrc
for %i in ($(SRC08)) do copy /Y %i tsrc
for %i in ($(SRC09)) do copy /Y %i tsrc
for %i in ($(SRC10)) do copy /Y %i tsrc
for %i in ($(SRC11)) do copy /Y %i tsrc
@@ -1861,8 +1963,8 @@ mptest: mptester.exe
move vdbe.new tsrc\vdbe.c
echo > .target_source
sqlite3.c: .target_source sqlite3ext.h sqlite3session.h $(MKSQLITE3C_TOOL)
$(TCLSH_CMD) $(MKSQLITE3C_TOOL) $(MKSQLITE3C_ARGS)
sqlite3.c: .target_source sqlite3ext.h sqlite3session.h $(MKSQLITE3C_TOOL) src-verify.exe
$(TCLSH_CMD) $(MKSQLITE3C_TOOL) $(MKSQLITE3C_ARGS) $(EXTRA_SRC)
sqlite3-all.c: sqlite3.c $(TOP)\tool\split-sqlite3c.tcl
$(TCLSH_CMD) $(TOP)\tool\split-sqlite3c.tcl
@@ -2053,6 +2155,9 @@ pcache1.lo: $(TOP)\src\pcache1.c $(HDR) $(TOP)\src\pcache.h
os.lo: $(TOP)\src\os.c $(HDR)
$(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(TOP)\src\os.c
os_kv.lo: $(TOP)\src\os_kv.c $(HDR)
$(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(TOP)\src\os_kv.c
os_unix.lo: $(TOP)\src\os_unix.c $(HDR)
$(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(TOP)\src\os_unix.c
@@ -2209,31 +2314,47 @@ mkkeywordhash.exe: $(TOP)\tool\mkkeywordhash.c
keywordhash.h: $(TOP)\tool\mkkeywordhash.c mkkeywordhash.exe
.\mkkeywordhash.exe > keywordhash.h
# Source files that go into making shell.c
SHELL_SRC = \
$(TOP)\src\shell.c.in \
$(TOP)\ext\misc\appendvfs.c \
$(TOP)\ext\misc\completion.c \
$(TOP)\ext\misc\decimal.c \
$(TOP)\ext\misc\fileio.c \
$(TOP)\ext\misc\ieee754.c \
$(TOP)\ext\misc\regexp.c \
$(TOP)\ext\misc\series.c \
$(TOP)\ext\misc\shathree.c \
$(TOP)\ext\misc\uint.c \
$(TOP)\ext\expert\sqlite3expert.c \
$(TOP)\ext\expert\sqlite3expert.h \
$(TOP)\ext\misc\memtrace.c \
$(TOP)\src\test_windirent.c
# Source and header files that shell.c depends on
SHELL_DEP = \
$(TOP)\src\shell.c.in \
$(TOP)\ext\expert\sqlite3expert.c \
$(TOP)\ext\expert\sqlite3expert.h \
$(TOP)\ext\intck\sqlite3intck.c \
$(TOP)\ext\intck\sqlite3intck.h \
$(TOP)\ext\misc\appendvfs.c \
$(TOP)\ext\misc\base64.c \
$(TOP)\ext\misc\base85.c \
$(TOP)\ext\misc\completion.c \
$(TOP)\ext\misc\decimal.c \
$(TOP)\ext\misc\fileio.c \
$(TOP)\ext\misc\ieee754.c \
$(TOP)\ext\misc\memtrace.c \
$(TOP)\ext\misc\pcachetrace.c \
$(TOP)\ext\misc\percentile.c \
$(TOP)\ext\misc\regexp.c \
$(TOP)\ext\misc\series.c \
$(TOP)\ext\misc\sha1.c \
$(TOP)\ext\misc\shathree.c \
$(TOP)\ext\misc\sqlar.c \
$(TOP)\ext\misc\sqlite3_stdio.c \
$(TOP)\ext\misc\sqlite3_stdio.h \
$(TOP)\ext\misc\uint.c \
$(TOP)\ext\misc\vfstrace.c \
$(TOP)\ext\misc\zipfile.c \
$(TOP)\ext\recover\dbdata.c \
$(TOP)\ext\recover\sqlite3recover.c \
$(TOP)\ext\recover\sqlite3recover.h \
$(TOP)\src\test_windirent.c \
$(TOP)\src\test_windirent.h
# If use of zlib is enabled, add the "zipfile.c" source file.
#
!IF $(USE_ZLIB)!=0
SHELL_SRC = $(SHELL_SRC) $(TOP)\ext\misc\sqlar.c
SHELL_SRC = $(SHELL_SRC) $(TOP)\ext\misc\zipfile.c
SHELL_DEP = $(SHELL_DEP) $(TOP)\ext\misc\sqlar.c
SHELL_DEP = $(SHELL_DEP) $(TOP)\ext\misc\zipfile.c
!ENDIF
shell.c: $(SHELL_SRC) $(TOP)\tool\mkshellc.tcl
shell.c: $(SHELL_DEP) $(TOP)\tool\mkshellc.tcl
$(TCLSH_CMD) $(TOP)\tool\mkshellc.tcl > shell.c
zlib:
@@ -2244,24 +2365,6 @@ zlib:
icu.lo: $(TOP)\ext\icu\icu.c $(HDR) $(EXTHDR)
$(LTCOMPILE) $(CORE_COMPILE_OPTS) $(NO_WARN) -DSQLITE_CORE -c $(TOP)\ext\icu\icu.c
fts2.lo: $(TOP)\ext\fts2\fts2.c $(HDR) $(EXTHDR)
$(LTCOMPILE) $(CORE_COMPILE_OPTS) $(NO_WARN) -DSQLITE_CORE -c $(TOP)\ext\fts2\fts2.c
fts2_hash.lo: $(TOP)\ext\fts2\fts2_hash.c $(HDR) $(EXTHDR)
$(LTCOMPILE) $(CORE_COMPILE_OPTS) $(NO_WARN) -DSQLITE_CORE -c $(TOP)\ext\fts2\fts2_hash.c
fts2_icu.lo: $(TOP)\ext\fts2\fts2_icu.c $(HDR) $(EXTHDR)
$(LTCOMPILE) $(CORE_COMPILE_OPTS) $(NO_WARN) -DSQLITE_CORE -c $(TOP)\ext\fts2\fts2_icu.c
fts2_porter.lo: $(TOP)\ext\fts2\fts2_porter.c $(HDR) $(EXTHDR)
$(LTCOMPILE) $(CORE_COMPILE_OPTS) $(NO_WARN) -DSQLITE_CORE -c $(TOP)\ext\fts2\fts2_porter.c
fts2_tokenizer.lo: $(TOP)\ext\fts2\fts2_tokenizer.c $(HDR) $(EXTHDR)
$(LTCOMPILE) $(CORE_COMPILE_OPTS) $(NO_WARN) -DSQLITE_CORE -c $(TOP)\ext\fts2\fts2_tokenizer.c
fts2_tokenizer1.lo: $(TOP)\ext\fts2\fts2_tokenizer1.c $(HDR) $(EXTHDR)
$(LTCOMPILE) $(CORE_COMPILE_OPTS) $(NO_WARN) -DSQLITE_CORE -c $(TOP)\ext\fts2\fts2_tokenizer1.c
fts3.lo: $(TOP)\ext\fts3\fts3.c $(HDR) $(EXTHDR)
$(LTCOMPILE) $(CORE_COMPILE_OPTS) $(NO_WARN) -DSQLITE_CORE -c $(TOP)\ext\fts3\fts3.c
@@ -2394,6 +2497,8 @@ TESTFIXTURE_FLAGS = $(TESTFIXTURE_FLAGS) -DSQLITE_ENABLE_DBPAGE_VTAB=1
TESTFIXTURE_FLAGS = $(TESTFIXTURE_FLAGS) -DSQLITE_ENABLE_BYTECODE_VTAB=1
TESTFIXTURE_FLAGS = $(TESTFIXTURE_FLAGS) -DSQLITE_CKSUMVFS_STATIC=1
TESTFIXTURE_FLAGS = $(TESTFIXTURE_FLAGS) $(TEST_CCONV_OPTS)
TESTFIXTURE_FLAGS = $(TESTFIXTURE_FLAGS) -DSQLITE_STATIC_RANDOMJSON
TESTFIXTURE_FLAGS = $(TESTFIXTURE_FLAGS) -DSQLITE_STRICT_SUBTYPE=1
TESTFIXTURE_SRC0 = $(TESTEXT) $(TESTSRC2)
TESTFIXTURE_SRC1 = $(TESTEXT) $(SQLITE3C)
@@ -2433,9 +2538,12 @@ extensiontest: testfixture.exe testloadext.dll
@set PATH=$(LIBTCLPATH);$(PATH)
.\testfixture.exe $(TOP)\test\loadext.test $(TESTOPTS)
coretestprogs: $(TESTPROGS)
tool-zip: testfixture.exe sqlite3.exe sqldiff.exe sqlite3_analyzer.exe $(TOP)\tool\mktoolzip.tcl
.\testfixture.exe $(TOP)\tool\mktoolzip.tcl
testprogs: coretestprogs srcck1.exe fuzzcheck.exe sessionfuzz.exe
coretestprogs: testfixture.exe sqlite3.exe
testprogs: $(TESTPROGS) srcck1.exe fuzzcheck.exe sessionfuzz.exe
fulltest: alltest fuzztest
@@ -2458,6 +2566,13 @@ queryplantest: testfixture.exe shell
fuzztest: fuzzcheck.exe
.\fuzzcheck.exe $(FUZZDATA)
# Legacy testing target for third-party integrators. The SQLite
# developers seldom use this target themselves. Instead
# they use "nmake /f Makefile.msc devtest" which runs tests on
# a standard set of options
#
test: $(TESTPROGS) sourcetest fuzztest tcltest
# Minimal testing that runs in less than 3 minutes (on a fast machine)
#
quicktest: testfixture.exe sourcetest
@@ -2467,7 +2582,6 @@ quicktest: testfixture.exe sourcetest
# This is the common case. Run many tests that do not take too long,
# including fuzzcheck, sqlite3_analyzer, and sqldiff tests.
#
test: $(TESTPROGS) sourcetest fuzztest tcltest
# The veryquick.test TCL tests.
#
@@ -2475,6 +2589,35 @@ tcltest: testfixture.exe
@set PATH=$(LIBTCLPATH);$(PATH)
.\testfixture.exe $(TOP)\test\veryquick.test $(TESTOPTS)
# Runs all the same tests cases as the "tcltest" target but uses
# the testrunner.tcl script to run them in multiple cores
# concurrently.
testrunner: testfixture.exe
.\testfixture.exe $(TOP)\test\testrunner.tcl
# This is the testing target preferred by the core SQLite developers.
# It runs tests under a standard configuration. The devs run
# "nmake /f Makefile.msc devtest" prior to each check-in, at a minimum.
# Probably other tests too, but at least this one.
#
devtest: srctree-check sourcetest
$(TCLSH_CMD) $(TOP)\test\testrunner.tcl mdevtest
mdevtest:
$(TCLSH_CMD) $(TOP)\test\testrunner.tcl mdevtest
# Validate that various generated files in the source tree
# are up-to-date.
#
srctree-check: $(TOP)\tool\srctree-check.tcl
$(TCLSH_CMD) $(TOP)\tool\srctree-check.tcl
# Testing for a release
#
releasetest:
$(TCLSH_CMD) $(TOP)\test\testrunner.tcl release
smoketest: $(TESTPROGS)
@set PATH=$(LIBTCLPATH);$(PATH)
.\testfixture.exe $(TOP)\test\main.test $(TESTOPTS)
@@ -2482,7 +2625,7 @@ smoketest: $(TESTPROGS)
shelltest: $(TESTPROGS)
.\testfixture.exe $(TOP)\test\permutations.test shell
sqlite3_analyzer.c: $(SQLITE3C) $(SQLITE3H) $(TOP)\src\tclsqlite.c $(TOP)\tool\spaceanal.tcl $(TOP)\tool\mkccode.tcl $(TOP)\tool\sqlite3_analyzer.c.in $(SQLITE_TCL_DEP)
sqlite3_analyzer.c: $(SQLITE3C) $(SQLITE3H) $(TOP)\src\tclsqlite.c $(TOP)\tool\spaceanal.tcl $(TOP)\tool\mkccode.tcl $(TOP)\tool\sqlite3_analyzer.c.in $(TOP)\ext\misc\sqlite3_stdio.h $(TOP)\ext\misc\sqlite3_stdio.c $(SQLITE_TCL_DEP)
$(TCLSH_CMD) $(TOP)\tool\mkccode.tcl $(TOP)\tool\sqlite3_analyzer.c.in > $@
sqlite3_analyzer.exe: sqlite3_analyzer.c $(LIBRESOBJS)
@@ -2500,14 +2643,14 @@ sqlite3_expert.exe: $(SQLITE3C) $(TOP)\ext\expert\sqlite3expert.h $(TOP)\ext\exp
$(LTLINK) $(NO_WARN) $(TOP)\ext\expert\sqlite3expert.c $(TOP)\ext\expert\expert.c $(SQLITE3C) $(TLIBS)
CHECKER_DEPS =\
$(TOP)/tool/mkccode.tcl \
$(TOP)\tool\mkccode.tcl \
sqlite3.c \
$(TOP)/src/tclsqlite.c \
$(TOP)/ext/repair/sqlite3_checker.tcl \
$(TOP)/ext/repair/checkindex.c \
$(TOP)/ext/repair/checkfreelist.c \
$(TOP)/ext/misc/btreeinfo.c \
$(TOP)/ext/repair/sqlite3_checker.c.in
$(TOP)\src\tclsqlite.c \
$(TOP)\ext\repair\sqlite3_checker.tcl \
$(TOP)\ext\repair\checkindex.c \
$(TOP)\ext\repair\checkfreelist.c \
$(TOP)\ext\misc\btreeinfo.c \
$(TOP)\ext\repair\sqlite3_checker.c.in
sqlite3_checker.c: $(CHECKER_DEPS)
$(TCLSH_CMD) $(TOP)\tool\mkccode.tcl $(TOP)\ext\repair\sqlite3_checker.c.in > $@
@@ -2637,7 +2780,7 @@ clean:
del /Q sqlite3.c sqlite3-*.c sqlite3.h 2>NUL
del /Q sqlite3rc.h 2>NUL
del /Q shell.c sqlite3ext.h sqlite3session.h 2>NUL
del /Q sqlite3_analyzer.exe sqlite3_analyzer.c 2>NUL
del /Q sqlite3_analyzer.exe sqlite3_analyzer.c sqlite3-rsync.exe 2>NUL
del /Q sqlite-*-output.vsix 2>NUL
del /Q fuzzershell.exe fuzzcheck.exe sqldiff.exe dbhash.exe 2>NUL
del /Q sqltclsh.* 2>NUL
@@ -2646,4 +2789,5 @@ clean:
del /Q showshm.exe sqlite3_checker.* sqlite3_expert.exe 2>NUL
del /Q fts5.* fts5parse.* 2>NUL
del /Q lsm.h lsm1.c 2>NUL
del /q src-verify.exe 2>NUL
# <</mark>>
+194 -91
View File
@@ -1,14 +1,19 @@
<h1 align="center">SQLite Source Repository</h1>
This repository contains the complete source code for the
[SQLite database engine](https://sqlite.org/). Some test scripts
are also included. However, many other test scripts
[SQLite database engine](https://sqlite.org/), including
many test scripts. However, other test scripts
and most of the documentation are managed separately.
See the [on-line documentation](https://sqlite.org/) for more information
about what SQLite is and how it works from a user's perspective. This
README file is about the source code that goes into building SQLite,
not about how SQLite is used.
## Version Control
SQLite sources are managed using the
[Fossil](https://www.fossil-scm.org/), a distributed version control system
SQLite sources are managed using
[Fossil](https://fossil-scm.org/), a distributed version control system
that was specifically designed and written to support SQLite development.
The [Fossil repository](https://sqlite.org/src/timeline) contains the urtext.
@@ -25,7 +30,28 @@ If you pulled your SQLite source code from a secondary source and want to
verify its integrity, there are hints on how to do that in the
[Verifying Code Authenticity](#vauth) section below.
## Obtaining The Code
## Contacting The SQLite Developers
The preferred way to ask questions or make comments about SQLite or to
report bugs against SQLite is to visit the
[SQLite Forum](https://sqlite.org/forum) at <https://sqlite.org/forum/>.
Anonymous postings are permitted.
If you think you have found a bug that has security implications and
you do not want to report it on the public forum, you can send a private
email to drh at sqlite dot org.
## Public Domain
The SQLite source code is in the public domain. See
<https://sqlite.org/copyright.html> for details.
Because SQLite is in the public domain, we do not normally accept pull
requests, because if we did take a pull request, the changes in that
pull request might carry a copyright and the SQLite source code would
then no longer be fully in the public domain.
## Obtaining The SQLite Source Code
If you do not want to use Fossil, you can download tarballs or ZIP
archives or [SQLite archives](https://sqlite.org/cli.html#sqlar) as follows:
@@ -47,24 +73,26 @@ archives or [SQLite archives](https://sqlite.org/cli.html#sqlar) as follows:
then click on the "Tarball" or "ZIP Archive" links on the information
page.
If you do want to use Fossil to check out the source tree,
To access sources directly using [Fossil](https://fossil-scm.org/home),
first install Fossil version 2.0 or later.
(Source tarballs and precompiled binaries available
[here](https://www.fossil-scm.org/fossil/uv/download.html). Fossil is
Source tarballs and precompiled binaries available at
<https://fossil-scm.org/home/uv/download.html>. Fossil is
a stand-alone program. To install, simply download or build the single
executable file and put that file someplace on your $PATH.)
executable file and put that file someplace on your $PATH.
Then run commands like this:
mkdir -p ~/sqlite ~/Fossils
mkdir -p ~/sqlite
cd ~/sqlite
fossil clone https://www.sqlite.org/src ~/Fossils/sqlite.fossil
fossil open ~/Fossils/sqlite.fossil
fossil open https://sqlite.org/src
After setting up a repository using the steps above, you can always
update to the latest version using:
The "fossil open" command will take two or three minutes. Afterwards,
you can do fast, bandwidth-efficient updates to the whatever versions
of SQLite you like. Some examples:
fossil update trunk ;# latest trunk check-in
fossil update release ;# latest official release
fossil update trunk ;# latest trunk check-in
fossil update release ;# latest official release
fossil update trunk:2024-01-01 ;# First trunk check-in after 2024-01-01
fossil update version-3.39.0 ;# Version 3.39.0
Or type "fossil ui" to get a web-based user interface.
@@ -78,15 +106,42 @@ script found at the root of the source tree. Then run "make".
For example:
tar xzf sqlite.tar.gz ;# Unpack the source tree into "sqlite"
mkdir bld ;# Build will occur in a sibling directory
cd bld ;# Change to the build directory
../sqlite/configure ;# Run the configure script
make ;# Run the makefile.
make sqlite3.c ;# Build the "amalgamation" source file
make test ;# Run some tests (requires Tcl)
apt install gcc make tcl-dev ;# Make sure you have all the necessary build tools
tar xzf sqlite.tar.gz ;# Unpack the source tree into "sqlite"
mkdir bld ;# Build will occur in a sibling directory
cd bld ;# Change to the build directory
../sqlite/configure ;# Run the configure script
make sqlite3 ;# Builds the "sqlite3" command-line tool
make sqlite3.c ;# Build the "amalgamation" source file
make sqldiff ;# Builds the "sqldiff" command-line tool
# Makefile targets below this point require tcl-dev
make tclextension-install ;# Build and install the SQLite TCL extension
make devtest ;# Run development tests
make releasetest ;# Run full release tests
make sqlite3_analyzer ;# Builds the "sqlite3_analyzer" tool
See the makefile for additional targets.
See the makefile for additional targets. For debugging builds, the
core developers typically run "configure" with options like this:
../sqlite/configure --enable-all --enable-debug CFLAGS='-O0 -g'
For release builds, the core developers usually do:
../sqlite/configure --enable-all
Almost all makefile targets require a "tclsh" TCL interpreter version 8.6 or
later. The "tclextension-install" target and the test targets that follow
all require TCL development libraries too. ("apt install tcl-dev"). It is
helpful, but is not required, to install the SQLite TCL extension (the
"tclextension-install" target) prior to running tests. The "releasetest"
target has additional requiremenst, such as "valgrind".
On "make" command-lines, one can add "OPTIONS=..." to specify additional
compile-time options over and above those set by ./configure. For example,
to compile with the SQLITE_OMIT_DEPRECATED compile-time option, one could say:
./configure --enable-all
make OPTIONS=-DSQLITE_OMIT_DEPRECATED sqlite3
The configure script uses autoconf 2.61 and libtool. If the configure
script does not work out for you, there is a generic makefile named
@@ -94,60 +149,85 @@ script does not work out for you, there is a generic makefile named
can copy and edit to suit your needs. Comments on the generic makefile
show what changes are needed.
## Using MSVC for Windows systems
## Compiling for Windows Using MSVC
On Windows, all applicable build products can be compiled with MSVC.
First open the command prompt window associated with the desired compiler
version (e.g. "Developer Command Prompt for VS2013"). Next, use NMAKE
with the provided "Makefile.msc" to build one of the supported targets.
On Windows, everything can be compiled with MSVC.
You will also need a working installation of TCL.
See the [compile-for-windows.md](doc/compile-for-windows.md) document for
additional information about how to install MSVC and TCL and configure your
build environment.
For example, from the parent directory of the source subtree named "sqlite":
If you want to run tests, you need to let SQLite know the location of your
TCL library, using a command like this:
mkdir bld
cd bld
nmake /f ..\sqlite\Makefile.msc TOP=..\sqlite
nmake /f ..\sqlite\Makefile.msc sqlite3.c TOP=..\sqlite
nmake /f ..\sqlite\Makefile.msc sqlite3.dll TOP=..\sqlite
nmake /f ..\sqlite\Makefile.msc sqlite3.exe TOP=..\sqlite
nmake /f ..\sqlite\Makefile.msc test TOP=..\sqlite
set TCLDIR=c:\Tcl
There are several build options that can be set via the NMAKE command
line. For example, to build for WinRT, simply add "FOR_WINRT=1" argument
to the "sqlite3.dll" command line above. When debugging into the SQLite
code, adding the "DEBUG=1" argument to one of the above command lines is
recommended.
SQLite uses "tclsh.exe" as part of the build process, and so that
program will need to be somewhere on your %PATH%. SQLite itself
does not contain any TCL code, but it does use TCL to help with the
build process and to run tests. You may need to install TCL development
libraries in order to successfully complete some makefile targets.
It is helpful, but is not required, to install the SQLite TCL extension
(the "tclextension-install" target) prior to running tests.
SQLite does not require [Tcl](http://www.tcl.tk/) to run, but a Tcl installation
is required by the makefiles (including those for MSVC). SQLite contains
a lot of generated code and Tcl is used to do much of that code generation.
Build using Makefile.msc. Example:
## Source Code Tour
nmake /f Makefile.msc sqlite3.exe
nmake /f Makefile.msc sqlite3.c
nmake /f Makefile.msc sqldiff.exe
# Makefile targets below this point require TCL development libraries
nmake /f Makefile.msc tclextension-install
nmake /f Makefile.msc devtest
nmake /f Makefile.msc releasetest
nmake /f Makefile.msc sqlite3_analyzer.exe
There are many other makefile targets. See comments in Makefile.msc for
details.
Most of the core source files are in the **src/** subdirectory. The
**src/** folder also contains files used to build the "testfixture" test
harness. The names of the source files used by "testfixture" all begin
with "test".
The **src/** also contains the "shell.c" file
which is the main program for the "sqlite3.exe"
[command-line shell](https://sqlite.org/cli.html) and
the "tclsqlite.c" file which implements the
[Tcl bindings](https://sqlite.org/tclsqlite.html) for SQLite.
(Historical note: SQLite began as a Tcl
extension and only later escaped to the wild as an independent library.)
As with the unix Makefile, the OPTIONS=... argument can be passed on the nmake
command-line to enable new compile-time options. For example:
Test scripts and programs are found in the **test/** subdirectory.
Additional test code is found in other source repositories.
See [How SQLite Is Tested](http://www.sqlite.org/testing.html) for
additional information.
nmake /f Makefile.msc OPTIONS=-DSQLITE_OMIT_DEPRECATED sqlite3.exe
The **ext/** subdirectory contains code for extensions. The
Full-text search engine is in **ext/fts3**. The R-Tree engine is in
**ext/rtree**. The **ext/misc** subdirectory contains a number of
smaller, single-file extensions, such as a REGEXP operator.
## Source Tree Map
The **tool/** subdirectory contains various scripts and programs used
for building generated source code files or for testing or for generating
accessory programs such as "sqlite3_analyzer(.exe)".
* **src/** - This directory contains the primary source code for the
SQLite core. For historical reasons, C-code used for testing is
also found here. Source files intended for testing begin with "`test`".
The `tclsqlite3.c` and `tclsqlite3.h` files are the TCL interface
for SQLite and are also not part of the core.
* **test/** - This directory and its subdirectories contains code used
for testing. Files that end in "`.test`" are TCL scripts that run
tests using an augmented TCL interpreter named "testfixture". Use
a command like "`make testfixture`" (unix) or
"`nmake /f Makefile.msc testfixture.exe`" (windows) to build that
augmented TCL interpreter, then run individual tests using commands like
"`testfixture test/main.test`". This test/ subdirectory also contains
additional C code modules and scripts for other kinds of testing.
* **tool/** - This directory contains programs and scripts used to
build some of the machine-generated code that goes into the SQLite
core, as well as to build and run tests and perform diagnostics.
The source code to [the Lemon parser generator](./doc/lemon.html) is
found here. There are also TCL scripts used to build and/or transform
source code files. For example, the tool/mksqlite3h.tcl script reads
the src/sqlite.h.in file and uses it as a template to construct
the deliverable "sqlite3.h" file that defines the SQLite interface.
* **ext/** - Various extensions to SQLite are found under this
directory. For example, the FTS5 subsystem is in "ext/fts5/".
Some of these extensions (ex: FTS3/4, FTS5, RTREE) might get built
into the SQLite amalgamation, but not all of them. The
"ext/misc/" subdirectory contains an assortment of one-file extensions,
many of which are omitted from the SQLite core, but which are included
in the [SQLite CLI](https://sqlite.org/cli.html).
* **doc/** - Some documentation files about SQLite internals are found
here. Note, however, that the primary documentation designed for
application developers and users of SQLite is in a completely separate
repository. Note also that the primary API documentation is derived
from specially constructed comments in the src/sqlite.h.in file.
### Generated Source Code Files
@@ -161,7 +241,7 @@ manually-edited files and automatically-generated files.
The SQLite interface is defined by the **sqlite3.h** header file, which is
generated from src/sqlite.h.in, ./manifest.uuid, and ./VERSION. The
[Tcl script](http://www.tcl.tk) at tool/mksqlite3h.tcl does the conversion.
[Tcl script](https://www.tcl.tk) at tool/mksqlite3h.tcl does the conversion.
The manifest.uuid file contains the SHA3 hash of the particular check-in
and is used to generate the SQLITE\_SOURCE\_ID macro. The VERSION file
contains the current SQLite version number. The sqlite3.h header is really
@@ -228,33 +308,39 @@ individual source file exceeds 32K lines in length.
## How It All Fits Together
SQLite is modular in design.
See the [architectural description](http://www.sqlite.org/arch.html)
See the [architectural description](https://www.sqlite.org/arch.html)
for details. Other documents that are useful in
(helping to understand how SQLite works include the
[file format](http://www.sqlite.org/fileformat2.html) description,
the [virtual machine](http://www.sqlite.org/opcode.html) that runs
helping to understand how SQLite works include the
[file format](https://www.sqlite.org/fileformat2.html) description,
the [virtual machine](https://www.sqlite.org/opcode.html) that runs
prepared statements, the description of
[how transactions work](http://www.sqlite.org/atomiccommit.html), and
the [overview of the query planner](http://www.sqlite.org/optoverview.html).
[how transactions work](https://www.sqlite.org/atomiccommit.html), and
the [overview of the query planner](https://www.sqlite.org/optoverview.html).
Years of effort have gone into optimizing SQLite, both
Decades of effort have gone into optimizing SQLite, both
for small size and high performance. And optimizations tend to result in
complex code. So there is a lot of complexity in the current SQLite
implementation. It will not be the easiest library in the world to hack.
Key files:
### Key source code files
* **sqlite.h.in** - This file defines the public interface to the SQLite
library. Readers will need to be familiar with this interface before
trying to understand how the library works internally.
trying to understand how the library works internally. This file is
really a template that is transformed into the "sqlite3.h" deliverable
using a script invoked by the makefile.
* **sqliteInt.h** - this header file defines many of the data objects
used internally by SQLite. In addition to "sqliteInt.h", some
subsystems have their own header files.
subsystems inside of sQLite have their own header files. These internal
interfaces are not for use by applications. They can and do change
from one release of SQLite to the next.
* **parse.y** - This file describes the LALR(1) grammar that SQLite uses
to parse SQL statements, and the actions that are taken at each step
in the parsing process.
in the parsing process. The file is processed by the
[Lemon Parser Generator](./doc/lemon.html) to produce the actual C code
used for parsing.
* **vdbe.c** - This file implements the virtual machine that runs
prepared statements. There are various helper files whose names
@@ -290,15 +376,17 @@ Key files:
is not part of the core SQLite library. But as most of the tests in this
repository are written in Tcl, the Tcl language bindings are important.
* **test*.c** - Files in the src/ folder that begin with "test" go into
* **test\*.c** - Files in the src/ folder that begin with "test" go into
building the "testfixture.exe" program. The testfixture.exe program is
an enhanced Tcl shell. The testfixture.exe program runs scripts in the
test/ folder to validate the core SQLite code. The testfixture program
(and some other test programs too) is built and run when you type
"make test".
* **ext/misc/json1.c** - This file implements the various JSON functions
that are built into SQLite.
* **VERSION**, **manifest**, and **manifest.uuid** - These files define
the current SQLite version number. The "VERSION" file is human generated,
but the "manifest" and "manifest.uuid" files are automatically generated
by the [Fossil version control system](https://fossil-scm.org/).
There are many other source files. Each has a succinct header comment that
describes its purpose and role within the larger system.
@@ -307,8 +395,8 @@ describes its purpose and role within the larger system.
## Verifying Code Authenticity
The `manifest` file at the root directory of the source tree
contains either a SHA3-256 hash (for newer files) or a SHA1 hash (for
older files) for every source file in the repository.
contains either a SHA3-256 hash or a SHA1 hash
for every source file in the repository.
The name of the version of the entire source tree is just the
SHA3-256 hash of the `manifest` file itself, possibly with the
last line of that file omitted if the last line begins with
@@ -316,14 +404,29 @@ last line of that file omitted if the last line begins with
The `manifest.uuid` file should contain the SHA3-256 hash of the
`manifest` file. If all of the above hash comparisons are correct, then
you can be confident that your source tree is authentic and unadulterated.
Details on the format for the `manifest` files are available
[on the Fossil website](https://fossil-scm.org/home/doc/trunk/www/fileformat.wiki#manifest).
The format of the `manifest` file should be mostly self-explanatory, but
if you want details, they are available
[here](https://fossil-scm.org/fossil/doc/trunk/www/fileformat.wiki#manifest).
The process of checking source code authenticity is automated by the
makefile:
> make verify-source
Or on windows:
> nmake /f Makefile.msc verify-source
Using the makefile to verify source integrity is good for detecting
accidental changes to the source tree, but malicious changes could be
hidden by also modifying the makefiles.
## Contacts
The main SQLite website is [http://www.sqlite.org/](http://www.sqlite.org/)
The main SQLite website is [https://sqlite.org/](https://sqlite.org/)
with geographically distributed backups at
[http://www2.sqlite.org/](http://www2.sqlite.org) and
[http://www3.sqlite.org/](http://www3.sqlite.org).
[https://www2.sqlite.org/](https://www2.sqlite.org) and
[https://www3.sqlite.org/](https://www3.sqlite.org).
Contact the SQLite developers through the
[SQLite Forum](https://sqlite.org/forum/). In an emergency, you
can send private email to the lead developer at drh at sqlite dot org.
+1 -1
View File
@@ -1 +1 @@
3.39.0
3.47.0
Binary file not shown.

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 3.3 KiB

+1 -1
View File
@@ -9,7 +9,7 @@ sqlite3_SOURCES = shell.c sqlite3.h
EXTRA_sqlite3_SOURCES = sqlite3.c
sqlite3_LDADD = @EXTRA_SHELL_OBJ@ @READLINE_LIBS@
sqlite3_DEPENDENCIES = @EXTRA_SHELL_OBJ@
sqlite3_CFLAGS = $(AM_CFLAGS) -DSQLITE_ENABLE_EXPLAIN_COMMENTS -DSQLITE_ENABLE_DBPAGE_VTAB -DSQLITE_ENABLE_STMTVTAB -DSQLITE_ENABLE_DBSTAT_VTAB $(SHELL_CFLAGS)
sqlite3_CFLAGS = $(AM_CFLAGS) -DSQLITE_ENABLE_EXPLAIN_COMMENTS -DSQLITE_DQS=0 -DSQLITE_ENABLE_DBPAGE_VTAB -DSQLITE_ENABLE_STMTVTAB -DSQLITE_ENABLE_DBSTAT_VTAB $(SHELL_CFLAGS)
include_HEADERS = sqlite3.h sqlite3ext.h
+49
View File
@@ -18,6 +18,13 @@
TOP = .
# Optionally set EXTRA_SRC to a list of C files to append to
# the generated sqlite3.c.
#
!IFNDEF EXTRA_SRC
EXTRA_SRC =
!ENDIF
# Set this non-0 to enable full warnings (-W4, etc) when compiling.
#
!IFNDEF USE_FULLWARN
@@ -52,6 +59,13 @@ MINIMAL_AMALGAMATION = 0
USE_STDCALL = 0
!ENDIF
# Use the USE_SEH=0 option on the nmake command line to omit structured
# exception handling (SEH) support. SEH is on by default.
#
!IFNDEF USE_SEH
USE_SEH = 1
!ENDIF
# Set this non-0 to have the shell executable link against the core dynamic
# link library.
#
@@ -180,6 +194,12 @@ WIN32HEAP = 0
OSTRACE = 0
!ENDIF
# enable address sanitizer using ASAN=1 on the command-line.
#
!IFNDEF ASAN
ASAN = 0
!ENDIF
# Set this to one of the following values to enable various debugging
# features. Each level includes the debugging options from the previous
# levels. Currently, the recognized values for DEBUG are:
@@ -281,8 +301,10 @@ SQLITE3EXEPDB = /pdb:sqlite3sh.pdb
# the Windows platform.
#
!IFNDEF OPT_FEATURE_FLAGS
OPT_FEATURE_FLAGS = $(OPT_XTRA)
!IF $(MINIMAL_AMALGAMATION)==0
OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_FTS3=1
OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_FTS5=1
OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_RTREE=1
OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_GEOPOLY=1
OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_STMTVTAB=1
@@ -293,6 +315,14 @@ OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_BYTECODE_VTAB=1
OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_COLUMN_METADATA=1
!ENDIF
# Additional feature-options above and beyond what are normally used can be
# be added using OPTIONS=.... on the command-line. These values are
# appended to the OPT_FEATURE_FLAGS variable.
#
!IFDEF OPTIONS
OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) $(OPTIONS)
!ENDIF
# Should the session extension be enabled? If so, add compilation options
# to enable it.
#
@@ -311,6 +341,14 @@ OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_MATH_FUNCTIONS
OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_RBU=1
!ENDIF
# Should structured exception handling (SEH) be enabled for WAL mode in
# the core library? It is on by default. Only omit it if the
# USE_SEH=0 option is provided on the nmake command-line.
#
!IF $(USE_SEH)==0
OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_OMIT_SEH=1
!ENDIF
# These are the "extended" SQLite compilation options used when compiling for
# the Windows 10 platform.
#
@@ -718,6 +756,13 @@ RCC = $(RCC) -DSQLITE_WIN32_MALLOC_VALIDATE=1
!ENDIF
# Address sanitizer if ASAN=1
#
!IF $(ASAN)>0
TCC = $(TCC) /fsanitize=address
!ENDIF
# Compiler options needed for programs that use the readline() library.
#
!IFNDEF READLINE_FLAGS
@@ -955,9 +1000,13 @@ LIBRESOBJS =
# when the shell is not being dynamically linked.
#
!IF $(DYNAMIC_SHELL)==0 && $(FOR_WIN10)==0
SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_DQS=0
SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_FTS4=1
SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_EXPLAIN_COMMENTS=1
SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_OFFSET_SQL_FUNC=1
SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_UNKNOWN_SQL_FUNCTION=1
SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_STMT_SCANSTATUS=1
SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_STRICT_SUBTYPE=1
!ENDIF
+109 -74
View File
@@ -11,8 +11,6 @@
#
# See the file "license.terms" for information on usage and redistribution
# of this file, and for a DISCLAIMER OF ALL WARRANTIES.
#
# RCS: @(#) $Id: Makefile.in,v 1.59 2005/07/26 19:17:02 mdejong Exp $
#========================================================================
# Add additional lines to handle any additional AC_SUBST cases that
@@ -60,6 +58,8 @@ PKG_HEADERS = @PKG_HEADERS@
#========================================================================
PKG_LIB_FILE = @PKG_LIB_FILE@
PKG_LIB_FILE8 = @PKG_LIB_FILE8@
PKG_LIB_FILE9 = @PKG_LIB_FILE9@
PKG_STUB_LIB_FILE = @PKG_STUB_LIB_FILE@
lib_BINARIES = $(PKG_LIB_FILE)
@@ -73,10 +73,11 @@ exec_prefix = @exec_prefix@
bindir = @bindir@
libdir = @libdir@
includedir = @includedir@
datarootdir = @datarootdir@
runstatedir = @runstatedir@
datadir = @datadir@
mandir = @mandir@
includedir = @includedir@
DESTDIR =
@@ -85,24 +86,25 @@ pkgdatadir = $(datadir)/$(PKG_DIR)
pkglibdir = $(libdir)/$(PKG_DIR)
pkgincludedir = $(includedir)/$(PKG_DIR)
top_builddir = .
top_builddir = @abs_top_builddir@
INSTALL = @INSTALL@
INSTALL_PROGRAM = @INSTALL_PROGRAM@
INSTALL_OPTIONS =
INSTALL = @INSTALL@ $(INSTALL_OPTIONS)
INSTALL_DATA_DIR = @INSTALL_DATA_DIR@
INSTALL_DATA = @INSTALL_DATA@
INSTALL_PROGRAM = @INSTALL_PROGRAM@
INSTALL_SCRIPT = @INSTALL_SCRIPT@
INSTALL_LIBRARY = @INSTALL_LIBRARY@
PACKAGE_NAME = @PACKAGE_NAME@
PACKAGE_VERSION = @PACKAGE_VERSION@
CC = @CC@
CCLD = @CCLD@
CFLAGS_DEFAULT = @CFLAGS_DEFAULT@
CFLAGS_WARNING = @CFLAGS_WARNING@
CLEANFILES = @CLEANFILES@
EXEEXT = @EXEEXT@
LDFLAGS_DEFAULT = @LDFLAGS_DEFAULT@
MAKE_LIB = @MAKE_LIB@
MAKE_SHARED_LIB = @MAKE_SHARED_LIB@
MAKE_STATIC_LIB = @MAKE_STATIC_LIB@
MAKE_STUB_LIB = @MAKE_STUB_LIB@
OBJEXT = @OBJEXT@
RANLIB = @RANLIB@
@@ -117,8 +119,6 @@ TCL_SRC_DIR = @TCL_SRC_DIR@
#TK_BIN_DIR = @TK_BIN_DIR@
#TK_SRC_DIR = @TK_SRC_DIR@
# This is no longer necessary even for packages that use private Tcl headers
#TCL_TOP_DIR_NATIVE = @TCL_TOP_DIR_NATIVE@
# Not used, but retained for reference of what libs Tcl required
#TCL_LIBS = @TCL_LIBS@
@@ -132,41 +132,52 @@ TCL_SRC_DIR = @TCL_SRC_DIR@
EXTRA_PATH = $(top_builddir):$(TCL_BIN_DIR)
#EXTRA_PATH = $(top_builddir):$(TCL_BIN_DIR):$(TK_BIN_DIR)
TCLLIBPATH = $(top_builddir)
TCLSH_ENV = TCL_LIBRARY=`@CYGPATH@ $(TCL_SRC_DIR)/library` \
@LD_LIBRARY_PATH_VAR@="$(EXTRA_PATH):$(@LD_LIBRARY_PATH_VAR@)" \
TCLSH_ENV = TCL_LIBRARY=`@CYGPATH@ $(TCL_SRC_DIR)/library`
PKG_ENV = @LD_LIBRARY_PATH_VAR@="$(EXTRA_PATH):$(@LD_LIBRARY_PATH_VAR@)" \
PATH="$(EXTRA_PATH):$(PATH)" \
TCLLIBPATH="$(TCLLIBPATH)"
# TK_LIBRARY=`@CYGPATH@ $(TK_SRC_DIR)/library`
TCLSH_PROG = @TCLSH_PROG@
TCLSH = $(TCLSH_ENV) $(TCLSH_PROG)
TCLSH = $(TCLSH_ENV) $(PKG_ENV) $(TCLSH_PROG)
#WISH_ENV = TK_LIBRARY=`@CYGPATH@ $(TK_SRC_DIR)/library`
#WISH_PROG = @WISH_PROG@
#WISH = $(TCLSH_ENV) $(WISH_PROG)
#WISH = $(TCLSH_ENV) $(WISH_ENV) $(PKG_ENV) $(WISH_PROG)
SHARED_BUILD = @SHARED_BUILD@
INCLUDES = @PKG_INCLUDES@ @TCL_INCLUDES@ -I$(srcdir)/..
INCLUDES = @PKG_INCLUDES@ @TCL_INCLUDES@ -I. -I$(srcdir)/..
#INCLUDES = @PKG_INCLUDES@ @TCL_INCLUDES@ @TK_INCLUDES@ @TK_XINCLUDES@
PKG_CFLAGS = @PKG_CFLAGS@
# TCL_DEFS is not strictly need here, but if you remove it, then you
# must make sure that configure.in checks for the necessary components
# must make sure that configure.ac checks for the necessary components
# that your library may use. TCL_DEFS can actually be a problem if
# you do not compile with a similar machine setup as the Tcl core was
# compiled with.
#DEFS = $(TCL_DEFS) @DEFS@ $(PKG_CFLAGS)
DEFS = @DEFS@ $(PKG_CFLAGS)
# Move pkgIndex.tcl to 'BINARIES' var if it is generated in the Makefile
CONFIG_CLEAN_FILES = Makefile pkgIndex.tcl
CLEANFILES = @CLEANFILES@
CPPFLAGS = @CPPFLAGS@
LIBS = @PKG_LIBS@ @LIBS@
AR = @AR@
CFLAGS = @CFLAGS@
COMPILE = $(CC) $(DEFS) $(INCLUDES) $(AM_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS)
LDFLAGS = @LDFLAGS@
LDFLAGS_DEFAULT = @LDFLAGS_DEFAULT@
COMPILE = $(CC) $(DEFS) $(INCLUDES) $(AM_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) \
$(CFLAGS_DEFAULT) $(CFLAGS_WARNING) $(SHLIB_CFLAGS) $(CFLAGS)
GDB = gdb
VALGRIND = valgrind
VALGRINDARGS = --tool=memcheck --num-callers=8 --leak-resolution=high \
--leak-check=yes --show-reachable=yes -v
.SUFFIXES: .c .$(OBJEXT)
#========================================================================
# Start of user-definable TARGETS section
@@ -174,7 +185,7 @@ COMPILE = $(CC) $(DEFS) $(INCLUDES) $(AM_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(C
#========================================================================
# TEA TARGETS. Please note that the "libraries:" target refers to platform
# independent files, and the "binaries:" target inclues executable programs and
# independent files, and the "binaries:" target includes executable programs and
# platform-dependent libraries. Modify these targets so that they install
# the various pieces of your package. The make and install rules
# for the BINARIES that you specified above have already been done.
@@ -193,7 +204,6 @@ binaries: $(BINARIES)
libraries:
#========================================================================
# Your doc target should differentiate from doc builds (by the developer)
# and doc installs (see install-doc), which just install the docs on the
@@ -216,11 +226,11 @@ install-binaries: binaries install-lib-binaries install-bin-binaries
#========================================================================
install-libraries: libraries
@mkdir -p $(DESTDIR)$(includedir)
@$(INSTALL_DATA_DIR) "$(DESTDIR)$(includedir)"
@echo "Installing header files in $(DESTDIR)$(includedir)"
@list='$(PKG_HEADERS)'; for i in $$list; do \
echo "Installing $(srcdir)/$$i" ; \
$(INSTALL_DATA) $(srcdir)/$$i $(DESTDIR)$(includedir) ; \
$(INSTALL_DATA) $(srcdir)/$$i "$(DESTDIR)$(includedir)" ; \
done;
#========================================================================
@@ -229,12 +239,11 @@ install-libraries: libraries
#========================================================================
install-doc: doc
@mkdir -p $(DESTDIR)$(mandir)/mann
@$(INSTALL_DATA_DIR) "$(DESTDIR)$(mandir)/mann"
@echo "Installing documentation in $(DESTDIR)$(mandir)"
@list='$(srcdir)/doc/*.n'; for i in $$list; do \
echo "Installing $$i"; \
rm -f $(DESTDIR)$(mandir)/mann/`basename $$i`; \
$(INSTALL_DATA) $$i $(DESTDIR)$(mandir)/mann ; \
$(INSTALL_DATA) $$i "$(DESTDIR)$(mandir)/mann" ; \
done
test: binaries libraries
@@ -244,7 +253,21 @@ shell: binaries libraries
@$(TCLSH) $(SCRIPT)
gdb:
$(TCLSH_ENV) gdb $(TCLSH_PROG) $(SCRIPT)
$(TCLSH_ENV) $(PKG_ENV) $(GDB) $(TCLSH_PROG) $(SCRIPT)
gdb-test: binaries libraries
$(TCLSH_ENV) $(PKG_ENV) $(GDB) \
--args $(TCLSH_PROG) `@CYGPATH@ $(srcdir)/tests/all.tcl` \
$(TESTFLAGS) -singleproc 1 \
-load "package ifneeded $(PACKAGE_NAME) $(PACKAGE_VERSION) \
[list load `@CYGPATH@ $(PKG_LIB_FILE)` [string totitle $(PACKAGE_NAME)]]"
valgrind: binaries libraries
$(TCLSH_ENV) $(PKG_ENV) $(VALGRIND) $(VALGRINDARGS) $(TCLSH_PROG) \
`@CYGPATH@ $(srcdir)/tests/all.tcl` $(TESTFLAGS)
valgrindshell: binaries libraries
$(TCLSH_ENV) $(PKG_ENV) $(VALGRIND) $(VALGRINDARGS) $(TCLSH_PROG) $(SCRIPT)
depend:
@@ -283,49 +306,70 @@ $(PKG_STUB_LIB_FILE): $(PKG_STUB_OBJECTS)
# As necessary, add $(srcdir):$(srcdir)/compat:....
#========================================================================
VPATH = $(srcdir):$(srcdir)/generic:$(srcdir)/unix:$(srcdir)/win
VPATH = $(srcdir):$(srcdir)/generic:$(srcdir)/unix:$(srcdir)/win:$(srcdir)/macosx
.c.@OBJEXT@:
$(COMPILE) -c `@CYGPATH@ $<` -o $@
tclsample.@OBJEXT@: sampleUuid.h
$(srcdir)/manifest.uuid:
printf "git-" >$(srcdir)/manifest.uuid
(cd $(srcdir); git rev-parse HEAD >>$(srcdir)/manifest.uuid || \
(printf "svn-r" >$(srcdir)/manifest.uuid ; \
svn info --show-item last-changed-revision >>$(srcdir)/manifest.uuid) || \
printf "unknown" >$(srcdir)/manifest.uuid)
sampleUuid.h: $(srcdir)/manifest.uuid
echo "#define SAMPLE_VERSION_UUID \\" >$@
cat $(srcdir)/manifest.uuid >>$@
echo "" >>$@
#========================================================================
# Distribution creation
# You may need to tweak this target to make it work correctly.
#========================================================================
#COMPRESS = tar cvf $(PKG_DIR).tar $(PKG_DIR); compress $(PKG_DIR).tar
COMPRESS = gtar zcvf $(PKG_DIR).tar.gz $(PKG_DIR)
COMPRESS = tar zcvf $(PKG_DIR).tar.gz $(PKG_DIR)
DIST_ROOT = /tmp/dist
DIST_DIR = $(DIST_ROOT)/$(PKG_DIR)
DIST_INSTALL_DATA = CPPROG='cp -p' $(INSTALL) -m 644
DIST_INSTALL_SCRIPT = CPPROG='cp -p' $(INSTALL) -m 755
dist-clean:
rm -rf $(DIST_DIR) $(DIST_ROOT)/$(PKG_DIR).tar.*
dist: dist-clean
mkdir -p $(DIST_DIR)
cp -p $(srcdir)/README* $(srcdir)/license* \
$(srcdir)/aclocal.m4 $(srcdir)/configure $(srcdir)/*.in \
$(DIST_DIR)/
chmod 664 $(DIST_DIR)/Makefile.in $(DIST_DIR)/aclocal.m4
chmod 775 $(DIST_DIR)/configure $(DIST_DIR)/configure.in
dist: dist-clean $(srcdir)/manifest.uuid
$(INSTALL_DATA_DIR) $(DIST_DIR)
for i in $(srcdir)/*.[ch]; do \
if [ -f $$i ]; then \
cp -p $$i $(DIST_DIR)/ ; \
fi; \
done;
# TEA files
$(DIST_INSTALL_DATA) $(srcdir)/Makefile.in \
$(srcdir)/aclocal.m4 $(srcdir)/configure.ac \
$(DIST_DIR)/
$(DIST_INSTALL_SCRIPT) $(srcdir)/configure $(DIST_DIR)/
mkdir $(DIST_DIR)/tclconfig
cp $(srcdir)/tclconfig/install-sh $(srcdir)/tclconfig/tcl.m4 \
$(DIST_DIR)/tclconfig/
chmod 664 $(DIST_DIR)/tclconfig/tcl.m4
chmod +x $(DIST_DIR)/tclconfig/install-sh
$(INSTALL_DATA_DIR) $(DIST_DIR)/tclconfig
$(DIST_INSTALL_DATA) $(srcdir)/tclconfig/README.txt \
$(srcdir)/manifest.uuid \
$(srcdir)/tclconfig/tcl.m4 $(srcdir)/tclconfig/install-sh \
$(DIST_DIR)/tclconfig/
list='demos doc generic library mac tests unix win'; \
# Extension files
$(DIST_INSTALL_DATA) \
$(srcdir)/ChangeLog \
$(srcdir)/README.sha \
$(srcdir)/license.terms \
$(srcdir)/README \
$(srcdir)/pkgIndex.tcl.in \
$(DIST_DIR)/
list='demos doc generic library macosx tests unix win'; \
for p in $$list; do \
if test -d $(srcdir)/$$p ; then \
mkdir $(DIST_DIR)/$$p; \
cp -p $(srcdir)/$$p/*.* $(DIST_DIR)/$$p/; \
$(INSTALL_DATA_DIR) $(DIST_DIR)/$$p; \
$(DIST_INSTALL_DATA) $(srcdir)/$$p/* $(DIST_DIR)/$$p/; \
fi; \
done
@@ -337,10 +381,10 @@ dist: dist-clean
#========================================================================
# Don't modify the file to clean here. Instead, set the "CLEANFILES"
# variable in configure.in
# variable in configure.ac
#========================================================================
clean:
clean:
-test -z "$(BINARIES)" || rm -f $(BINARIES)
-rm -f *.$(OBJEXT) core *.core
-test -z "$(CLEANFILES)" || rm -f $(CLEANFILES)
@@ -348,7 +392,7 @@ clean:
distclean: clean
-rm -f *.tab.c
-rm -f $(CONFIG_CLEAN_FILES)
-rm -f config.h config.cache config.log config.status
-rm -f config.cache config.log config.status
#========================================================================
# Install binary object libraries. On Windows this includes both .dll and
@@ -362,25 +406,17 @@ distclean: clean
#========================================================================
install-lib-binaries: binaries
@mkdir -p $(DESTDIR)$(pkglibdir)
@$(INSTALL_DATA_DIR) "$(DESTDIR)$(pkglibdir)"
@list='$(lib_BINARIES)'; for p in $$list; do \
if test -f $$p; then \
echo " $(INSTALL_PROGRAM) $$p $(DESTDIR)$(pkglibdir)/$$p"; \
$(INSTALL_PROGRAM) $$p $(DESTDIR)$(pkglibdir)/$$p; \
stub=`echo $$p|sed -e "s/.*\(stub\).*/\1/"`; \
if test "x$$stub" = "xstub"; then \
echo " $(RANLIB_STUB) $(DESTDIR)$(pkglibdir)/$$p"; \
$(RANLIB_STUB) $(DESTDIR)$(pkglibdir)/$$p; \
else \
echo " $(RANLIB) $(DESTDIR)$(pkglibdir)/$$p"; \
$(RANLIB) $(DESTDIR)$(pkglibdir)/$$p; \
fi; \
echo " $(INSTALL_LIBRARY) $$p $(DESTDIR)$(pkglibdir)/$$p"; \
$(INSTALL_LIBRARY) $$p "$(DESTDIR)$(pkglibdir)/$$p"; \
ext=`echo $$p|sed -e "s/.*\.//"`; \
if test "x$$ext" = "xdll"; then \
lib=`basename $$p|sed -e 's/.[^.]*$$//'`.lib; \
if test -f $$lib; then \
echo " $(INSTALL_DATA) $$lib $(DESTDIR)$(pkglibdir)/$$lib"; \
$(INSTALL_DATA) $$lib $(DESTDIR)$(pkglibdir)/$$lib; \
$(INSTALL_DATA) $$lib "$(DESTDIR)$(pkglibdir)/$$lib"; \
fi; \
fi; \
fi; \
@@ -389,12 +425,12 @@ install-lib-binaries: binaries
if test -f $(srcdir)/$$p; then \
destp=`basename $$p`; \
echo " Install $$destp $(DESTDIR)$(pkglibdir)/$$destp"; \
$(INSTALL_DATA) $(srcdir)/$$p $(DESTDIR)$(pkglibdir)/$$destp; \
$(INSTALL_DATA) $(srcdir)/$$p "$(DESTDIR)$(pkglibdir)/$$destp"; \
fi; \
done
@if test "x$(SHARED_BUILD)" = "x1"; then \
echo " Install pkgIndex.tcl $(DESTDIR)$(pkglibdir)"; \
$(INSTALL_DATA) pkgIndex.tcl $(DESTDIR)$(pkglibdir); \
$(INSTALL_DATA) pkgIndex.tcl "$(DESTDIR)$(pkglibdir)"; \
fi
#========================================================================
@@ -407,33 +443,32 @@ install-lib-binaries: binaries
#========================================================================
install-bin-binaries: binaries
@mkdir -p $(DESTDIR)$(bindir)
@$(INSTALL_DATA_DIR) "$(DESTDIR)$(bindir)"
@list='$(bin_BINARIES)'; for p in $$list; do \
if test -f $$p; then \
echo " $(INSTALL_PROGRAM) $$p $(DESTDIR)$(bindir)/$$p"; \
$(INSTALL_PROGRAM) $$p $(DESTDIR)$(bindir)/$$p; \
$(INSTALL_PROGRAM) $$p "$(DESTDIR)$(bindir)/$$p"; \
fi; \
done
.SUFFIXES: .c .$(OBJEXT)
Makefile: $(srcdir)/Makefile.in $(top_builddir)/config.status
cd $(top_builddir) \
&& CONFIG_FILES=$@ CONFIG_HEADERS= $(SHELL) ./config.status
uninstall-binaries:
list='$(lib_BINARIES)'; for p in $$list; do \
rm -f $(DESTDIR)$(pkglibdir)/$$p; \
rm -f "$(DESTDIR)$(pkglibdir)/$$p"; \
done
list='$(PKG_TCL_SOURCES)'; for p in $$list; do \
p=`basename $$p`; \
rm -f $(DESTDIR)$(pkglibdir)/$$p; \
rm -f "$(DESTDIR)$(pkglibdir)/$$p"; \
done
list='$(bin_BINARIES)'; for p in $$list; do \
rm -f $(DESTDIR)$(bindir)/$$p; \
rm -f "$(DESTDIR)$(bindir)/$$p"; \
done
.PHONY: all binaries clean depend distclean doc install libraries test
.PHONY: gdb gdb-test valgrind valgrindshell
# Tell versions [3.59,3.63) of GNU make to not export all variables.
# Otherwise a system limit (for SysV at least) may be exceeded.
+60 -34
View File
@@ -2,11 +2,9 @@
dnl This file is an input file used by the GNU "autoconf" program to
dnl generate the file "configure", which is run during Tcl installation
dnl to configure the system for the local environment.
#
# RCS: @(#) $Id: configure.in,v 1.43 2005/07/26 19:17:05 mdejong Exp $
#-----------------------------------------------------------------------
# Sample configure.in for Tcl Extensions. The only places you should
# Sample configure.ac for Tcl Extensions. The only places you should
# need to modify this file are marked by the string __CHANGE__
#-----------------------------------------------------------------------
@@ -17,9 +15,11 @@ dnl to configure the system for the local environment.
# This initializes the environment with PACKAGE_NAME and PACKAGE_VERSION
# set as provided. These will also be added as -D defs in your Makefile
# so you can encode the package version directly into the source files.
# This will also define a special symbol for Windows (BUILD_<PACKAGE_NAME>
# so that we create the export library with the dll.
#-----------------------------------------------------------------------
AC_INIT([sqlite], [3.32.0])
AC_INIT([sqlite],[3.47.0])
#--------------------------------------------------------------------
# Call TEA_INIT as the first TEA_ macro to set up initial vars.
@@ -27,7 +27,7 @@ AC_INIT([sqlite], [3.32.0])
# as well as PKG_LIB_FILE and PKG_STUB_LIB_FILE.
#--------------------------------------------------------------------
TEA_INIT([3.9])
TEA_INIT()
AC_CONFIG_AUX_DIR(tclconfig)
@@ -55,8 +55,8 @@ TEA_PREFIX
#-----------------------------------------------------------------------
# Standard compiler checks.
# This sets up CC by using the CC env var, or looks for gcc otherwise.
# This also calls AC_PROG_CC, AC_PROG_INSTALL and a few others to create
# the basic setup necessary to compile executables.
# This also calls AC_PROG_CC and a few others to create the basic setup
# necessary to compile executables.
#-----------------------------------------------------------------------
TEA_SETUP_COMPILER
@@ -73,11 +73,19 @@ TEA_SETUP_COMPILER
TEA_ADD_SOURCES([tclsqlite3.c])
TEA_ADD_HEADERS([])
TEA_ADD_INCLUDES([-I\"`\${CYGPATH} \${srcdir}/generic`\"])
TEA_ADD_INCLUDES([])
TEA_ADD_LIBS([])
TEA_ADD_CFLAGS([-DSQLITE_ENABLE_FTS3=1])
TEA_ADD_CFLAGS([-DSQLITE_ENABLE_FTS4=1])
TEA_ADD_CFLAGS([-DSQLITE_ENABLE_FTS5=1])
TEA_ADD_CFLAGS([-DSQLITE_3_SUFFIX_ONLY=1])
TEA_ADD_CFLAGS([-DSQLITE_ENABLE_RTREE=1])
TEA_ADD_CFLAGS([-DSQLITE_ENABLE_GEOPOLY=1])
TEA_ADD_CFLAGS([-DSQLITE_ENABLE_MATH_FUNCTIONS=1])
TEA_ADD_CFLAGS([-DSQLITE_ENABLE_DESERIALIZE=1])
TEA_ADD_CFLAGS([-DSQLITE_ENABLE_DBPAGE_VTAB=1])
TEA_ADD_CFLAGS([-DSQLITE_ENABLE_BYTECODE_VTAB=1])
TEA_ADD_CFLAGS([-DSQLITE_ENABLE_DBSTAT_VTAB=1])
TEA_ADD_STUB_SOURCES([])
TEA_ADD_TCL_SOURCES([])
@@ -99,6 +107,31 @@ if test x$with_system_sqlite != xno; then
LIBS="$LIBS -lsqlite3"])])
fi
#--------------------------------------------------------------------
# __CHANGE__
#
# You can add more files to clean if your extension creates any extra
# files by extending CLEANFILES.
# Add pkgIndex.tcl if it is generated in the Makefile instead of ./configure
# and change Makefile.in to move it from CONFIG_CLEAN_FILES to BINARIES var.
#
# A few miscellaneous platform-specific items:
# TEA_ADD_* any platform specific compiler/build info here.
#--------------------------------------------------------------------
#CLEANFILES="$CLEANFILES pkgIndex.tcl"
if test "${TEA_PLATFORM}" = "windows" ; then
# Ensure no empty if clauses
:
#TEA_ADD_SOURCES([win/winFile.c])
#TEA_ADD_INCLUDES([-I\"$(${CYGPATH} ${srcdir}/win)\"])
else
# Ensure no empty else clauses
:
#TEA_ADD_SOURCES([unix/unixFile.c])
#TEA_ADD_LIBS([-lsuperfly])
fi
#--------------------------------------------------------------------
# __CHANGE__
# Choose which headers you need. Extension authors should try very
@@ -151,28 +184,6 @@ TEA_CONFIG_CFLAGS
TEA_ENABLE_SYMBOLS
#--------------------------------------------------------------------
# Everyone should be linking against the Tcl stub library. If you
# can't for some reason, remove this definition. If you aren't using
# stubs, you also need to modify the SHLIB_LD_LIBS setting below to
# link against the non-stubbed Tcl library. Add Tk too if necessary.
#--------------------------------------------------------------------
AC_DEFINE(USE_TCL_STUBS, 1, [Use Tcl stubs])
#AC_DEFINE(USE_TK_STUBS, 1, [Use Tk stubs])
#--------------------------------------------------------------------
# Redefine fdatasync as fsync on systems that lack fdatasync
#--------------------------------------------------------------------
#
#AC_CHECK_FUNC(fdatasync, , AC_DEFINE(fdatasync, fsync))
# Check for library functions that SQLite can optionally use.
AC_CHECK_FUNCS([fdatasync usleep fullfsync localtime_r gmtime_r])
AC_FUNC_STRERROR_R
#--------------------------------------------------------------------
# This macro generates a line to use when building a library. It
# depends on values set by the TEA_ENABLE_SHARED, TEA_ENABLE_SYMBOLS,
@@ -193,9 +204,24 @@ TEA_PROG_TCLSH
#TEA_PROG_WISH
#--------------------------------------------------------------------
# Finally, substitute all of the various values into the Makefile.
# You may alternatively have a special pkgIndex.tcl.in or other files
# which require substituting th AC variables in. Include these here.
# Setup a *Config.sh.in configuration file.
#--------------------------------------------------------------------
AC_OUTPUT([Makefile pkgIndex.tcl])
#TEA_EXPORT_CONFIG([sample])
#AC_SUBST(SAMPLE_VAR)
#--------------------------------------------------------------------
# Specify files to substitute AC variables in. You may alternatively
# have a special pkgIndex.tcl.in or other files which require
# substituting the AC variables in. Include these here.
#--------------------------------------------------------------------
AC_CONFIG_FILES([Makefile pkgIndex.tcl])
#AC_CONFIG_FILES([sampleConfig.sh])
#--------------------------------------------------------------------
# Finally, substitute all of the various values into the files
# specified with AC_CONFIG_FILES.
#--------------------------------------------------------------------
AC_OUTPUT
+9 -6
View File
@@ -1,7 +1,10 @@
# -*- tcl -*-
# Tcl package index file, version 1.1
#
# Tcl package index file
#
# Note sqlite*3* init specifically
#
package ifneeded sqlite3 @PACKAGE_VERSION@ \
[list load [file join $dir @PKG_LIB_FILE@] Sqlite3]
if {[package vsatisfies [package provide Tcl] 9.0-]} {
package ifneeded sqlite3 @PACKAGE_VERSION@ \
[list load [file join $dir @PKG_LIB_FILE9@] sqlite3]
} else {
package ifneeded sqlite3 @PACKAGE_VERSION@ \
[list load [file join $dir @PKG_LIB_FILE8@] sqlite3]
}
File diff suppressed because it is too large Load Diff
+12 -1
View File
@@ -251,7 +251,18 @@ INCLUDES = $(SQL_INCLUDES) $(TCL_INCLUDES) -I"$(WINDIR)" \
-I"$(GENERICDIR)" -I"$(ROOT)\.."
BASE_CLFAGS = $(cflags) $(cdebug) $(crt) $(INCLUDES) \
-DSQLITE_3_SUFFIX_ONLY=1 -DSQLITE_ENABLE_RTREE=1 \
-DSQLITE_ENABLE_FTS3=1 -DSQLITE_OMIT_DEPRECATED=1
-DSQLITE_ENABLE_FTS3=1 -DSQLITE_OMIT_DEPRECATED=1 \
-DSQLITE_ENABLE_FTS4=1 \
-DSQLITE_ENABLE_FTS5=1 \
-DSQLITE_3_SUFFIX_ONLY=1 \
-DSQLITE_ENABLE_RTREE=1 \
-DSQLITE_ENABLE_GEOPOLY=1 \
-DSQLITE_ENABLE_MATH_FUNCTIONS=1 \
-DSQLITE_ENABLE_DESERIALIZE=1 \
-DSQLITE_ENABLE_DBPAGE_VTAB=1 \
-DSQLITE_ENABLE_BYTECODE_VTAB=1 \
-DSQLITE_ENABLE_DBSTAT_VTAB=1
CON_CFLAGS = $(cflags) $(cdebug) $(crt) -DCONSOLE -DSQLITE_ENABLE_FTS3=1
TCL_CFLAGS = -DBUILD_sqlite -DUSE_TCL_STUBS \
-DPACKAGE_VERSION="\"$(DOTVERSION)\"" $(BASE_CLFAGS) \
-1
View File
@@ -708,4 +708,3 @@ TK_INCLUDES = -I"$(_TKDIR)\generic" -I"$(_TKDIR)\win" -I"$(_TKDIR)\xlib"
!message *** Link options '$(LINKERFLAGS)'
!endif
Vendored
+262 -256
View File
@@ -1,6 +1,6 @@
#! /bin/sh
# Guess values for system-dependent variables and create Makefiles.
# Generated by GNU Autoconf 2.69 for sqlite 3.39.0.
# Generated by GNU Autoconf 2.69 for sqlite 3.47.0.
#
#
# Copyright (C) 1992-1996, 1998-2012 Free Software Foundation, Inc.
@@ -726,8 +726,8 @@ MAKEFLAGS=
# Identity of this package.
PACKAGE_NAME='sqlite'
PACKAGE_TARNAME='sqlite'
PACKAGE_VERSION='3.39.0'
PACKAGE_STRING='sqlite 3.39.0'
PACKAGE_VERSION='3.47.0'
PACKAGE_STRING='sqlite 3.47.0'
PACKAGE_BUGREPORT=''
PACKAGE_URL=''
@@ -776,22 +776,11 @@ OPT_FEATURE_FLAGS
HAVE_ZLIB
USE_AMALGAMATION
TARGET_DEBUG
TARGET_HAVE_LINENOISE
TARGET_HAVE_EDITLINE
TARGET_HAVE_READLINE
TARGET_READLINE_INC
TARGET_READLINE_LIBS
HAVE_TCL
TCL_SHLIB_SUFFIX
TCL_STUB_LIB_SPEC
TCL_STUB_LIB_FLAG
TCL_STUB_LIB_FILE
TCL_LIB_SPEC
TCL_LIB_FLAG
TCL_LIB_FILE
TCL_INCLUDE_SPEC
TCL_SRC_DIR
TCL_BIN_DIR
TCL_VERSION
TARGET_EXEEXT
SQLITE_OS_WIN
SQLITE_OS_UNIX
@@ -800,10 +789,16 @@ TEMP_STORE
ALLOWRELEASE
SQLITE_THREADSAFE
BUILD_CC
HAVE_WASI_SDK
RELEASE
VERSION
program_prefix
TSTRNNR_OPTS
TCLLIBDIR
HAVE_TCL
TCL_STUB_LIB_SPEC
TCL_LIB_SPEC
TCL_INCLUDE_SPEC
TCLSH_CMD
INSTALL_DATA
INSTALL_SCRIPT
@@ -892,15 +887,19 @@ enable_fast_install
with_gnu_ld
enable_libtool_lock
enable_largefile
with_tclsh
with_tcl
enable_tcl
enable_test_status
with_wasi_sdk
enable_threadsafe
enable_releasemode
enable_tempstore
enable_tcl
with_tcl
enable_editline
enable_readline
with_readline_lib
with_readline_inc
with_linenoise
enable_debug
enable_amalgamation
enable_load_extension
@@ -1468,7 +1467,7 @@ if test "$ac_init_help" = "long"; then
# Omit some internal or obsolete options to make the list less imposing.
# This message is too long to be a string in the A/UX 3.1 sh.
cat <<_ACEOF
\`configure' configures sqlite 3.39.0 to adapt to many kinds of systems.
\`configure' configures sqlite 3.47.0 to adapt to many kinds of systems.
Usage: $0 [OPTION]... [VAR=VALUE]...
@@ -1533,7 +1532,7 @@ fi
if test -n "$ac_init_help"; then
case $ac_init_help in
short | recursive ) echo "Configuration of sqlite 3.39.0:";;
short | recursive ) echo "Configuration of sqlite 3.47.0:";;
esac
cat <<\_ACEOF
@@ -1547,11 +1546,13 @@ Optional Features:
optimize for fast installation [default=yes]
--disable-libtool-lock avoid locking (might break parallel builds)
--disable-largefile omit support for large files
--disable-tcl omit building accessory programs that require
TCL-dev
--enable-test-status Full-screen status of tests
--disable-threadsafe Disable mutexing
--enable-releasemode Support libtool link to release mode
--enable-tempstore Use an in-ram database for temporary tables
(never,no,yes,always)
--disable-tcl do not build TCL extension
--enable-editline enable BSD editline support
--disable-readline disable readline support
--enable-debug enable debugging & verbose explain
@@ -1579,10 +1580,13 @@ Optional Packages:
--with-pic try to use only PIC/non-PIC objects [default=use
both]
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
--with-tcl=DIR directory containing tcl configuration
(tclConfig.sh)
--with-tclsh=PATHNAME full pathname of a tclsh to use
--with-tcl=DIR directory containing (tclConfig.sh)
--with-wasi-sdk=DIR directory containing the WASI SDK. Triggers
cross-compile to WASM.
--with-readline-lib specify readline library
--with-readline-inc specify readline include paths
--with-linenoise=DIR source directory for linenoise library
Some influential environment variables:
CC C compiler command
@@ -1661,7 +1665,7 @@ fi
test -n "$ac_init_help" && exit $ac_status
if $ac_init_version; then
cat <<\_ACEOF
sqlite configure 3.39.0
sqlite configure 3.47.0
generated by GNU Autoconf 2.69
Copyright (C) 2012 Free Software Foundation, Inc.
@@ -2080,7 +2084,7 @@ cat >config.log <<_ACEOF
This file contains any messages produced by compilers while
running configure, to aid debugging if configure makes a mistake.
It was created by sqlite $as_me 3.39.0, which was
It was created by sqlite $as_me 3.47.0, which was
generated by GNU Autoconf 2.69. Invocation command line was
$ $0 $@
@@ -3938,13 +3942,13 @@ if ${lt_cv_nm_interface+:} false; then :
else
lt_cv_nm_interface="BSD nm"
echo "int some_variable = 0;" > conftest.$ac_ext
(eval echo "\"\$as_me:3941: $ac_compile\"" >&5)
(eval echo "\"\$as_me:3945: $ac_compile\"" >&5)
(eval "$ac_compile" 2>conftest.err)
cat conftest.err >&5
(eval echo "\"\$as_me:3944: $NM \\\"conftest.$ac_objext\\\"\"" >&5)
(eval echo "\"\$as_me:3948: $NM \\\"conftest.$ac_objext\\\"\"" >&5)
(eval "$NM \"conftest.$ac_objext\"" 2>conftest.err > conftest.out)
cat conftest.err >&5
(eval echo "\"\$as_me:3947: output\"" >&5)
(eval echo "\"\$as_me:3951: output\"" >&5)
cat conftest.out >&5
if $GREP 'External.*some_variable' conftest.out > /dev/null; then
lt_cv_nm_interface="MS dumpbin"
@@ -5150,7 +5154,7 @@ ia64-*-hpux*)
;;
*-*-irix6*)
# Find out which ABI we are using.
echo '#line 5153 "configure"' > conftest.$ac_ext
echo '#line 5157 "configure"' > conftest.$ac_ext
if { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$ac_compile\""; } >&5
(eval $ac_compile) 2>&5
ac_status=$?
@@ -6675,11 +6679,11 @@ else
-e 's:.*FLAGS}\{0,1\} :&$lt_compiler_flag :; t' \
-e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \
-e 's:$: $lt_compiler_flag:'`
(eval echo "\"\$as_me:6678: $lt_compile\"" >&5)
(eval echo "\"\$as_me:6682: $lt_compile\"" >&5)
(eval "$lt_compile" 2>conftest.err)
ac_status=$?
cat conftest.err >&5
echo "$as_me:6682: \$? = $ac_status" >&5
echo "$as_me:6686: \$? = $ac_status" >&5
if (exit $ac_status) && test -s "$ac_outfile"; then
# The compiler can only warn and ignore the option if not recognized
# So say no if there are warnings other than the usual output.
@@ -7014,11 +7018,11 @@ else
-e 's:.*FLAGS}\{0,1\} :&$lt_compiler_flag :; t' \
-e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \
-e 's:$: $lt_compiler_flag:'`
(eval echo "\"\$as_me:7017: $lt_compile\"" >&5)
(eval echo "\"\$as_me:7021: $lt_compile\"" >&5)
(eval "$lt_compile" 2>conftest.err)
ac_status=$?
cat conftest.err >&5
echo "$as_me:7021: \$? = $ac_status" >&5
echo "$as_me:7025: \$? = $ac_status" >&5
if (exit $ac_status) && test -s "$ac_outfile"; then
# The compiler can only warn and ignore the option if not recognized
# So say no if there are warnings other than the usual output.
@@ -7119,11 +7123,11 @@ else
-e 's:.*FLAGS}\{0,1\} :&$lt_compiler_flag :; t' \
-e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \
-e 's:$: $lt_compiler_flag:'`
(eval echo "\"\$as_me:7122: $lt_compile\"" >&5)
(eval echo "\"\$as_me:7126: $lt_compile\"" >&5)
(eval "$lt_compile" 2>out/conftest.err)
ac_status=$?
cat out/conftest.err >&5
echo "$as_me:7126: \$? = $ac_status" >&5
echo "$as_me:7130: \$? = $ac_status" >&5
if (exit $ac_status) && test -s out/conftest2.$ac_objext
then
# The compiler can only warn and ignore the option if not recognized
@@ -7174,11 +7178,11 @@ else
-e 's:.*FLAGS}\{0,1\} :&$lt_compiler_flag :; t' \
-e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \
-e 's:$: $lt_compiler_flag:'`
(eval echo "\"\$as_me:7177: $lt_compile\"" >&5)
(eval echo "\"\$as_me:7181: $lt_compile\"" >&5)
(eval "$lt_compile" 2>out/conftest.err)
ac_status=$?
cat out/conftest.err >&5
echo "$as_me:7181: \$? = $ac_status" >&5
echo "$as_me:7185: \$? = $ac_status" >&5
if (exit $ac_status) && test -s out/conftest2.$ac_objext
then
# The compiler can only warn and ignore the option if not recognized
@@ -9554,7 +9558,7 @@ else
lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2
lt_status=$lt_dlunknown
cat > conftest.$ac_ext <<_LT_EOF
#line 9557 "configure"
#line 9561 "configure"
#include "confdefs.h"
#if HAVE_DLFCN_H
@@ -9650,7 +9654,7 @@ else
lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2
lt_status=$lt_dlunknown
cat > conftest.$ac_ext <<_LT_EOF
#line 9653 "configure"
#line 9657 "configure"
#include "confdefs.h"
#if HAVE_DLFCN_H
@@ -10307,11 +10311,31 @@ done
USE_AMALGAMATION=1
#########
# See whether we can run specific tclsh versions known to work well;
# if not, then we fall back to plain tclsh.
# TODO: try other versions before falling back?
# Figure out all the name of a working tclsh and parameters needed to compile against Tcl.
# The --with-tcl= and/or --with-tclsh= configuration arguments might be useful for this.
#
for ac_prog in tclsh8.7 tclsh8.6 tclsh8.5 tclsh
# Check whether --with-tclsh was given.
if test "${with_tclsh+set}" = set; then :
withval=$with_tclsh;
fi
# Check whether --with-tcl was given.
if test "${with_tcl+set}" = set; then :
withval=$with_tcl;
fi
# Check whether --enable-tcl was given.
if test "${enable_tcl+set}" = set; then :
enableval=$enable_tcl; use_tcl=$enableval
else
use_tcl=yes
fi
original_use_tcl=${use_tcl}
if test x"${with_tclsh}" == x -a x"${with_tcl}" == x; then
for ac_prog in tclsh8.6 tclsh tclsh9.0
do
# Extract the first word of "$ac_prog", so it can be a program name with args.
set dummy $ac_prog; ac_word=$2
@@ -10354,10 +10378,99 @@ fi
done
test -n "$TCLSH_CMD" || TCLSH_CMD="none"
with_tclsh=${TCLSH_CMD}
fi
if test x"${with_tclsh}" != x -a x"${with_tclsh}" != xnone; then
TCLSH_CMD=${with_tclsh}
{ $as_echo "$as_me:${as_lineno-$LINENO}: result: using tclsh at \"$TCLSH_CMD\"" >&5
$as_echo "using tclsh at \"$TCLSH_CMD\"" >&6; }
if test x"${use_tcl}" = "xyes"; then
with_tcl=`${with_tclsh} <${srcdir}/tool/find_tclconfig.tcl`
if test x"${with_tcl}" != x; then
{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $TCLSH_CMD recommends the tclConfig.sh at ${with_tcl}" >&5
$as_echo "$TCLSH_CMD recommends the tclConfig.sh at ${with_tcl}" >&6; }
else
{ $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: $TCLSH_CMD is unable to recommend a tclConfig.sh" >&5
$as_echo "$as_me: WARNING: $TCLSH_CMD is unable to recommend a tclConfig.sh" >&2;}
use_tcl=no
fi
fi
fi
if test x"${use_tcl}" = "xyes"; then
if test x"${with_tcl}" != x; then
if test -r ${with_tcl}/tclConfig.sh; then
tclconfig="${with_tcl}/tclConfig.sh"
else
for i in tcl8.6 tcl9.0 lib; do
if test -r ${with_tcl}/$i/tclConfig.sh; then
tclconfig=${with_tcl}/$i/tclConfig.sh
break
fi
done
fi
if test ! -r "${tclconfig}"; then
as_fn_error $? "no tclConfig.sh file found under ${with_tcl}" "$LINENO" 5
fi
else
# If we have not yet found a tclConfig.sh file, look in $libdir whic is
# set automatically by autoconf or by the --prefix command-line option.
# See https://sqlite.org/forum/forumpost/e04e693439a22457
libdir=${prefix}/lib
if test -r ${libdir}/tclConfig.sh; then
tclconfig=${libdir}/tclConfig.sh
else
for i in tcl8.6 tcl9.0 lib; do
if test -r ${libdir}/$i/tclConfig.sh; then
tclconfig=${libdir}/$i/tclConfig.sh
break
fi
done
fi
if test ! -r "${tclconfig}"; then
as_fn_error $? "cannot find a usable tclConfig.sh file.
Use --with-tcl=DIR to specify a directory where tclConfig.sh can be found.
SQLite does not use TCL internally, but TCL is required to build SQLite
from canonical sources and TCL is required for testing." "$LINENO" 5
fi
fi
{ $as_echo "$as_me:${as_lineno-$LINENO}: result: loading TCL configuration from ${tclconfig}" >&5
$as_echo "loading TCL configuration from ${tclconfig}" >&6; }
. ${tclconfig}
# There are lots of other configuration variables that are provided by the
# tclConfig.sh file and that could be included here. But as of right now,
# TCL_LIB_SPEC is the only what that the Makefile uses.
HAVE_TCL=1
elif test x"${original_use_tcl}" = "xno"; then
{ $as_echo "$as_me:${as_lineno-$LINENO}: result: unable to run tests because of --disable-tcl" >&5
$as_echo "unable to run tests because of --disable-tcl" >&6; }
HAVE_TCL=0
else
{ $as_echo "$as_me:${as_lineno-$LINENO}: result: unable to run tests because no tclConfig.sh file could be located" >&5
$as_echo "unable to run tests because no tclConfig.sh file could be located" >&6; }
HAVE_TCL=0
fi
if test x"$TCLSH_CMD" == x; then
TCLSH_CMD=${TCL_EXEC_PREFIX}/bin/tclsh${TCL_VERSION}
if test ! -x ${TCLSH_CMD}; then
TCLSH_CMD_2=${TCL_EXEC_PREFIX}/bin/tclsh
if test ! -x ${TCLSH_CMD_2}; then
{ $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: cannot find a usable tclsh at either ${TCLSH_CMD} or ${TCLSH_CMD_2}" >&5
$as_echo "$as_me: WARNING: cannot find a usable tclsh at either ${TCLSH_CMD} or ${TCLSH_CMD_2}" >&2;}
TCLSH_CMD=none
else
TCLSH_CMD=${TCLSH_CMD_2}
fi
fi
fi
if test "$TCLSH_CMD" = "none"; then
# If we can't find a local tclsh, then building the amalgamation will fail.
# We act as though --disable-amalgamation has been used.
echo "Warning: can't find tclsh - defaulting to non-amalgamation build."
{ $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: Warning: can't find tclsh - defaulting to non-amalgamation build." >&5
$as_echo "$as_me: WARNING: Warning: can't find tclsh - defaulting to non-amalgamation build." >&2;}
USE_AMALGAMATION=0
TCLSH_CMD="tclsh"
fi
@@ -10365,7 +10478,6 @@ fi
if test "x${TCLLIBDIR+set}" != "xset" ; then
TCLLIBDIR='$(libdir)'
for i in `echo 'puts stdout $auto_path' | ${TCLSH_CMD}` ; do
if test -d $i ; then
TCLLIBDIR=$i
@@ -10375,6 +10487,23 @@ if test "x${TCLLIBDIR+set}" != "xset" ; then
TCLLIBDIR="${TCLLIBDIR}/sqlite3"
fi
#########
# Set up options for running tests.
#
# Check whether --enable-test-status was given.
if test "${enable_test_status+set}" = set; then :
enableval=$enable_test_status; use_vt100=$enableval
else
use_vt100=no
fi
if test $use_vt100 != no; then
TSTRNNR_OPTS=--status
else
TSTRNNR_OPTS=
fi
#########
# Set up an appropriate program prefix
@@ -10393,6 +10522,68 @@ RELEASE=`cat $srcdir/VERSION`
$as_echo "$as_me: Release set to $RELEASE" >&6;}
##########
# Handle --with-wasi-sdk=DIR
#
# This must be early because it changes the toolchain.
#
# Check whether --with-wasi-sdk was given.
if test "${with_wasi_sdk+set}" = set; then :
withval=$with_wasi_sdk; with_wasisdk=${withval}
fi
{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for WASI SDK directory" >&5
$as_echo_n "checking for WASI SDK directory... " >&6; }
if ${ac_cv_c_wasi_sdk+:} false; then :
$as_echo_n "(cached) " >&6
else
# First check to see if --with-tcl was specified.
if test x"${with_wasi_sdk}" != x ; then
if ! test -d "${with_wasi_sdk}" ; then
as_fn_error $? "${with_wasi_sdk} directory doesn't exist" "$LINENO" 5
fi
{ $as_echo "$as_me:${as_lineno-$LINENO}: result: ${with_wasi_sdk}: using wasi-sdk clang, disabling: tcl, CLI shell, DLL" >&5
$as_echo "${with_wasi_sdk}: using wasi-sdk clang, disabling: tcl, CLI shell, DLL" >&6; }
use_wasi_sdk=yes
else
use_wasi_sdk=no
fi
fi
if test "${use_wasi_sdk}" = "no" ; then
HAVE_WASI_SDK=""
{ $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
$as_echo "no" >&6; }
else
HAVE_WASI_SDK=1
# Changing --host and --target have no effect here except to possibly
# cause confusion. autoconf has finished processing them by this
# point.
#
# host_alias=wasm32-wasi
# target=wasm32-wasi
#
# Merely changing CC and LD to the wasi-sdk's is enough to get
# sqlite3.o building in WASM format.
CC="${with_wasi_sdk}/bin/clang"
LD="${with_wasi_sdk}/bin/wasm-ld"
RANLIB="${with_wasi_sdk}/bin/llvm-ranlib"
cross_compiling=yes
enable_threadsafe=no
use_tcl=no
enable_tcl=no
# libtool is apparently hard-coded to use gcc for linking DLLs, so
# we disable the DLL build...
enable_shared=no
{ $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5
$as_echo "yes" >&6; }
fi
#########
# Locate a compiler for the build machine. This compiler should
# generate command-line programs that run on the build machine.
@@ -10700,213 +10891,6 @@ fi
##########
# Figure out all the parameters needed to compile against Tcl.
#
# This code is derived from the SC_PATH_TCLCONFIG and SC_LOAD_TCLCONFIG
# macros in the in the tcl.m4 file of the standard TCL distribution.
# Those macros could not be used directly since we have to make some
# minor changes to accomodate systems that do not have TCL installed.
#
# Check whether --enable-tcl was given.
if test "${enable_tcl+set}" = set; then :
enableval=$enable_tcl; use_tcl=$enableval
else
use_tcl=yes
fi
if test "${use_tcl}" = "yes" ; then
# Check whether --with-tcl was given.
if test "${with_tcl+set}" = set; then :
withval=$with_tcl; with_tclconfig=${withval}
fi
{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for Tcl configuration" >&5
$as_echo_n "checking for Tcl configuration... " >&6; }
if ${ac_cv_c_tclconfig+:} false; then :
$as_echo_n "(cached) " >&6
else
# First check to see if --with-tcl was specified.
if test x"${with_tclconfig}" != x ; then
if test -f "${with_tclconfig}/tclConfig.sh" ; then
ac_cv_c_tclconfig=`(cd ${with_tclconfig}; pwd)`
else
as_fn_error $? "${with_tclconfig} directory doesn't contain tclConfig.sh" "$LINENO" 5
fi
fi
# Start autosearch by asking tclsh
if test x"${ac_cv_c_tclconfig}" = x ; then
if test x"$cross_compiling" = xno; then
for i in `echo 'puts stdout $auto_path' | ${TCLSH_CMD}`
do
if test -f "$i/tclConfig.sh" ; then
ac_cv_c_tclconfig="$i"
break
fi
done
fi
fi
# On ubuntu 14.10, $auto_path on tclsh is not quite correct.
# So try again after applying corrections.
if test x"${ac_cv_c_tclconfig}" = x ; then
if test x"$cross_compiling" = xno; then
for i in `echo 'puts stdout $auto_path' | ${TCLSH_CMD} | sed 's,/tcltk/tcl,/tcl,g'`
do
if test -f "$i/tclConfig.sh" ; then
ac_cv_c_tclconfig="$i"
break
fi
done
fi
fi
# Recent versions of Xcode on Macs hid the tclConfig.sh file
# in a strange place.
if test x"${ac_cv_c_tclconfig}" = x ; then
if test x"$cross_compiling" = xno; then
for i in /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX*.sdk/usr/lib
do
if test -f "$i/tclConfig.sh" ; then
ac_cv_c_tclconfig="$i"
break
fi
done
fi
fi
# then check for a private Tcl installation
if test x"${ac_cv_c_tclconfig}" = x ; then
for i in \
../tcl \
`ls -dr ../tcl[8-9].[0-9].[0-9]* 2>/dev/null` \
`ls -dr ../tcl[8-9].[0-9] 2>/dev/null` \
`ls -dr ../tcl[8-9].[0-9]* 2>/dev/null` \
../../tcl \
`ls -dr ../../tcl[8-9].[0-9].[0-9]* 2>/dev/null` \
`ls -dr ../../tcl[8-9].[0-9] 2>/dev/null` \
`ls -dr ../../tcl[8-9].[0-9]* 2>/dev/null` \
../../../tcl \
`ls -dr ../../../tcl[8-9].[0-9].[0-9]* 2>/dev/null` \
`ls -dr ../../../tcl[8-9].[0-9] 2>/dev/null` \
`ls -dr ../../../tcl[8-9].[0-9]* 2>/dev/null`
do
if test -f "$i/unix/tclConfig.sh" ; then
ac_cv_c_tclconfig=`(cd $i/unix; pwd)`
break
fi
done
fi
# check in a few common install locations
if test x"${ac_cv_c_tclconfig}" = x ; then
for i in \
`ls -d ${libdir} 2>/dev/null` \
`ls -d /usr/local/lib 2>/dev/null` \
`ls -d /usr/contrib/lib 2>/dev/null` \
`ls -d /usr/lib 2>/dev/null`
do
if test -f "$i/tclConfig.sh" ; then
ac_cv_c_tclconfig=`(cd $i; pwd)`
break
fi
done
fi
# check in a few other private locations
if test x"${ac_cv_c_tclconfig}" = x ; then
for i in \
${srcdir}/../tcl \
`ls -dr ${srcdir}/../tcl[8-9].[0-9].[0-9]* 2>/dev/null` \
`ls -dr ${srcdir}/../tcl[8-9].[0-9] 2>/dev/null` \
`ls -dr ${srcdir}/../tcl[8-9].[0-9]* 2>/dev/null`
do
if test -f "$i/unix/tclConfig.sh" ; then
ac_cv_c_tclconfig=`(cd $i/unix; pwd)`
break
fi
done
fi
fi
if test x"${ac_cv_c_tclconfig}" = x ; then
use_tcl=no
{ $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: Can't find Tcl configuration definitions" >&5
$as_echo "$as_me: WARNING: Can't find Tcl configuration definitions" >&2;}
{ $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: *** Without Tcl the regression tests cannot be executed ***" >&5
$as_echo "$as_me: WARNING: *** Without Tcl the regression tests cannot be executed ***" >&2;}
{ $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: *** Consider using --with-tcl=... to define location of Tcl ***" >&5
$as_echo "$as_me: WARNING: *** Consider using --with-tcl=... to define location of Tcl ***" >&2;}
else
TCL_BIN_DIR=${ac_cv_c_tclconfig}
{ $as_echo "$as_me:${as_lineno-$LINENO}: result: found $TCL_BIN_DIR/tclConfig.sh" >&5
$as_echo "found $TCL_BIN_DIR/tclConfig.sh" >&6; }
{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for existence of $TCL_BIN_DIR/tclConfig.sh" >&5
$as_echo_n "checking for existence of $TCL_BIN_DIR/tclConfig.sh... " >&6; }
if test -f "$TCL_BIN_DIR/tclConfig.sh" ; then
{ $as_echo "$as_me:${as_lineno-$LINENO}: result: loading" >&5
$as_echo "loading" >&6; }
. $TCL_BIN_DIR/tclConfig.sh
else
{ $as_echo "$as_me:${as_lineno-$LINENO}: result: file not found" >&5
$as_echo "file not found" >&6; }
fi
#
# If the TCL_BIN_DIR is the build directory (not the install directory),
# then set the common variable name to the value of the build variables.
# For example, the variable TCL_LIB_SPEC will be set to the value
# of TCL_BUILD_LIB_SPEC. An extension should make use of TCL_LIB_SPEC
# instead of TCL_BUILD_LIB_SPEC since it will work with both an
# installed and uninstalled version of Tcl.
#
if test -f $TCL_BIN_DIR/Makefile ; then
TCL_LIB_SPEC=${TCL_BUILD_LIB_SPEC}
TCL_STUB_LIB_SPEC=${TCL_BUILD_STUB_LIB_SPEC}
TCL_STUB_LIB_PATH=${TCL_BUILD_STUB_LIB_PATH}
fi
#
# eval is required to do the TCL_DBGX substitution
#
eval "TCL_LIB_FILE=\"${TCL_LIB_FILE}\""
eval "TCL_LIB_FLAG=\"${TCL_LIB_FLAG}\""
eval "TCL_LIB_SPEC=\"${TCL_LIB_SPEC}\""
eval "TCL_STUB_LIB_FILE=\"${TCL_STUB_LIB_FILE}\""
eval "TCL_STUB_LIB_FLAG=\"${TCL_STUB_LIB_FLAG}\""
eval "TCL_STUB_LIB_SPEC=\"${TCL_STUB_LIB_SPEC}\""
fi
fi
if test "${use_tcl}" = "no" ; then
HAVE_TCL=""
else
HAVE_TCL=1
fi
##########
# Figure out what C libraries are required to compile programs
# that use "readline()" library.
@@ -11179,6 +11163,27 @@ fi
fi
fi
# Check whether --with-linenoise was given.
if test "${with_linenoise+set}" = set; then :
withval=$with_linenoise; with_linenoise=$withval
else
with_linenoise="no"
fi
if test "x$with_linenoise" != "xno"; then
TARGET_HAVE_READLINE=0
TARGET_HAVE_EDITLINE=0
TARGET_HAVE_LINENOISE=1
TARGET_READLINE_INC="-I${with_linenoise}"
TARGET_READLINE_LIBS="${with_linenoise}/linenoise.c"
echo "using linenoise source code at ${with_linenoise}"
else
TARGET_HAVE_LINENOISE=0
echo "not using linenoise"
fi
@@ -11255,7 +11260,7 @@ fi
{ $as_echo "$as_me:${as_lineno-$LINENO}: checking build type" >&5
$as_echo_n "checking build type... " >&6; }
if test "${enable_debug}" = "yes" ; then
TARGET_DEBUG="-DSQLITE_DEBUG=1 -DSQLITE_ENABLE_SELECTTRACE -DSQLITE_ENABLE_WHERETRACE -O0"
TARGET_DEBUG="-DSQLITE_DEBUG=1 -DSQLITE_ENABLE_SELECTTRACE -DSQLITE_ENABLE_WHERETRACE -O0 -Wall"
{ $as_echo "$as_me:${as_lineno-$LINENO}: result: debug" >&5
$as_echo "debug" >&6; }
else
@@ -11267,6 +11272,7 @@ fi
#########
# See whether we should use the amalgamation to build
# Check whether --enable-amalgamation was given.
if test "${enable_amalgamation+set}" = set; then :
enableval=$enable_amalgamation;
@@ -11875,7 +11881,7 @@ fi
#########
# Output the config header
ac_config_headers="$ac_config_headers config.h"
ac_config_headers="$ac_config_headers sqlite_cfg.h"
#########
@@ -12390,7 +12396,7 @@ cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1
# report actual input values of CONFIG_FILES etc. instead of their
# values after options handling.
ac_log="
This file was extended by sqlite $as_me 3.39.0, which was
This file was extended by sqlite $as_me 3.47.0, which was
generated by GNU Autoconf 2.69. Invocation command line was
CONFIG_FILES = $CONFIG_FILES
@@ -12456,7 +12462,7 @@ _ACEOF
cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1
ac_cs_config="`$as_echo "$ac_configure_args" | sed 's/^ //; s/[\\""\`\$]/\\\\&/g'`"
ac_cs_version="\\
sqlite config.status 3.39.0
sqlite config.status 3.47.0
configured by $0, generated by GNU Autoconf 2.69,
with options \\"\$ac_cs_config\\"
@@ -12838,7 +12844,7 @@ for ac_config_target in $ac_config_targets
do
case $ac_config_target in
"libtool") CONFIG_COMMANDS="$CONFIG_COMMANDS libtool" ;;
"config.h") CONFIG_HEADERS="$CONFIG_HEADERS config.h" ;;
"sqlite_cfg.h") CONFIG_HEADERS="$CONFIG_HEADERS sqlite_cfg.h" ;;
"Makefile") CONFIG_FILES="$CONFIG_FILES Makefile" ;;
"sqlite3.pc") CONFIG_FILES="$CONFIG_FILES sqlite3.pc" ;;
+211 -221
View File
@@ -70,11 +70,11 @@
# target platform. "" for Unix and ".exe" for windows.
#
# This configure.in file is easy to reuse on other projects. Just
# change the argument to AC_INIT(). And disable any features that
# change the argument to AC_INIT. And disable any features that
# you don't need (for example BLT) by erasing or commenting out
# the corresponding code.
#
AC_INIT(sqlite, m4_esyscmd([cat VERSION | tr -d '\n']))
AC_INIT([sqlite],m4_esyscmd(cat VERSION | tr -d '\n'))
dnl Make sure the local VERSION file matches this configure script
sqlite_version_sanity_check=`cat $srcdir/VERSION | tr -d '\n'`
@@ -88,7 +88,7 @@ fi
#########
# Programs needed
#
AC_PROG_LIBTOOL
LT_INIT
AC_PROG_INSTALL
#########
@@ -116,15 +116,101 @@ AC_CHECK_FUNCS([fdatasync gmtime_r isnan localtime_r localtime_s malloc_usable_s
USE_AMALGAMATION=1
#########
# See whether we can run specific tclsh versions known to work well;
# if not, then we fall back to plain tclsh.
# TODO: try other versions before falling back?
#
AC_CHECK_PROGS(TCLSH_CMD, [tclsh8.7 tclsh8.6 tclsh8.5 tclsh], none)
# Figure out all the name of a working tclsh and parameters needed to compile against Tcl.
# The --with-tcl= and/or --with-tclsh= configuration arguments might be useful for this.
#
AC_ARG_WITH(tclsh, AS_HELP_STRING([--with-tclsh=PATHNAME],[full pathname of a tclsh to use]))
AC_ARG_WITH(tcl, AS_HELP_STRING([--with-tcl=DIR],[directory containing (tclConfig.sh)]))
AC_ARG_ENABLE(tcl, AS_HELP_STRING([--disable-tcl],[omit building accessory programs that require TCL-dev]),
[use_tcl=$enableval],[use_tcl=yes])
original_use_tcl=${use_tcl}
if test x"${with_tclsh}" == x -a x"${with_tcl}" == x; then
AC_CHECK_PROGS(TCLSH_CMD, [tclsh8.6 tclsh tclsh9.0],none)
with_tclsh=${TCLSH_CMD}
fi
if test x"${with_tclsh}" != x -a x"${with_tclsh}" != xnone; then
TCLSH_CMD=${with_tclsh}
AC_MSG_RESULT([using tclsh at "$TCLSH_CMD"])
if test x"${use_tcl}" = "xyes"; then
with_tcl=`${with_tclsh} <${srcdir}/tool/find_tclconfig.tcl`
if test x"${with_tcl}" != x; then
AC_MSG_RESULT([$TCLSH_CMD recommends the tclConfig.sh at ${with_tcl}])
else
AC_MSG_WARN([$TCLSH_CMD is unable to recommend a tclConfig.sh])
use_tcl=no
fi
fi
fi
if test x"${use_tcl}" = "xyes"; then
if test x"${with_tcl}" != x; then
if test -r ${with_tcl}/tclConfig.sh; then
tclconfig="${with_tcl}/tclConfig.sh"
else
for i in tcl8.6 tcl9.0 lib; do
if test -r ${with_tcl}/$i/tclConfig.sh; then
tclconfig=${with_tcl}/$i/tclConfig.sh
break
fi
done
fi
if test ! -r "${tclconfig}"; then
AC_MSG_ERROR([no tclConfig.sh file found under ${with_tcl}])
fi
else
# If we have not yet found a tclConfig.sh file, look in $libdir whic is
# set automatically by autoconf or by the --prefix command-line option.
# See https://sqlite.org/forum/forumpost/e04e693439a22457
libdir=${prefix}/lib
if test -r ${libdir}/tclConfig.sh; then
tclconfig=${libdir}/tclConfig.sh
else
for i in tcl8.6 tcl9.0 lib; do
if test -r ${libdir}/$i/tclConfig.sh; then
tclconfig=${libdir}/$i/tclConfig.sh
break
fi
done
fi
if test ! -r "${tclconfig}"; then
AC_MSG_ERROR([cannot find a usable tclConfig.sh file.
Use --with-tcl=DIR to specify a directory where tclConfig.sh can be found.
SQLite does not use TCL internally, but TCL is required to build SQLite
from canonical sources and TCL is required for testing.])
fi
fi
AC_MSG_RESULT([loading TCL configuration from ${tclconfig}])
. ${tclconfig}
AC_SUBST(TCL_INCLUDE_SPEC)
AC_SUBST(TCL_LIB_SPEC)
AC_SUBST(TCL_STUB_LIB_SPEC)
# There are lots of other configuration variables that are provided by the
# tclConfig.sh file and that could be included here. But as of right now,
# TCL_LIB_SPEC is the only what that the Makefile uses.
HAVE_TCL=1
elif test x"${original_use_tcl}" = "xno"; then
AC_MSG_RESULT([unable to run tests because of --disable-tcl])
HAVE_TCL=0
else
AC_MSG_RESULT([unable to run tests because no tclConfig.sh file could be located])
HAVE_TCL=0
fi
AC_SUBST(HAVE_TCL)
if test x"$TCLSH_CMD" == x; then
TCLSH_CMD=${TCL_EXEC_PREFIX}/bin/tclsh${TCL_VERSION}
if test ! -x ${TCLSH_CMD}; then
TCLSH_CMD_2=${TCL_EXEC_PREFIX}/bin/tclsh
if test ! -x ${TCLSH_CMD_2}; then
AC_MSG_WARN([cannot find a usable tclsh at either ${TCLSH_CMD} or ${TCLSH_CMD_2}])
TCLSH_CMD=none
else
TCLSH_CMD=${TCLSH_CMD_2}
fi
fi
fi
if test "$TCLSH_CMD" = "none"; then
# If we can't find a local tclsh, then building the amalgamation will fail.
# We act as though --disable-amalgamation has been used.
echo "Warning: can't find tclsh - defaulting to non-amalgamation build."
AC_MSG_WARN([Warning: can't find tclsh - defaulting to non-amalgamation build.])
USE_AMALGAMATION=0
TCLSH_CMD="tclsh"
fi
@@ -132,7 +218,6 @@ AC_SUBST(TCLSH_CMD)
AC_ARG_VAR([TCLLIBDIR], [Where to install tcl plugin])
if test "x${TCLLIBDIR+set}" != "xset" ; then
TCLLIBDIR='$(libdir)'
for i in `echo 'puts stdout $auto_path' | ${TCLSH_CMD}` ; do
if test -d $i ; then
TCLLIBDIR=$i
@@ -142,6 +227,18 @@ if test "x${TCLLIBDIR+set}" != "xset" ; then
TCLLIBDIR="${TCLLIBDIR}/sqlite3"
fi
#########
# Set up options for running tests.
#
AC_ARG_ENABLE(test-status, AS_HELP_STRING([--enable-test-status],[Full-screen status of tests]),
[use_vt100=$enableval],[use_vt100=no])
if test $use_vt100 != no; then
TSTRNNR_OPTS=--status
else
TSTRNNR_OPTS=
fi
AC_SUBST(TSTRNNR_OPTS)
#########
# Set up an appropriate program prefix
@@ -158,6 +255,56 @@ RELEASE=`cat $srcdir/VERSION`
AC_MSG_NOTICE(Release set to $RELEASE)
AC_SUBST(RELEASE)
##########
# Handle --with-wasi-sdk=DIR
#
# This must be early because it changes the toolchain.
#
AC_ARG_WITH(wasi-sdk,
AS_HELP_STRING([--with-wasi-sdk=DIR],
[directory containing the WASI SDK. Triggers cross-compile to WASM.]), with_wasisdk=${withval})
AC_MSG_CHECKING([for WASI SDK directory])
AC_CACHE_VAL(ac_cv_c_wasi_sdk,[
# First check to see if --with-tcl was specified.
if test x"${with_wasi_sdk}" != x ; then
if ! test -d "${with_wasi_sdk}" ; then
AC_MSG_ERROR([${with_wasi_sdk} directory doesn't exist])
fi
AC_MSG_RESULT([${with_wasi_sdk}: using wasi-sdk clang, disabling: tcl, CLI shell, DLL])
use_wasi_sdk=yes
else
use_wasi_sdk=no
fi
])
if test "${use_wasi_sdk}" = "no" ; then
HAVE_WASI_SDK=""
AC_MSG_RESULT([no])
else
HAVE_WASI_SDK=1
# Changing --host and --target have no effect here except to possibly
# cause confusion. autoconf has finished processing them by this
# point.
#
# host_alias=wasm32-wasi
# target=wasm32-wasi
#
# Merely changing CC and LD to the wasi-sdk's is enough to get
# sqlite3.o building in WASM format.
CC="${with_wasi_sdk}/bin/clang"
LD="${with_wasi_sdk}/bin/wasm-ld"
RANLIB="${with_wasi_sdk}/bin/llvm-ranlib"
cross_compiling=yes
enable_threadsafe=no
use_tcl=no
enable_tcl=no
# libtool is apparently hard-coded to use gcc for linking DLLs, so
# we disable the DLL build...
enable_shared=no
AC_MSG_RESULT([yes])
fi
AC_SUBST(HAVE_WASI_SDK)
#########
# Locate a compiler for the build machine. This compiler should
# generate command-line programs that run on the build machine.
@@ -179,7 +326,7 @@ AC_SUBST(BUILD_CC)
# Do we want to support multithreaded use of sqlite
#
AC_ARG_ENABLE(threadsafe,
AC_HELP_STRING([--disable-threadsafe],[Disable mutexing]))
AS_HELP_STRING([--disable-threadsafe],[Disable mutexing]))
AC_MSG_CHECKING([whether to support threadsafe operation])
if test "$enable_threadsafe" = "no"; then
SQLITE_THREADSAFE=0
@@ -199,7 +346,7 @@ fi
# Do we want to support release
#
AC_ARG_ENABLE(releasemode,
AC_HELP_STRING([--enable-releasemode],[Support libtool link to release mode]),,enable_releasemode=no)
AS_HELP_STRING([--enable-releasemode],[Support libtool link to release mode]),,enable_releasemode=no)
AC_MSG_CHECKING([whether to support shared library linked as release mode or not])
if test "$enable_releasemode" = "no"; then
ALLOWRELEASE=""
@@ -214,7 +361,7 @@ AC_SUBST(ALLOWRELEASE)
# Do we want temporary databases in memory
#
AC_ARG_ENABLE(tempstore,
AC_HELP_STRING([--enable-tempstore],[Use an in-ram database for temporary tables (never,no,yes,always)]),,enable_tempstore=no)
AS_HELP_STRING([--enable-tempstore],[Use an in-ram database for temporary tables (never,no,yes,always)]),,enable_tempstore=no)
AC_MSG_CHECKING([whether to use an in-ram database for temporary tables])
case "$enable_tempstore" in
never )
@@ -254,7 +401,15 @@ else
AC_MSG_RESULT(unknown)
fi
if test "$CYGWIN" != "yes"; then
AC_CYGWIN
m4_warn([obsolete],
[AC_CYGWIN is obsolete: use AC_CANONICAL_HOST and check if $host_os
matches *cygwin*])dnl
AC_CANONICAL_HOST
case $host_os in
*cygwin* ) CYGWIN=yes;;
* ) CYGWIN=no;;
esac
fi
if test "$CYGWIN" = "yes"; then
BUILD_EXEEXT=.exe
@@ -281,190 +436,6 @@ AC_SUBST(SQLITE_OS_UNIX)
AC_SUBST(SQLITE_OS_WIN)
AC_SUBST(TARGET_EXEEXT)
##########
# Figure out all the parameters needed to compile against Tcl.
#
# This code is derived from the SC_PATH_TCLCONFIG and SC_LOAD_TCLCONFIG
# macros in the in the tcl.m4 file of the standard TCL distribution.
# Those macros could not be used directly since we have to make some
# minor changes to accomodate systems that do not have TCL installed.
#
AC_ARG_ENABLE(tcl, AC_HELP_STRING([--disable-tcl],[do not build TCL extension]),
[use_tcl=$enableval],[use_tcl=yes])
if test "${use_tcl}" = "yes" ; then
AC_ARG_WITH(tcl, AC_HELP_STRING([--with-tcl=DIR],[directory containing tcl configuration (tclConfig.sh)]), with_tclconfig=${withval})
AC_MSG_CHECKING([for Tcl configuration])
AC_CACHE_VAL(ac_cv_c_tclconfig,[
# First check to see if --with-tcl was specified.
if test x"${with_tclconfig}" != x ; then
if test -f "${with_tclconfig}/tclConfig.sh" ; then
ac_cv_c_tclconfig=`(cd ${with_tclconfig}; pwd)`
else
AC_MSG_ERROR([${with_tclconfig} directory doesn't contain tclConfig.sh])
fi
fi
# Start autosearch by asking tclsh
if test x"${ac_cv_c_tclconfig}" = x ; then
if test x"$cross_compiling" = xno; then
for i in `echo 'puts stdout $auto_path' | ${TCLSH_CMD}`
do
if test -f "$i/tclConfig.sh" ; then
ac_cv_c_tclconfig="$i"
break
fi
done
fi
fi
# On ubuntu 14.10, $auto_path on tclsh is not quite correct.
# So try again after applying corrections.
if test x"${ac_cv_c_tclconfig}" = x ; then
if test x"$cross_compiling" = xno; then
for i in `echo 'puts stdout $auto_path' | ${TCLSH_CMD} | sed 's,/tcltk/tcl,/tcl,g'`
do
if test -f "$i/tclConfig.sh" ; then
ac_cv_c_tclconfig="$i"
break
fi
done
fi
fi
# Recent versions of Xcode on Macs hid the tclConfig.sh file
# in a strange place.
if test x"${ac_cv_c_tclconfig}" = x ; then
if test x"$cross_compiling" = xno; then
for i in /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX*.sdk/usr/lib
do
if test -f "$i/tclConfig.sh" ; then
ac_cv_c_tclconfig="$i"
break
fi
done
fi
fi
# then check for a private Tcl installation
if test x"${ac_cv_c_tclconfig}" = x ; then
for i in \
../tcl \
`ls -dr ../tcl[[8-9]].[[0-9]].[[0-9]]* 2>/dev/null` \
`ls -dr ../tcl[[8-9]].[[0-9]] 2>/dev/null` \
`ls -dr ../tcl[[8-9]].[[0-9]]* 2>/dev/null` \
../../tcl \
`ls -dr ../../tcl[[8-9]].[[0-9]].[[0-9]]* 2>/dev/null` \
`ls -dr ../../tcl[[8-9]].[[0-9]] 2>/dev/null` \
`ls -dr ../../tcl[[8-9]].[[0-9]]* 2>/dev/null` \
../../../tcl \
`ls -dr ../../../tcl[[8-9]].[[0-9]].[[0-9]]* 2>/dev/null` \
`ls -dr ../../../tcl[[8-9]].[[0-9]] 2>/dev/null` \
`ls -dr ../../../tcl[[8-9]].[[0-9]]* 2>/dev/null`
do
if test -f "$i/unix/tclConfig.sh" ; then
ac_cv_c_tclconfig=`(cd $i/unix; pwd)`
break
fi
done
fi
# check in a few common install locations
if test x"${ac_cv_c_tclconfig}" = x ; then
for i in \
`ls -d ${libdir} 2>/dev/null` \
`ls -d /usr/local/lib 2>/dev/null` \
`ls -d /usr/contrib/lib 2>/dev/null` \
`ls -d /usr/lib 2>/dev/null`
do
if test -f "$i/tclConfig.sh" ; then
ac_cv_c_tclconfig=`(cd $i; pwd)`
break
fi
done
fi
# check in a few other private locations
if test x"${ac_cv_c_tclconfig}" = x ; then
for i in \
${srcdir}/../tcl \
`ls -dr ${srcdir}/../tcl[[8-9]].[[0-9]].[[0-9]]* 2>/dev/null` \
`ls -dr ${srcdir}/../tcl[[8-9]].[[0-9]] 2>/dev/null` \
`ls -dr ${srcdir}/../tcl[[8-9]].[[0-9]]* 2>/dev/null`
do
if test -f "$i/unix/tclConfig.sh" ; then
ac_cv_c_tclconfig=`(cd $i/unix; pwd)`
break
fi
done
fi
])
if test x"${ac_cv_c_tclconfig}" = x ; then
use_tcl=no
AC_MSG_WARN(Can't find Tcl configuration definitions)
AC_MSG_WARN(*** Without Tcl the regression tests cannot be executed ***)
AC_MSG_WARN(*** Consider using --with-tcl=... to define location of Tcl ***)
else
TCL_BIN_DIR=${ac_cv_c_tclconfig}
AC_MSG_RESULT(found $TCL_BIN_DIR/tclConfig.sh)
AC_MSG_CHECKING([for existence of $TCL_BIN_DIR/tclConfig.sh])
if test -f "$TCL_BIN_DIR/tclConfig.sh" ; then
AC_MSG_RESULT([loading])
. $TCL_BIN_DIR/tclConfig.sh
else
AC_MSG_RESULT([file not found])
fi
#
# If the TCL_BIN_DIR is the build directory (not the install directory),
# then set the common variable name to the value of the build variables.
# For example, the variable TCL_LIB_SPEC will be set to the value
# of TCL_BUILD_LIB_SPEC. An extension should make use of TCL_LIB_SPEC
# instead of TCL_BUILD_LIB_SPEC since it will work with both an
# installed and uninstalled version of Tcl.
#
if test -f $TCL_BIN_DIR/Makefile ; then
TCL_LIB_SPEC=${TCL_BUILD_LIB_SPEC}
TCL_STUB_LIB_SPEC=${TCL_BUILD_STUB_LIB_SPEC}
TCL_STUB_LIB_PATH=${TCL_BUILD_STUB_LIB_PATH}
fi
#
# eval is required to do the TCL_DBGX substitution
#
eval "TCL_LIB_FILE=\"${TCL_LIB_FILE}\""
eval "TCL_LIB_FLAG=\"${TCL_LIB_FLAG}\""
eval "TCL_LIB_SPEC=\"${TCL_LIB_SPEC}\""
eval "TCL_STUB_LIB_FILE=\"${TCL_STUB_LIB_FILE}\""
eval "TCL_STUB_LIB_FLAG=\"${TCL_STUB_LIB_FLAG}\""
eval "TCL_STUB_LIB_SPEC=\"${TCL_STUB_LIB_SPEC}\""
AC_SUBST(TCL_VERSION)
AC_SUBST(TCL_BIN_DIR)
AC_SUBST(TCL_SRC_DIR)
AC_SUBST(TCL_INCLUDE_SPEC)
AC_SUBST(TCL_LIB_FILE)
AC_SUBST(TCL_LIB_FLAG)
AC_SUBST(TCL_LIB_SPEC)
AC_SUBST(TCL_STUB_LIB_FILE)
AC_SUBST(TCL_STUB_LIB_FLAG)
AC_SUBST(TCL_STUB_LIB_SPEC)
AC_SUBST(TCL_SHLIB_SUFFIX)
fi
fi
if test "${use_tcl}" = "no" ; then
HAVE_TCL=""
else
HAVE_TCL=1
fi
AC_SUBST(HAVE_TCL)
##########
# Figure out what C libraries are required to compile programs
# that use "readline()" library.
@@ -474,11 +445,11 @@ TARGET_READLINE_INC=""
TARGET_HAVE_READLINE=0
TARGET_HAVE_EDITLINE=0
AC_ARG_ENABLE([editline],
[AC_HELP_STRING([--enable-editline],[enable BSD editline support])],
[AS_HELP_STRING([--enable-editline],[enable BSD editline support])],
[with_editline=$enableval],
[with_editline=auto])
AC_ARG_ENABLE([readline],
[AC_HELP_STRING([--disable-readline],[disable readline support])],
[AS_HELP_STRING([--disable-readline],[disable readline support])],
[with_readline=$enableval],
[with_readline=auto])
@@ -494,7 +465,7 @@ if test x"$with_readline" != xno; then
found="yes"
AC_ARG_WITH([readline-lib],
[AC_HELP_STRING([--with-readline-lib],[specify readline library])],
[AS_HELP_STRING([--with-readline-lib],[specify readline library])],
[with_readline_lib=$withval],
[with_readline_lib="auto"])
if test "x$with_readline_lib" = xauto; then
@@ -509,7 +480,7 @@ if test x"$with_readline" != xno; then
fi
AC_ARG_WITH([readline-inc],
[AC_HELP_STRING([--with-readline-inc],[specify readline include paths])],
[AS_HELP_STRING([--with-readline-inc],[specify readline include paths])],
[with_readline_inc=$withval],
[with_readline_inc="auto"])
if test "x$with_readline_inc" = xauto; then
@@ -540,11 +511,28 @@ if test x"$with_readline" != xno; then
TARGET_HAVE_READLINE=1
fi
fi
AC_ARG_WITH([linenoise],
[AS_HELP_STRING([--with-linenoise=DIR],[source directory for linenoise library])],
[with_linenoise=$withval],
[with_linenoise="no"])
if test "x$with_linenoise" != "xno"; then
TARGET_HAVE_READLINE=0
TARGET_HAVE_EDITLINE=0
TARGET_HAVE_LINENOISE=1
TARGET_READLINE_INC="-I${with_linenoise}"
TARGET_READLINE_LIBS="${with_linenoise}/linenoise.c"
echo "using linenoise source code at ${with_linenoise}"
else
TARGET_HAVE_LINENOISE=0
echo "not using linenoise"
fi
AC_SUBST(TARGET_READLINE_LIBS)
AC_SUBST(TARGET_READLINE_INC)
AC_SUBST(TARGET_HAVE_READLINE)
AC_SUBST(TARGET_HAVE_EDITLINE)
AC_SUBST(TARGET_HAVE_LINENOISE)
##########
# Figure out what C libraries are required to compile programs
@@ -554,10 +542,10 @@ AC_SEARCH_LIBS(fdatasync, [rt])
#########
# check for debug enabled
AC_ARG_ENABLE(debug, AC_HELP_STRING([--enable-debug],[enable debugging & verbose explain]))
AC_ARG_ENABLE(debug, AS_HELP_STRING([--enable-debug],[enable debugging & verbose explain]))
AC_MSG_CHECKING([build type])
if test "${enable_debug}" = "yes" ; then
TARGET_DEBUG="-DSQLITE_DEBUG=1 -DSQLITE_ENABLE_SELECTTRACE -DSQLITE_ENABLE_WHERETRACE -O0"
TARGET_DEBUG="-DSQLITE_DEBUG=1 -DSQLITE_ENABLE_SELECTTRACE -DSQLITE_ENABLE_WHERETRACE -O0 -Wall"
AC_MSG_RESULT([debug])
else
TARGET_DEBUG="-DNDEBUG"
@@ -567,7 +555,8 @@ AC_SUBST(TARGET_DEBUG)
#########
# See whether we should use the amalgamation to build
AC_ARG_ENABLE(amalgamation, AC_HELP_STRING([--disable-amalgamation],
AC_ARG_ENABLE(amalgamation, AS_HELP_STRING([--disable-amalgamation],
[Disable the amalgamation and instead build all files separately]))
if test "${enable_amalgamation}" = "no" ; then
USE_AMALGAMATION=0
@@ -582,7 +571,7 @@ AC_SUBST(HAVE_ZLIB)
#########
# See whether we should allow loadable extensions
AC_ARG_ENABLE(load-extension, AC_HELP_STRING([--disable-load-extension],
AC_ARG_ENABLE(load-extension, AS_HELP_STRING([--disable-load-extension],
[Disable loading of external extensions]),,[enable_load_extension=yes])
if test "${enable_load_extension}" = "yes" ; then
OPT_FEATURE_FLAGS=""
@@ -595,7 +584,7 @@ fi
# Do we want to support math functions
#
AC_ARG_ENABLE(math,
AC_HELP_STRING([--disable-math],[Disable math functions]))
AS_HELP_STRING([--disable-math],[Disable math functions]))
AC_MSG_CHECKING([whether to support math functions])
if test "$enable_math" = "no"; then
AC_MSG_RESULT([no])
@@ -609,7 +598,7 @@ fi
# Do we want to support JSON functions
#
AC_ARG_ENABLE(json,
AC_HELP_STRING([--disable-json],[Disable JSON functions]))
AS_HELP_STRING([--disable-json],[Disable JSON functions]))
AC_MSG_CHECKING([whether to support JSON functions])
if test "$enable_json" = "no"; then
AC_MSG_RESULT([no])
@@ -621,14 +610,14 @@ fi
########
# The --enable-all argument is short-hand to enable
# multiple extensions.
AC_ARG_ENABLE(all, AC_HELP_STRING([--enable-all],
AC_ARG_ENABLE(all, AS_HELP_STRING([--enable-all],
[Enable FTS4, FTS5, Geopoly, RTree, Sessions]))
##########
# Do we want to support memsys3 and/or memsys5
#
AC_ARG_ENABLE(memsys5,
AC_HELP_STRING([--enable-memsys5],[Enable MEMSYS5]))
AS_HELP_STRING([--enable-memsys5],[Enable MEMSYS5]))
AC_MSG_CHECKING([whether to support MEMSYS5])
if test "${enable_memsys5}" = "yes"; then
OPT_FEATURE_FLAGS="${OPT_FEATURE_FLAGS} -DSQLITE_ENABLE_MEMSYS5"
@@ -637,7 +626,7 @@ else
AC_MSG_RESULT([no])
fi
AC_ARG_ENABLE(memsys3,
AC_HELP_STRING([--enable-memsys3],[Enable MEMSYS3]))
AS_HELP_STRING([--enable-memsys3],[Enable MEMSYS3]))
AC_MSG_CHECKING([whether to support MEMSYS3])
if test "${enable_memsys3}" = "yes" -a "${enable_memsys5}" = "no"; then
OPT_FEATURE_FLAGS="${OPT_FEATURE_FLAGS} -DSQLITE_ENABLE_MEMSYS3"
@@ -648,7 +637,7 @@ fi
#########
# See whether we should enable Full Text Search extensions
AC_ARG_ENABLE(fts3, AC_HELP_STRING([--enable-fts3],
AC_ARG_ENABLE(fts3, AS_HELP_STRING([--enable-fts3],
[Enable the FTS3 extension]))
AC_MSG_CHECKING([whether to support FTS3])
if test "${enable_fts3}" = "yes" ; then
@@ -657,7 +646,7 @@ if test "${enable_fts3}" = "yes" ; then
else
AC_MSG_RESULT([no])
fi
AC_ARG_ENABLE(fts4, AC_HELP_STRING([--enable-fts4],
AC_ARG_ENABLE(fts4, AS_HELP_STRING([--enable-fts4],
[Enable the FTS4 extension]))
AC_MSG_CHECKING([whether to support FTS4])
if test "${enable_fts4}" = "yes" -o "${enable_all}" = "yes" ; then
@@ -667,7 +656,7 @@ if test "${enable_fts4}" = "yes" -o "${enable_all}" = "yes" ; then
else
AC_MSG_RESULT([no])
fi
AC_ARG_ENABLE(fts5, AC_HELP_STRING([--enable-fts5],
AC_ARG_ENABLE(fts5, AS_HELP_STRING([--enable-fts5],
[Enable the FTS5 extension]))
AC_MSG_CHECKING([whether to support FTS5])
if test "${enable_fts5}" = "yes" -o "${enable_all}" = "yes" ; then
@@ -681,7 +670,7 @@ fi
#########
# See whether we should enable the LIMIT clause on UPDATE and DELETE
# statements.
AC_ARG_ENABLE(update-limit, AC_HELP_STRING([--enable-update-limit],
AC_ARG_ENABLE(update-limit, AS_HELP_STRING([--enable-update-limit],
[Enable the UPDATE/DELETE LIMIT clause]))
AC_MSG_CHECKING([whether to support LIMIT on UPDATE and DELETE statements])
if test "${enable_update_limit}" = "yes" ; then
@@ -693,7 +682,7 @@ fi
#########
# See whether we should enable GEOPOLY
AC_ARG_ENABLE(geopoly, AC_HELP_STRING([--enable-geopoly],
AC_ARG_ENABLE(geopoly, AS_HELP_STRING([--enable-geopoly],
[Enable the GEOPOLY extension]),
[enable_geopoly=yes],[enable_geopoly=no])
AC_MSG_CHECKING([whether to support GEOPOLY])
@@ -707,7 +696,7 @@ fi
#########
# See whether we should enable RTREE
AC_ARG_ENABLE(rtree, AC_HELP_STRING([--enable-rtree],
AC_ARG_ENABLE(rtree, AS_HELP_STRING([--enable-rtree],
[Enable the RTREE extension]))
AC_MSG_CHECKING([whether to support RTREE])
if test "${enable_rtree}" = "yes" ; then
@@ -719,7 +708,7 @@ fi
#########
# See whether we should enable the SESSION extension
AC_ARG_ENABLE(session, AC_HELP_STRING([--enable-session],
AC_ARG_ENABLE(session, AS_HELP_STRING([--enable-session],
[Enable the SESSION extension]))
AC_MSG_CHECKING([whether to support SESSION])
if test "${enable_session}" = "yes" -o "${enable_all}" = "yes" ; then
@@ -783,7 +772,7 @@ BUILD_CFLAGS=$ac_temp_BUILD_CFLAGS
#########
# See whether we should use GCOV
AC_ARG_ENABLE(gcov, AC_HELP_STRING([--enable-gcov],
AC_ARG_ENABLE(gcov, AS_HELP_STRING([--enable-gcov],
[Enable coverage testing using gcov]))
if test "${use_gcov}" = "yes" ; then
USE_GCOV=1
@@ -806,13 +795,14 @@ AC_SUBST(AMALGAMATION_LINE_MACROS)
#########
# Output the config header
AC_CONFIG_HEADERS(config.h)
AC_CONFIG_HEADERS(sqlite_cfg.h)
#########
# Generate the output files.
#
AC_SUBST(BUILD_CFLAGS)
AC_OUTPUT([
AC_CONFIG_FILES([
Makefile
sqlite3.pc
])
AC_OUTPUT
+107
View File
@@ -0,0 +1,107 @@
Begin Concurrent
================
## Overview
Usually, SQLite allows at most one writer to proceed concurrently. The
BEGIN CONCURRENT enhancement allows multiple writers to process write
transactions simultanously if the database is in "wal" or "wal2" mode,
although the system still serializes COMMIT commands.
When a write-transaction is opened with "BEGIN CONCURRENT", actually
locking the database is deferred until a COMMIT is executed. This means
that any number of transactions started with BEGIN CONCURRENT may proceed
concurrently. The system uses optimistic page-level-locking to prevent
conflicting concurrent transactions from being committed.
When a BEGIN CONCURRENT transaction is committed, the system checks whether
or not any of the database pages that the transaction has read have been
modified since the BEGIN CONCURRENT was opened. In other words - it asks
if the transaction being committed operates on a different set of data than
all other concurrently executing transactions. If the answer is "yes, this
transaction did not read or modify any data modified by any concurrent
transaction", then the transaction is committed as normal. Otherwise, if the
transaction does conflict, it cannot be committed and an SQLITE&#95;BUSY&#95;SNAPSHOT
error is returned. At this point, all the client can do is ROLLBACK the
transaction.
If SQLITE&#95;BUSY&#95;SNAPSHOT is returned, messages are output via the sqlite3&#95;log
mechanism indicating the page and table or index on which the conflict
occurred. This can be useful when optimizing concurrency.
## Application Programming Notes
In order to serialize COMMIT processing, SQLite takes a lock on the database
as part of each COMMIT command and releases it before returning. At most one
writer may hold this lock at any one time. If a writer cannot obtain the lock,
it uses SQLite's busy-handler to pause and retry for a while:
<a href=https://www.sqlite.org/c3ref/busy_handler.html>
https://www.sqlite.org/c3ref/busy_handler.html
</a>
If there is significant contention for the writer lock, this mechanism can be
inefficient. In this case it is better for the application to use a mutex or
some other mechanism that supports blocking to ensure that at most one writer
is attempting to COMMIT a BEGIN CONCURRENT transaction at a time. This is
usually easier if all writers are part of the same operating system process.
If all database clients (readers and writers) are located in the same OS
process, and if that OS is a Unix variant, then it can be more efficient to
the built-in VFS "unix-excl" instead of the default "unix". This is because it
uses more efficient locking primitives.
The key to maximizing concurrency using BEGIN CONCURRENT is to ensure that
there are a large number of non-conflicting transactions. In SQLite, each
table and each index is stored as a separate b-tree, each of which is
distributed over a discrete set of database pages. This means that:
* Two transactions that write to different sets of tables never
conflict, and that
* Two transactions that write to the same tables or indexes only
conflict if the values of the keys (either primary keys or indexed
rows) are fairly close together. For example, given a large
table with the schema:
<pre> CREATE TABLE t1(a INTEGER PRIMARY KEY, b BLOB);</pre>
writing two rows with adjacent values for "a" probably will cause a
conflict (as the two keys are stored on the same page), but writing two
rows with vastly different values for "a" will not (as the keys will likly
be stored on different pages).
Note that, in SQLite, if values are not explicitly supplied for an INTEGER
PRIMARY KEY, as for example in:
>
INSERT INTO t1(b) VALUES(&lt;blob-value>);
then monotonically increasing values are assigned automatically. This is
terrible for concurrency, as it all but ensures that all new rows are
added to the same database page. In such situations, it is better to
explicitly assign random values to INTEGER PRIMARY KEY fields.
This problem also comes up for non-WITHOUT ROWID tables that do not have an
explicit INTEGER PRIMARY KEY column. In these cases each table has an implicit
INTEGER PRIMARY KEY column that is assigned increasing values, leading to the
same problem as omitting to assign a value to an explicit INTEGER PRIMARY KEY
column.
For both explicit and implicit INTEGER PRIMARY KEYs, it is possible to have
SQLite assign values at random (instead of the monotonically increasing
values) by writing a row with a rowid equal to the largest possible signed
64-bit integer to the table. For example:
INSERT INTO t1(a) VALUES(9223372036854775807);
Applications should take care not to malfunction due to the presence of such
rows.
The nature of some types of indexes, for example indexes on timestamp fields,
can also cause problems (as concurrent transactions may assign similar
timestamps that will be stored on the same db page to new records). In these
cases the database schema may need to be rethought to increase the concurrency
provided by page-level-locking.
+167
View File
@@ -0,0 +1,167 @@
# Notes On Compiling SQLite On Windows 11
Here are step-by-step instructions on how to build SQLite from
canonical source on a new Windows 11 PC, as of 2023-11-01:
1. Install Microsoft Visual Studio. The free "community edition"
will work fine. Do a standard install for C++ development.
SQLite only needs the
"cl" compiler and the "nmake" build tool.
2. Under the "Start" menu, find "All Apps" then go to "Visual Studio 20XX"
and find "x64 Native Tools Command Prompt for VS 20XX". Pin that
application to your task bar, as you will use it a lot. Bring up
an instance of this command prompt and do all of the subsequent steps
in that "x64 Native Tools" command prompt. (Or use "x86" if you want
a 32-bit build.) The subsequent steps will not work in a vanilla
DOS prompt. Nor will they work in PowerShell.
3. Install TCL development libraries. This note assumes that you will
install the TCL development libraries in the "`c:\Tcl`" directory.
Make adjustments
if you want TCL installed somewhere else. SQLite needs both the
"tclsh.exe" command-line tool as part of the build process, and
the "tcl86.lib" library in order to run tests. You will need
TCL version 8.6 or later.
<ol type="a">
<li>Get the TCL source archive, perhaps from
[https://www.tcl.tk/software/tcltk/download.html](https://www.tcl.tk/software/tcltk/download.html).
<li>Untar or unzip the source archive. CD into the "win/" subfolder
of the source tree.
<li>Run: `nmake /f makefile.vc release`
<li>Run: `nmake /f makefile.vc INSTALLDIR=c:\Tcl install`
<li>CD to `c:\Tcl\lib`. In that subfolder make a copy of the
"`tcl86t.lib`" file to the alternative name "`tcl86.lib`"
(omitting the second 't'). Leave the copy in the same directory
as the original.
<li>CD to `c:\Tcl\bin`. Make a copy of the "`tclsh86t.exe`"
file into "`tclsh.exe`" (without the "86t") in the same directory.
<li>Add `c:\Tcl\bin` to your %PATH%. To do this, go to Settings
and search for "path". Select "edit environment variables for
your account" and modify your default PATH accordingly.
You will need to close and reopen your command prompts after
making this change.
</ol>
4. Download the SQLite source tree and unpack it. CD into the
toplevel directory of the source tree.
5. Set the TCLDIR environment variable to point to your TCL installation.
Like this:
<ul>
<li> `set TCLDIR=c:\Tcl`
</ul>
6. Run the "`Makefile.msc`" makefile with an appropriate target.
Examples:
<ul>
<li> `nmake /f makefile.msc`
<li> `nmake /f makefile.msc sqlite3.c`
<li> `nmake /f makefile.msc sqlite3.exe`
<li> `nmake /f makefile.msc sqldiff.exe`
<li> `nmake /f makefile.msc tclextension-install`
<li> `nmake /f makefile.msc devtest`
<li> `nmake /f makefile.msc releasetest`
<li> `nmake /f makefile.msc sqlite3_analyzer.exe`
</ul>
It is not required that you run the "tclextension-install" target prior to
running tests. However, the tests will run more smoothly if you do.
The version of SQLite used for the TCL extension does *not* need to
correspond to the version of SQLite under test. So you can install the
SQLite TCL extension once, and then use it to test many different versions
of SQLite.
7. For a debugging build of the CLI, where the ".treetrace" and ".wheretrace"
commands work, add the DEBUG=3 argument to nmake. Like this:
<ul>
<li> `nmake /f makefile.msc DEBUG=3 clean sqlite3.exe`
</ul>
## 32-bit Builds
Doing a 32-bit build is just like doing a 64-bit build with the
following minor changes:
1. Use the "x86 Native Tools Command Prompt" instead of
"x64 Native Tools Command Prompt". "**x86**" instead of "**x64**".
2. Use a different installation directory for TCL.
The recommended directory is `c:\tcl32`. Thus you end up
with two TCL builds:
<ul>
<li> `c:\tcl` &larr; 64-bit (the default)
<li> `c:\tcl32` &larr; 32-bit
</ul>
3. Ensure that `c:\tcl32\bin` comes before `c:\tcl\bin` on
your PATH environment variable. You can achieve this using
a command like:
<ul>
<li> `set PATH=c:\tcl32\bin;%PATH%`
</ul>
## Building a DLL
The command the developers use for building the deliverable DLL on the
[download page](https://sqlite.org/download.html) is as follows:
> ~~~~
nmake /f Makefile.msc sqlite3.dll USE_NATIVE_LIBPATHS=1 "OPTS=-DSQLITE_ENABLE_FTS3=1 -DSQLITE_ENABLE_FTS4=1 -DSQLITE_ENABLE_FTS5=1 -DSQLITE_ENABLE_RTREE=1 -DSQLITE_ENABLE_JSON1=1 -DSQLITE_ENABLE_GEOPOLY=1 -DSQLITE_ENABLE_SESSION=1 -DSQLITE_ENABLE_PREUPDATE_HOOK=1 -DSQLITE_ENABLE_SERIALIZE=1 -DSQLITE_ENABLE_MATH_FUNCTIONS=1"
~~~~
That command generates both the sqlite3.dll and sqlite3.def files. The same
command works for both 32-bit and 64-bit builds.
## Statically Linking The TCL Library
Some utility programs associated with SQLite need to be linked
with TCL in order to function. The [sqlite3_analyzer.exe program](https://sqlite.org/sqlanalyze.html)
is an example. You can build as described above, and then
enter:
> ~~~~
nmake /f Makefile.msc sqlite3_analyzer.exe
~~~~
And you will end up with a working executable. However, that executable
will depend on having the "tcl86.dll" library somewhere on your %PATH%.
Use the following steps to build an executable that has the TCL library
statically linked so that it does not depend on separate DLL:
1. Use the appropriate "Command Prompt" window - either x86 or
x64, depending on whether you want a 32-bit or 64-bit executable.
2. Untar the TCL source tarball into a fresh directory. CD into
the "win/" subfolder.
3. Run: `nmake /f makefile.vc OPTS=nothreads,static shell`
4. CD into the "Release*" subfolder that is created (note the
wildcard - the full name of the directory might vary). There
you will find the "tcl86s.lib" file. Copy this file into the
same directory that you put the "tcl86.lib" on your initial
installation. (In this document, that directory is
"`C:\Tcl32\lib`" for 32-bit builds and
"`C:\Tcl\lib`" for 64-bit builds.)
5. CD into your SQLite source code directory and build the desired
utility program, but add the following extra arguments to the
nmake command line:
<blockquote><pre>
CCOPTS="-DSTATIC_BUILD" LIBTCL="tcl86s.lib netapi32.lib user32.lib"
</pre></blockquote>
<p>So, for example, to build a statically linked version of
sqlite3_analyzer.exe, you might type:
<blockquote><pre>
nmake /f Makefile.msc CCOPTS="-DSTATIC_BUILD" LIBTCL="tcl86s.lib netapi32.lib user32.lib" sqlite3_analyzer.exe
</pre></blockquote>
6. After your executable is built, you can verify that it does not
depend on the TCL DLL by running:
<blockquote><pre>
dumpbin /dependents sqlite3_analyzer.exe
</pre></blockquote>
+290
View File
@@ -0,0 +1,290 @@
# The JSONB Format
This document describes SQLite's JSONB binary encoding of
JSON.
## 1.0 What Is JSONB?
Beginning with version 3.45.0 (circa 2024-01-01), SQLite supports an
alternative binary encoding of JSON which we call "JSONB". JSONB is
a binary format that stored as a BLOB.
The advantage of JSONB over ordinary text RFC 8259 JSON is that JSONB
is both slightly smaller (by between 5% and 10% in most cases) and
can be processed in less than half the number of CPU cycles. The built-in
[JSON SQL functions] of SQLite can accept either ordinary text JSON
or the binary JSONB encoding for any of their JSON inputs.
The "JSONB" name is inspired by [PostgreSQL](https://postgresql.org), but the
on-disk format for SQLite's JSONB is not the same as PostgreSQL's.
The two formats have the same name, but they have wildly different internal
representations and are not in any way binary compatible.
The central idea behind this JSONB specification is that each element
begins with a header that includes the size and type of that element.
The header takes the place of punctuation such as double-quotes,
curly-brackes, square-brackets, commas, and colons. Since the size
and type of each element is contained in its header, the element can
be read faster since it is no longer necessary to carefully scan forward
looking for the closing delimiter. The payload of JSONB is the same
as for corresponding text JSON. The same payload bytes occur in the
same order. The only real difference between JSONB and ordinary text
JSON is that JSONB includes a binary header on
each element and omits delimiter and separator punctuation.
### 1.1 Internal Use Only
The details of the JSONB are not intended to be visible to application
developers. Application developers should look at JSONB as an opaque BLOB
used internally by SQLite. Nevertheless, we want the format to be backwards
compatible across all future versions of SQLite. To that end, the format
is documented by this file in the source tree. But this file should be
used only by SQLite core developers, not by developers of applications
that only use SQLite.
## 2.0 The Purpose Of This Document
JSONB is not intended as an external format to be used by
applications. JSONB is designed for internal use by SQLite only.
Programmers do not need to understand the JSONB format in order to
use it effectively.
Applications should access JSONB only through the [JSON SQL functions],
not by looking at individual bytes of the BLOB.
However, JSONB is intended to be portable and backwards compatible
for all future versions of SQLite. In other words, you should not have
to export and reimport your SQLite database files when you upgrade to
a newer SQLite version. For that reason, the JSONB format needs to
be well-defined.
This document is therefore similar in purpose to the
[SQLite database file format] document that describes the on-disk
format of an SQLite database file. Applications are not expected
to directly read and write the bits and bytes of SQLite database files.
The SQLite database file format is carefully documented so that it
can be stable and enduring. In the same way, the JSONB representation
of JSON is documented here so that it too can be stable and enduring,
not so that applications can read or writes individual bytes.
## 3.0 Encoding
JSONB is a direct translation of the underlying text JSON. The difference
is that JSONB uses a binary encoding that is faster to parse compared to
the detailed syntax of text JSON.
Each JSON element is encoded as a header and a payload. The header
determines type of element (string, numeric, boolean, null, object, or
array) and the size of the payload. The header can be between 1 and
9 bytes in size. The payload can be any size from zero bytes up to the
maximum allowed BLOB size.
### 3.1 Payload Size
The upper four bits of the first byte of the header determine size of the
header and possibly also the size of the payload.
If the upper four bits have a value between 0 and 11, then the header is
exactly one byte in size and the payload size is determined by those
upper four bits. If the upper four bits have a value between 12 and 15,
that means that the total header size is 2, 3, 5, or 9 bytes and the
payload size is unsigned big-endian integer that is contained in the
subsequent bytes. The size integer is the one byte that following the
initial header byte if the upper four bits
are 12, two bytes if the upper bits are 13, four bytes if the upper bits
are 14, and eight bytes if the upper bits are 15. The current design
of SQLite does not support BLOB values larger than 2GiB, so the eight-byte
variant of the payload size integer will never be used by the current code.
The eight-byte payload size integer is included in the specification
to allow for future expansion.
The header for an element does *not* need to be in its simplest
form. For example, consider the JSON numeric value "`1`".
That element can be encode in five different ways:
* `0x13 0x31`
* `0xc3 0x01 0x31`
* `0xd3 0x00 0x01 0x31`
* `0xe3 0x00 0x00 0x00 0x01 0x31`
* `0xf3 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x01 0x31`
The shortest encoding is preferred, of course, and usually happens with
primitive elements such as numbers. However the total size of an array
or object might not be known exactly when the header of the element is
first generated. It is convenient to reserve space for the largest
possible header and then go back and fill in the correct payload size
at the end. This technique can result in array or object headers that
are larger than absolutely necessary.
### 3.2 Element Type
The least-significant four bits of the first byte of the header (the first
byte masked against 0x0f) determine element type. The following codes are
used:
<ol>
<li type="0"><p><b>NULL</b> &rarr;
The element is a JSON "null". The payload size for a true JSON NULL must
must be zero. Future versions of SQLite might extend the JSONB format
with elements that have a zero element type but a non-zero size. In that
way, legacy versions of SQLite will interpret the element as a NULL
for backwards compatibility while newer versions will interpret the
element in some other way.
<li value="1"><p><b>TRUE</b> &rarr;
The element is a JSON "true". The payload size must be zero for a actual
"true" value. Elements with type 1 and a non-zero payload size are
reserved for future expansion. Legacy implementations that see an element
type of 1 with a non-zero payload size should continue to interpret that
element as "true" for compatibility.
<li value="2"><p><b>FALSE</b> &rarr;
The element is a JSON "false". The payload size must be zero for a actual
"false" value. Elements with type 2 and a non-zero payload size are
reserved for future expansion. Legacy implementations that see an element
type of 2 with a non-zero payload size should continue to interpret that
element as "false" for compatibility.
<li value="3"><p><b>INT</b> &rarr;
The element is a JSON integer value in the canonical
RFC 8259 format, without extensions. The payload is the ASCII
text representation of that numeric value.
<li value="4"><p><b>INT5</b> &rarr;
The element is a JSON integer value that is not in the
canonical format. The payload is the ASCII
text representation of that numeric value. Because the payload is in a
non-standard format, it will need to be translated when the JSONB is
converted into RFC 8259 text JSON.
<li value="5"><p><b>FLOAT</b> &rarr;
The element is a JSON floating-point value in the canonical
RFC 8259 format, without extensions. The payload is the ASCII
text representation of that numeric value.
<li value="6"><p><b>FLOAT5</b> &rarr;
The element is a JSON floating-point value that is not in the
canonical format. The payload is the ASCII
text representation of that numeric value. Because the payload is in a
non-standard format, it will need to be translated when the JSONB is
converted into RFC 8259 text JSON.
<li value="7"><p><b>TEXT</b> &rarr;
The element is a JSON string value that does not contain
any escapes nor any characters that need to be escaped for either SQL or
JSON. The payload is the UTF8 text representation of the string value.
The payload does <i>not</i> include string delimiters.
<li value="8"><p><b>TEXTJ</b> &rarr;
The element is a JSON string value that contains
RFC 8259 character escapes (such as "<tt>\n</tt>" or "<tt>\u0020</tt>").
Those escapes will need to be translated into actual UTF8 if this element
is [json_extract|extracted] into SQL.
The payload is the UTF8 text representation of the escaped string value.
The payload does <i>not</i> include string delimiters.
<li value="9"><p><b>TEXT5</b> &rarr;
The element is a JSON string value that contains
character escapes, including some character escapes that part of JSON5
and which are not found in the canonical RFC 8259 spec.
Those escapes will need to be translated into standard JSON prior to
rendering the JSON as text, or into their actual UTF8 characters if this
element is [json_extract|extracted] into SQL.
The payload is the UTF8 text representation of the escaped string value.
The payload does <i>not</i> include string delimiters.
<li value="10"><p><b>TEXTRAW</b> &rarr;
The element is a JSON string value that contains
UTF8 characters that need to be escaped if this string is rendered into
standard JSON text.
The payload does <i>not</i> include string delimiters.
<li value="11"><p><b>ARRAY</b> &rarr;
The element is a JSON array. The payload contains
JSONB elements that comprise values contained within the array.
<li value="12"><p><b>OBJECT</b> &rarr;
The element is a JSON object. The payload contains
pairs of JSONB elements that comprise entries for the JSON object.
The first element in each pair must be a string (types 7 through 10).
The second element of each pair may be any types, including nested
arrays or objects.
<li value="13"><p><b>RESERVED-13</b> &rarr;
Reserved for future expansion. Legacy implements that encounter this
element type should raise an error.
<li value="14"><p><b>RESERVED-14</b> &rarr;
Reserved for future expansion. Legacy implements that encounter this
element type should raise an error.
<li value="15"><p><b>RESERVED-15</b> &rarr;
Reserved for future expansion. Legacy implements that encounter this
element type should raise an error.
</ol>
Element types outside the range of 0 to 12 are reserved for future
expansion. The current implement raises an error if see an element type
other than those listed above. However, future versions of SQLite might
use of the three remaining element types to implement indexing or similar
optimizations, to speed up lookup against large JSON arrays and/or objects.
### 3.3 Design Rationale For Element Types
A key goal of JSONB is that it should be quick to translate
to and from text JSON and/or be constructed from SQL values.
When converting from text into JSONB, we do not want the
converter subroutine to burn CPU cycles converting elements
values into some standard format which might never be used.
Format conversion is "lazy" - it is deferred until actually
needed. This has implications for the JSONB format design:
1. Numeric values are stored as text, not a numbers. The values are
a direct copy of the text JSON values from which they are derived.
2. There are multiple element types depending on the details of value
formats. For example, INT is used for pure RFC-8259 integer
literals and INT5 exists for JSON5 extensions such as hexadecimal
notation. FLOAT is used for pure RFC-8259 floating point literals
and FLOAT5 is used for JSON5 extensions. There are four different
representations of strings, depending on where the string came from
and how special characters within the string are escaped.
A second goal of JSONB is that it should be capable of serving as the
"parse tree" for JSON when a JSON value is being processed by the
various [JSON SQL functions] built into SQLite. Before JSONB was
developed, operations such [json_replace()] and [json_patch()]
and similar worked in three stages:
1. Translate the text JSON into a internal format that is
easier to scan and edit.
2. Perform the requested operation on the JSON.
3. Translate the internal format back into text.
JSONB seeks to serve as the internal format directly - bypassing
the first and third stages of that process. Since most of the CPU
cycles are spent on the first and third stages, that suggests that
JSONB processing will be much faster than text JSON processing.
So when processing JSONB, only the second stage of the three-stage
process is required. But when processing text JSON, it is still necessary
to do stages one and three. If JSONB is to be used as the internal
binary representation, this is yet another reason to store numeric
values as text. Storing numbers as text minimizes the amount of
conversion work needed for stages one and three. This is also why
there are four different representations of text in JSONB. Different
text representations are used for text coming from different sources
(RFC-8259 JSON, JSON5, or SQL string values) and conversions only
happen if and when they are actually needed.
### 3.4 Valid JSONB BLOBs
A valid JSONB BLOB consists of a single JSON element. The element must
exactly fill the BLOB. This one element is often a JSON object or array
and those usually contain additional elements as its payload, but the
element can be a primite value such a string, number, boolean, or null.
When the built-in JSON functions are attempting to determine if a BLOB
argument is a JSONB or just a random BLOB, they look at the header of
the outer element to see that it is well-formed and that the element
completely fills the BLOB. If these conditions are met, then the BLOB
is accepted as a JSONB value.
+21 -3
View File
@@ -322,7 +322,7 @@ These are the steps:
<ul>
<li> Declare a local variable of type "yyParser"
<li> Initialize the variable using ParseInit()
<li> Pass a pointer to the variable in calls ot Parse()
<li> Pass a pointer to the variable in calls to Parse()
<li> Deallocate substructure in the parse variable using ParseFinalize().
</ul>
@@ -683,6 +683,7 @@ other than that, the order of directives in Lemon is arbitrary.</p>
<li><tt><a href='#pifdef'>%endif</a></tt>
<li><tt><a href='#extraarg'>%extra_argument</a></tt>
<li><tt><a href='#pfallback'>%fallback</a></tt>
<li><tt><a href='#reallc'>%free</a></tt>
<li><tt><a href='#pifdef'>%if</a></tt>
<li><tt><a href='#pifdef'>%ifdef</a></tt>
<li><tt><a href='#pifdef'>%ifndef</a></tt>
@@ -693,6 +694,7 @@ other than that, the order of directives in Lemon is arbitrary.</p>
<li><tt><a href='#parse_accept'>%parse_accept</a></tt>
<li><tt><a href='#parse_failure'>%parse_failure</a></tt>
<li><tt><a href='#pright'>%right</a></tt>
<li><tt><a href='#reallc'>%realloc</a></tt>
<li><tt><a href='#stack_overflow'>%stack_overflow</a></tt>
<li><tt><a href='#stack_size'>%stack_size</a></tt>
<li><tt><a href='#start_symbol'>%start_symbol</a></tt>
@@ -1200,6 +1202,21 @@ match any input token.</p>
the wildcard token and some other token, the other token is always used.
The wildcard token is only matched if there are no alternatives.</p>
<a id='reallc'></a>
<h4>4.4.26 The <tt>%realloc</tt> and <tt>%free</tt> directives</h4>
<p>The <tt>%realloc</tt> and <tt>%free</tt> directives defines function
that allocate and free heap memory. The signatures of these functions
should be the same as the realloc() and free() functions from the standard
C library.
<p>If both of these functions are defined
then these functions are used to allocate and free
memory for supplemental parser stack space, if the initial
parse stack space is exceeded. The initial parser stack size
is specified by either <tt>%stack_size</tt> or the
-DYYSTACKDEPTH compile-time flag.
<a id='errors'></a>
<h2>5.0 Error Processing</h2>
@@ -1224,13 +1241,14 @@ to begin parsing a new file. This is what will happen at the very
first syntax error, of course, if there are no instances of the
"error" non-terminal in your grammar.</p>
<a id='history'></a>
<h2>6.0 History of Lemon</h2>
<p>Lemon was originally written by Richard Hipp sometime in the late
1980s on a Sun4 Workstation using K&amp;R C.
There was a companion LL(1) parser generator program named "Lime", the
source code to which as been lost.</p>
There was a companion LL(1) parser generator program named "Lime".
The Lime source code has been lost.</p>
<p>The lemon.c source file was originally many separate files that were
compiled together to generate the "lemon" executable. Sometime in the
+356
View File
@@ -0,0 +1,356 @@
# The testrunner.tcl Script
<ul type=none>
<li> 1. <a href=#overview>Overview</a>
<li> 2. <a href=#binary_tests>Binary Tests</a>
<ul type=none>
<li> 2.1. <a href=#organization_tests>Organization of Tcl Tests</a>
<li> 2.2. <a href=#run_tests>Commands to Run Tests</a>
<li> 2.3. <a href=#binary_test_failures>Investigating Binary Test Failures</a>
</ul>
<li> 3. <a href=#source_code_tests>Source Tests</a>
<ul type=none>
<li> 3.1. <a href=#commands_to_run_tests>Commands to Run SQLite Tests</a>
<li> 3.2. <a href=#zipvfs_tests>Running ZipVFS Tests</a>
<li> 3.3. <a href=#source_code_test_failures>Investigating Source Code Test Failures</a>
</ul>
<li> 4. <a href=#testrunner_options>Extra testrunner.tcl Options</a>
<li> 5. <a href=#cpu_cores>Controlling CPU Core Utilization</a>
</ul>
<a name=overview></a>
# 1. Overview
testrunner.tcl is a Tcl script used to run multiple SQLite tests using
multiple jobs. It supports the following types of tests:
* Tcl test scripts.
* Tests run with `make` commands. Examples:
- `make mdevtest`
- `make releasetest`
- `make sdevtest`
- `make testrunner`
testrunner.tcl pipes the output of all tests and builds run into log file
**testrunner.log**, created in the current working directory. Search this
file to find details of errors. Suggested search commands:
* `grep "^!" testrunner.log`
* `grep failed testrunner.log`
testrunner.tcl also populates SQLite database **testrunner.db**. This database
contains details of all tests run, running and to be run. A useful query
might be:
```
SELECT * FROM script WHERE state='failed'
```
Running the command:
```
./testfixture $(TESTDIR)/testrunner.tcl status
```
in the directory containing the testrunner.db database runs various queries
to produce a succinct report on the state of a running testrunner.tcl script.
Running:
```
watch ./testfixture $(TESTDIR)/testrunner.tcl status
```
in another terminal is a good way to keep an eye on a long running test.
Sometimes testrunner.tcl uses the `testfixture` binary that it is run with
to run tests (see "Binary Tests" below). Sometimes it builds testfixture and
other binaries in specific configurations to test (see "Source Tests").
<a name=binary_tests></a>
# 2. Binary Tests
The commands described in this section all run various combinations of the Tcl
test scripts using the `testfixture` binary used to run the testrunner.tcl
script (i.e. they do not invoke the compiler to build new binaries, or the
`make` command to run tests that are not Tcl scripts). The procedure to run
these tests is therefore:
1. Build the "testfixture" (or "testfixture.exe" for windows) binary using
whatever method seems convenient.
2. Test the binary built in step 1 by running testrunner.tcl with it,
perhaps with various options.
The following sub-sections describe the various options that can be
passed to testrunner.tcl to test binary testfixture builds.
<a name=organization_tests></a>
## 2.1. Organization of Tcl Tests
Tcl tests are stored in files that match the pattern *\*.test*. They are
found in both the $TOP/test/ directory, and in the various sub-directories
of the $TOP/ext/ directory of the source tree. Not all *\*.test* files
contain Tcl tests - a handful are Tcl scripts designed to invoke other
*\*.test* files.
The **veryquick** set of tests is a subset of all Tcl test scripts in the
source tree. In includes most tests, but excludes some that are very slow.
Almost all fault-injection tests (those that test the response of the library
to OOM or IO errors) are excluded. It is defined in source file
*test/permutations.test*.
The **full** set of tests includes all Tcl test scripts in the source tree.
To run a "full" test is to run all Tcl test scripts that can be found in the
source tree.
File *permutations.test* defines various test "permutations". A permutation
consists of:
* A subset of Tcl test scripts, and
* Runtime configuration to apply before running each test script
(e.g. enabling auto-vacuum, or disable lookaside).
Running **all** tests is to run all tests in the full test set, plus a dozen
or so permutations. The specific permutations that are run as part of "all"
are defined in file *testrunner_data.tcl*.
<a name=run_tests></a>
## 2.2. Commands to Run Tests
To run the "veryquick" test set, use either of the following:
```
./testfixture $TESTDIR/testrunner.tcl
./testfixture $TESTDIR/testrunner.tcl veryquick
```
To run the "full" test suite:
```
./testfixture $TESTDIR/testrunner.tcl full
```
To run the subset of the "full" test suite for which the test file name matches
a specified pattern (e.g. all tests that start with "fts5"), either of:
```
./testfixture $TESTDIR/testrunner.tcl fts5%
./testfixture $TESTDIR/testrunner.tcl 'fts5*'
```
Strictly speaking, for a test to be run the pattern must match the script
filename, not including the directory, using the rules of Tcl's
\[string match\] command. Except that before the matching is done, any "%"
characters specified as part of the pattern are transformed to "\*".
To run "all" tests (full + permutations):
```
./testfixture $TESTDIR/testrunner.tcl all
```
<a name=binary_test_failures></a>
## 2.3. Investigating Binary Test Failures
If a test fails, testrunner.tcl reports name of the Tcl test script and, if
applicable, the name of the permutation, to stdout. This information can also
be retrieved from either *testrunner.log* or *testrunner.db*.
If there is no permutation, the individual test script may be run with:
```
./testfixture $PATH_TO_SCRIPT
```
Or, if the failure occured as part of a permutation:
```
./testfixture $TESTDIR/testrunner.tcl $PERMUTATION $PATH_TO_SCRIPT
```
TODO: An example instead of "$PERMUTATION" and $PATH\_TO\_SCRIPT?
<a name=source_code_tests></a>
# 3. Source Code Tests
The commands described in this section invoke the C compiler to build
binaries from the source tree, then use those binaries to run Tcl and
other tests. The advantages of this are that:
* it is possible to test multiple build configurations with a single
command, and
* it ensures that tests are always run using binaries created with the
same set of compiler options.
The testrunner.tcl commands described in this section may be run using
either a *testfixture* (or testfixture.exe) build, or with any other Tcl
shell that supports SQLite 3.31.1 or newer via "package require sqlite3".
TODO: ./configure + Makefile.msc build systems.
<a name=commands_to_run_tests></a>
## 3.1. Commands to Run SQLite Tests
The **mdevtest** command is equivalent to running the veryquick tests and
the `make fuzztest` target once for each of two --enable-all builds - one
with debugging enabled and one without:
```
tclsh $TESTDIR/testrunner.tcl mdevtest
```
In other words, it is equivalent to running:
```
$TOP/configure --enable-all --enable-debug
make fuzztest
make testfixture
./testfixture $TOP/test/testrunner.tcl veryquick
# Then, after removing files created by the tests above:
$TOP/configure --enable-all OPTS="-O0"
make fuzztest
make testfixture
./testfixture $TOP/test/testrunner.tcl veryquick
```
The **sdevtest** command is identical to the mdevtest command, except that the
second of the two builds is a sanitizer build. Specifically, this means that
OPTS="-fsanitize=address,undefined" is specified instead of OPTS="-O0":
```
tclsh $TESTDIR/testrunner.tcl sdevtest
```
The **release** command runs lots of tests under lots of builds. It runs
different combinations of builds and tests depending on whether it is run
on Linux, Windows or OSX. Refer to *testrunner\_data.tcl* for the details
of the specific tests run.
```
tclsh $TESTDIR/testrunner.tcl release
```
As with <a href=#source code tests>source code tests</a>, one or more patterns
may be appended to any of the above commands (mdevtest, sdevtest or release).
In that case only Tcl tests (no fuzz or other tests) that match the specified
pattern are run. For example, to run the just the Tcl rtree tests in all
builds and configurations supported by "release":
```
tclsh $TESTDIR/testrunner.tcl release rtree%
```
<a name=zipvfs_tests></a>
## 3.2. Running ZipVFS Tests
testrunner.tcl can build a zipvfs-enabled testfixture and use it to run
tests from the Zipvfs project with the following command:
```
tclsh $TESTDIR/testrunner.tcl --zipvfs $PATH_TO_ZIPVFS
```
This can be combined with any of "mdevtest", "sdevtest" or "release" to
test both SQLite and Zipvfs with a single command:
```
tclsh $TESTDIR/testrunner.tcl --zipvfs $PATH_TO_ZIPVFS mdevtest
```
<a name=source_code_test_failures></a>
## 3.3. Investigating Source Code Test Failures
Investigating a test failure that occurs during source code testing is a
two step process:
1. Recreating the build configuration in which the test failed, and
2. Re-running the actual test.
To recreate a build configuration, use the testrunner.tcl **script** command
to create a build script. A build script is a bash script on Linux or OSX, or
a dos \*.bat file on windows. For example:
```
# Create a script that recreates build configuration "Device-One" on
# Linux or OSX:
tclsh $TESTDIR/testrunner.tcl script Device-One > make.sh
# Create a script that recreates build configuration "Have-Not" on Windows:
tclsh $TESTDIR/testrunner.tcl script Have-Not > make.bat
```
The generated bash or \*.bat file script accepts a single argument - a makefile
target to build. This may be used either to run a `make` command test directly,
or else to build a testfixture (or testfixture.exe) binary with which to
run a Tcl test script, as <a href=#binary_test_failures>described above</a>.
<a name=testrunner_options></a>
# 4. Extra testrunner.tcl Options
The testrunner.tcl script options in this section may be used with both source
code and binary tests.
The **--buildonly** option instructs testrunner.tcl just to build the binaries
required by a test, not to run any actual tests. For example:
```
# Build binaries required by release test.
tclsh $TESTDIR/testrunner.tcl --buildonly release"
```
The **--dryrun** option prevents testrunner.tcl from building any binaries
or running any tests. Instead, it just writes the shell commands that it
would normally execute into the testrunner.log file. Example:
```
# Log the shell commmands that make up the mdevtest test.
tclsh $TESTDIR/testrunner.tcl --dryrun mdevtest"
```
The **--explain** option is similar to --dryrun in that it prevents testrunner.tcl
from building any binaries or running any tests. The difference is that --explain
prints on standard output a human-readable summary of all the builds and tests that
would have been run.
```
# Show what builds and tests would have been run
tclsh $TESTDIR/testrunner.tcl --explain mdevtest
```
<a name=cpu_cores></a>
# 5. Controlling CPU Core Utilization
When running either binary or source code tests, testrunner.tcl reports the
number of jobs it intends to use to stdout. e.g.
```
$ ./testfixture $TESTDIR/testrunner.tcl
splitting work across 16 jobs
... more output ...
```
By default, testfixture.tcl attempts to set the number of jobs to the number
of real cores on the machine. This can be overridden using the "--jobs" (or -j)
switch:
```
$ ./testfixture $TESTDIR/testrunner.tcl --jobs 8
splitting work across 8 jobs
... more output ...
```
The number of jobs may also be changed while an instance of testrunner.tcl is
running by exucuting the following command from the directory containing the
testrunner.log and testrunner.db files:
```
$ ./testfixture $TESTDIR/testrunner.tcl njob $NEW_NUMBER_OF_JOBS
```
+98
View File
@@ -0,0 +1,98 @@
Wal2 Mode Notes
===============
## Activating/Deactivating Wal2 Mode
"Wal2" mode is very similar to "wal" mode. To change a database to wal2 mode,
use the command:
>
PRAGMA journal_mode = wal2;
It is not possible to change a database directly from "wal" mode to "wal2"
mode. Instead, it must first be changed to rollback mode. So, to change a wal
mode database to wal2 mode, the following two commands may be used:
>
PRAGMA journal_mode = delete;
PRAGMA journal_mode = wal2;
A database in wal2 mode may only be accessed by versions of SQLite compiled
from this branch. Attempting to use any other version of SQLite results in an
SQLITE&#95;NOTADB error. A wal2 mode database may be changed back to rollback mode
(making it accessible by all versions of SQLite) using:
>
PRAGMA journal_mode = delete;
## The Advantage of Wal2 Mode
In legacy wal mode, when a writer writes data to the database, it doesn't
modify the database file directly. Instead, it appends new data to the
"&lt;database>-wal" file. Readers read data from both the original database
file and the "&lt;database>-wal" file. At some point, data is copied from the
"&lt;database>-wal" file into the database file, after which the wal file can
be deleted or overwritten. Copying data from the wal file into the database
file is called a "checkpoint", and may be done explictly (either by "PRAGMA
wal&#95;checkpoint" or sqlite3&#95;wal&#95;checkpoint&#95;v2()), or
automatically (by configuring "PRAGMA wal&#95;autocheckpoint" - this is the
default).
Checkpointers do not block writers, and writers do not block checkpointers.
However, if a writer writes to the database while a checkpoint is ongoing,
then the new data is appended to the end of the wal file. This means that,
even following the checkpoint, the wal file cannot be overwritten or deleted,
and so all subsequent transactions must also be appended to the wal file. The
work of the checkpointer is not wasted - SQLite remembers which parts of the
wal file have already been copied into the db file so that the next checkpoint
does not have to do so again - but it does mean that the wal file may grow
indefinitely if the checkpointer never gets a chance to finish without a
writer appending to the wal file. There are also circumstances in which
long-running readers may prevent a checkpointer from checkpointing the entire
wal file - also causing the wal file to grow indefinitely in a busy system.
Wal2 mode does not have this problem. In wal2 mode, wal files do not grow
indefinitely even if the checkpointer never has a chance to finish
uninterrupted.
In wal2 mode, the system uses two wal files instead of one. The files are named
"&lt;database>-wal" and "&lt;database>-wal2", where "&lt;database>" is of
course the name of the database file. When data is written to the database, the
writer begins by appending the new data to the first wal file. Once the first
wal file has grown large enough, writers switch to appending data to the second
wal file. At this point the first wal file can be checkpointed (after which it
can be overwritten). Then, once the second wal file has grown large enough and
the first wal file has been checkpointed, writers switch back to the first wal
file. And so on.
## Application Programming
From the point of view of the user, the main differences between wal and
wal2 mode are to do with checkpointing:
* In wal mode, a checkpoint may be attempted at any time. In wal2
mode, the checkpointer has to wait until writers have switched
to the "other" wal file before a checkpoint can take place.
* In wal mode, the wal-hook (callback registered using
sqlite3&#95;wal&#95;hook()) is invoked after a transaction is committed
with the total number of pages in the wal file as an argument. In wal2
mode, the argument is either the total number of uncheckpointed pages in
both wal files, or - if the "other" wal file is empty or already
checkpointed - 0.
Clients are recommended to use the same strategies for checkpointing wal2 mode
databases as for wal databases - by registering a wal-hook using
sqlite3&#95;wal&#95;hook() and attempting a checkpoint when the parameter
exceeds a certain threshold.
However, it should be noted that although the wal-hook is invoked after each
transaction is committed to disk and database locks released, it is still
invoked from within the sqlite3&#95;step() call used to execute the "COMMIT"
command. In BEGIN CONCURRENT systems, where the "COMMIT" is often protected by
an application mutex, this may reduce concurrency. In such systems, instead of
executing a checkpoint from within the wal-hook, a thread might defer this
action until after the application mutex has been released.
+773
View File
@@ -0,0 +1,773 @@
/*
** 2023 November 4
**
** The author disclaims copyright to this source code. In place of
** a legal notice, here is a blessing:
**
** May you do good and not evil.
** May you find forgiveness for yourself and forgive others.
** May you share freely, never taking more than you give.
**
********************************************************************************
** This file implements various interfaces used for console and stream I/O
** by the SQLite project command-line tools, as explained in console_io.h .
** Functions prefixed by "SQLITE_INTERNAL_LINKAGE" behave as described there.
*/
#ifndef SQLITE_CDECL
# define SQLITE_CDECL
#endif
#ifndef SHELL_NO_SYSINC
# include <stdarg.h>
# include <string.h>
# include <stdlib.h>
# include <limits.h>
# include <assert.h>
# include "sqlite3.h"
#endif
#ifndef HAVE_CONSOLE_IO_H
# include "console_io.h"
#endif
#if defined(_MSC_VER)
# pragma warning(disable : 4204)
#endif
#ifndef SQLITE_CIO_NO_TRANSLATE
# if (defined(_WIN32) || defined(WIN32)) && !SQLITE_OS_WINRT
# ifndef SHELL_NO_SYSINC
# include <io.h>
# include <fcntl.h>
# undef WIN32_LEAN_AND_MEAN
# define WIN32_LEAN_AND_MEAN
# include <windows.h>
# endif
# define CIO_WIN_WC_XLATE 1 /* Use WCHAR Windows APIs for console I/O */
# else
# ifndef SHELL_NO_SYSINC
# include <unistd.h>
# endif
# define CIO_WIN_WC_XLATE 0 /* Use plain C library stream I/O at console */
# endif
#else
# define CIO_WIN_WC_XLATE 0 /* Not exposing translation routines at all */
#endif
#if CIO_WIN_WC_XLATE
static HANDLE handleOfFile(FILE *pf){
int fileDesc = _fileno(pf);
union { intptr_t osfh; HANDLE fh; } fid = {
(fileDesc>=0)? _get_osfhandle(fileDesc) : (intptr_t)INVALID_HANDLE_VALUE
};
return fid.fh;
}
#endif
#ifndef SQLITE_CIO_NO_TRANSLATE
typedef struct PerStreamTags {
# if CIO_WIN_WC_XLATE
HANDLE hx;
DWORD consMode;
char acIncomplete[4];
# else
short reachesConsole;
# endif
FILE *pf;
} PerStreamTags;
/* Define NULL-like value for things which can validly be 0. */
# define SHELL_INVALID_FILE_PTR ((FILE *)~0)
# if CIO_WIN_WC_XLATE
# define SHELL_INVALID_CONS_MODE 0xFFFF0000
# endif
# if CIO_WIN_WC_XLATE
# define PST_INITIALIZER { INVALID_HANDLE_VALUE, SHELL_INVALID_CONS_MODE, \
{0,0,0,0}, SHELL_INVALID_FILE_PTR }
# else
# define PST_INITIALIZER { 0, SHELL_INVALID_FILE_PTR }
# endif
/* Quickly say whether a known output is going to the console. */
# if CIO_WIN_WC_XLATE
static short pstReachesConsole(PerStreamTags *ppst){
return (ppst->hx != INVALID_HANDLE_VALUE);
}
# else
# define pstReachesConsole(ppst) 0
# endif
# if CIO_WIN_WC_XLATE
static void restoreConsoleArb(PerStreamTags *ppst){
if( pstReachesConsole(ppst) ) SetConsoleMode(ppst->hx, ppst->consMode);
}
# else
# define restoreConsoleArb(ppst)
# endif
/* Say whether FILE* appears to be a console, collect associated info. */
static short streamOfConsole(FILE *pf, /* out */ PerStreamTags *ppst){
# if CIO_WIN_WC_XLATE
short rv = 0;
DWORD dwCM = SHELL_INVALID_CONS_MODE;
HANDLE fh = handleOfFile(pf);
ppst->pf = pf;
if( INVALID_HANDLE_VALUE != fh ){
rv = (GetFileType(fh) == FILE_TYPE_CHAR && GetConsoleMode(fh,&dwCM));
}
ppst->hx = (rv)? fh : INVALID_HANDLE_VALUE;
ppst->consMode = dwCM;
return rv;
# else
ppst->pf = pf;
ppst->reachesConsole = ( (short)isatty(fileno(pf)) );
return ppst->reachesConsole;
# endif
}
# ifndef ENABLE_VIRTUAL_TERMINAL_PROCESSING
# define ENABLE_VIRTUAL_TERMINAL_PROCESSING (0x4)
# endif
# if CIO_WIN_WC_XLATE
/* Define console modes for use with the Windows Console API. */
# define SHELL_CONI_MODE \
(ENABLE_ECHO_INPUT | ENABLE_INSERT_MODE | ENABLE_LINE_INPUT | 0x80 \
| ENABLE_QUICK_EDIT_MODE | ENABLE_EXTENDED_FLAGS | ENABLE_PROCESSED_INPUT)
# define SHELL_CONO_MODE (ENABLE_PROCESSED_OUTPUT | ENABLE_WRAP_AT_EOL_OUTPUT \
| ENABLE_VIRTUAL_TERMINAL_PROCESSING)
# endif
typedef struct ConsoleInfo {
PerStreamTags pstSetup[3];
PerStreamTags pstDesignated[3];
StreamsAreConsole sacSetup;
} ConsoleInfo;
static short isValidStreamInfo(PerStreamTags *ppst){
return (ppst->pf != SHELL_INVALID_FILE_PTR);
}
static ConsoleInfo consoleInfo = {
{ /* pstSetup */ PST_INITIALIZER, PST_INITIALIZER, PST_INITIALIZER },
{ /* pstDesignated[] */ PST_INITIALIZER, PST_INITIALIZER, PST_INITIALIZER },
SAC_NoConsole /* sacSetup */
};
SQLITE_INTERNAL_LINKAGE FILE* invalidFileStream = (FILE *)~0;
# if CIO_WIN_WC_XLATE
static void maybeSetupAsConsole(PerStreamTags *ppst, short odir){
if( pstReachesConsole(ppst) ){
DWORD cm = odir? SHELL_CONO_MODE : SHELL_CONI_MODE;
SetConsoleMode(ppst->hx, cm);
}
}
# else
# define maybeSetupAsConsole(ppst,odir)
# endif
SQLITE_INTERNAL_LINKAGE void consoleRenewSetup(void){
# if CIO_WIN_WC_XLATE
int ix = 0;
while( ix < 6 ){
PerStreamTags *ppst = (ix<3)?
&consoleInfo.pstSetup[ix] : &consoleInfo.pstDesignated[ix-3];
maybeSetupAsConsole(ppst, (ix % 3)>0);
++ix;
}
# endif
}
SQLITE_INTERNAL_LINKAGE StreamsAreConsole
consoleClassifySetup( FILE *pfIn, FILE *pfOut, FILE *pfErr ){
StreamsAreConsole rv = SAC_NoConsole;
FILE* apf[3] = { pfIn, pfOut, pfErr };
int ix;
for( ix = 2; ix >= 0; --ix ){
PerStreamTags *ppst = &consoleInfo.pstSetup[ix];
if( streamOfConsole(apf[ix], ppst) ){
rv |= (SAC_InConsole<<ix);
}
consoleInfo.pstDesignated[ix] = *ppst;
if( ix > 0 ) fflush(apf[ix]);
}
consoleInfo.sacSetup = rv;
consoleRenewSetup();
return rv;
}
SQLITE_INTERNAL_LINKAGE void SQLITE_CDECL consoleRestore( void ){
# if CIO_WIN_WC_XLATE
static ConsoleInfo *pci = &consoleInfo;
if( pci->sacSetup ){
int ix;
for( ix=0; ix<3; ++ix ){
if( pci->sacSetup & (SAC_InConsole<<ix) ){
PerStreamTags *ppst = &pci->pstSetup[ix];
SetConsoleMode(ppst->hx, ppst->consMode);
}
}
}
# endif
}
#endif /* !defined(SQLITE_CIO_NO_TRANSLATE) */
#ifdef SQLITE_CIO_INPUT_REDIR
/* Say whether given FILE* is among those known, via either
** consoleClassifySetup() or set{Output,Error}Stream, as
** readable, and return an associated PerStreamTags pointer
** if so. Otherwise, return 0.
*/
static PerStreamTags * isKnownReadable(FILE *pf){
static PerStreamTags *apst[] = {
&consoleInfo.pstDesignated[0], &consoleInfo.pstSetup[0], 0
};
int ix = 0;
do {
if( apst[ix]->pf == pf ) break;
} while( apst[++ix] != 0 );
return apst[ix];
}
#endif
#ifndef SQLITE_CIO_NO_TRANSLATE
/* Say whether given FILE* is among those known, via either
** consoleClassifySetup() or set{Output,Error}Stream, as
** writable, and return an associated PerStreamTags pointer
** if so. Otherwise, return 0.
*/
static PerStreamTags * isKnownWritable(FILE *pf){
static PerStreamTags *apst[] = {
&consoleInfo.pstDesignated[1], &consoleInfo.pstDesignated[2],
&consoleInfo.pstSetup[1], &consoleInfo.pstSetup[2], 0
};
int ix = 0;
do {
if( apst[ix]->pf == pf ) break;
} while( apst[++ix] != 0 );
return apst[ix];
}
static FILE *designateEmitStream(FILE *pf, unsigned chix){
FILE *rv = consoleInfo.pstDesignated[chix].pf;
if( pf == invalidFileStream ) return rv;
else{
/* Setting a possibly new output stream. */
PerStreamTags *ppst = isKnownWritable(pf);
if( ppst != 0 ){
PerStreamTags pst = *ppst;
consoleInfo.pstDesignated[chix] = pst;
}else streamOfConsole(pf, &consoleInfo.pstDesignated[chix]);
}
return rv;
}
SQLITE_INTERNAL_LINKAGE FILE *setOutputStream(FILE *pf){
return designateEmitStream(pf, 1);
}
# ifdef CONSIO_SET_ERROR_STREAM
SQLITE_INTERNAL_LINKAGE FILE *setErrorStream(FILE *pf){
return designateEmitStream(pf, 2);
}
# endif
#endif /* !defined(SQLITE_CIO_NO_TRANSLATE) */
#ifndef SQLITE_CIO_NO_SETMODE
# if CIO_WIN_WC_XLATE
static void setModeFlushQ(FILE *pf, short bFlush, int mode){
if( bFlush ) fflush(pf);
_setmode(_fileno(pf), mode);
}
# else
# define setModeFlushQ(f, b, m) if(b) fflush(f)
# endif
SQLITE_INTERNAL_LINKAGE void setBinaryMode(FILE *pf, short bFlush){
setModeFlushQ(pf, bFlush, _O_BINARY);
}
SQLITE_INTERNAL_LINKAGE void setTextMode(FILE *pf, short bFlush){
setModeFlushQ(pf, bFlush, _O_TEXT);
}
# undef setModeFlushQ
#else /* defined(SQLITE_CIO_NO_SETMODE) */
# define setBinaryMode(f, bFlush) do{ if((bFlush)) fflush(f); }while(0)
# define setTextMode(f, bFlush) do{ if((bFlush)) fflush(f); }while(0)
#endif /* defined(SQLITE_CIO_NO_SETMODE) */
#ifndef SQLITE_CIO_NO_TRANSLATE
# if CIO_WIN_WC_XLATE
/* Write buffer cBuf as output to stream known to reach console,
** limited to ncTake char's. Return ncTake on success, else 0. */
static int conZstrEmit(PerStreamTags *ppst, const char *z, int ncTake){
int rv = 0;
if( z!=NULL ){
int nwc = MultiByteToWideChar(CP_UTF8,0, z,ncTake, 0,0);
if( nwc > 0 ){
WCHAR *zw = sqlite3_malloc64(nwc*sizeof(WCHAR));
if( zw!=NULL ){
nwc = MultiByteToWideChar(CP_UTF8,0, z,ncTake, zw,nwc);
if( nwc > 0 ){
/* Translation from UTF-8 to UTF-16, then WCHARs out. */
if( WriteConsoleW(ppst->hx, zw,nwc, 0, NULL) ){
rv = ncTake;
}
}
sqlite3_free(zw);
}
}
}
return rv;
}
/* For {f,o,e}PrintfUtf8() when stream is known to reach console. */
static int conioVmPrintf(PerStreamTags *ppst, const char *zFormat, va_list ap){
char *z = sqlite3_vmprintf(zFormat, ap);
if( z ){
int rv = conZstrEmit(ppst, z, (int)strlen(z));
sqlite3_free(z);
return rv;
}else return 0;
}
# endif /* CIO_WIN_WC_XLATE */
# ifdef CONSIO_GET_EMIT_STREAM
static PerStreamTags * getDesignatedEmitStream(FILE *pf, unsigned chix,
PerStreamTags *ppst){
PerStreamTags *rv = isKnownWritable(pf);
short isValid = (rv!=0)? isValidStreamInfo(rv) : 0;
if( rv != 0 && isValid ) return rv;
streamOfConsole(pf, ppst);
return ppst;
}
# endif
/* Get stream info, either for designated output or error stream when
** chix equals 1 or 2, or for an arbitrary stream when chix == 0.
** In either case, ppst references a caller-owned PerStreamTags
** struct which may be filled in if none of the known writable
** streams is being held by consoleInfo. The ppf parameter is a
** byref output when chix!=0 and a byref input when chix==0.
*/
static PerStreamTags *
getEmitStreamInfo(unsigned chix, PerStreamTags *ppst,
/* in/out */ FILE **ppf){
PerStreamTags *ppstTry;
FILE *pfEmit;
if( chix > 0 ){
ppstTry = &consoleInfo.pstDesignated[chix];
if( !isValidStreamInfo(ppstTry) ){
ppstTry = &consoleInfo.pstSetup[chix];
pfEmit = ppst->pf;
}else pfEmit = ppstTry->pf;
if( !isValidStreamInfo(ppstTry) ){
pfEmit = (chix > 1)? stderr : stdout;
ppstTry = ppst;
streamOfConsole(pfEmit, ppstTry);
}
*ppf = pfEmit;
}else{
ppstTry = isKnownWritable(*ppf);
if( ppstTry != 0 ) return ppstTry;
streamOfConsole(*ppf, ppst);
return ppst;
}
return ppstTry;
}
SQLITE_INTERNAL_LINKAGE int oPrintfUtf8(const char *zFormat, ...){
va_list ap;
int rv;
FILE *pfOut;
PerStreamTags pst = PST_INITIALIZER; /* for unknown streams */
# if CIO_WIN_WC_XLATE
PerStreamTags *ppst = getEmitStreamInfo(1, &pst, &pfOut);
# else
getEmitStreamInfo(1, &pst, &pfOut);
# endif
assert(zFormat!=0);
va_start(ap, zFormat);
# if CIO_WIN_WC_XLATE
if( pstReachesConsole(ppst) ){
rv = conioVmPrintf(ppst, zFormat, ap);
}else{
# endif
rv = vfprintf(pfOut, zFormat, ap);
# if CIO_WIN_WC_XLATE
}
# endif
va_end(ap);
return rv;
}
SQLITE_INTERNAL_LINKAGE int ePrintfUtf8(const char *zFormat, ...){
va_list ap;
int rv;
FILE *pfErr;
PerStreamTags pst = PST_INITIALIZER; /* for unknown streams */
# if CIO_WIN_WC_XLATE
PerStreamTags *ppst = getEmitStreamInfo(2, &pst, &pfErr);
# else
getEmitStreamInfo(2, &pst, &pfErr);
# endif
assert(zFormat!=0);
va_start(ap, zFormat);
# if CIO_WIN_WC_XLATE
if( pstReachesConsole(ppst) ){
rv = conioVmPrintf(ppst, zFormat, ap);
}else{
# endif
rv = vfprintf(pfErr, zFormat, ap);
# if CIO_WIN_WC_XLATE
}
# endif
va_end(ap);
return rv;
}
SQLITE_INTERNAL_LINKAGE int fPrintfUtf8(FILE *pfO, const char *zFormat, ...){
va_list ap;
int rv;
PerStreamTags pst = PST_INITIALIZER; /* for unknown streams */
# if CIO_WIN_WC_XLATE
PerStreamTags *ppst = getEmitStreamInfo(0, &pst, &pfO);
# else
getEmitStreamInfo(0, &pst, &pfO);
# endif
assert(zFormat!=0);
va_start(ap, zFormat);
# if CIO_WIN_WC_XLATE
if( pstReachesConsole(ppst) ){
maybeSetupAsConsole(ppst, 1);
rv = conioVmPrintf(ppst, zFormat, ap);
if( 0 == isKnownWritable(ppst->pf) ) restoreConsoleArb(ppst);
}else{
# endif
rv = vfprintf(pfO, zFormat, ap);
# if CIO_WIN_WC_XLATE
}
# endif
va_end(ap);
return rv;
}
SQLITE_INTERNAL_LINKAGE int fPutsUtf8(const char *z, FILE *pfO){
PerStreamTags pst = PST_INITIALIZER; /* for unknown streams */
# if CIO_WIN_WC_XLATE
PerStreamTags *ppst = getEmitStreamInfo(0, &pst, &pfO);
# else
getEmitStreamInfo(0, &pst, &pfO);
# endif
assert(z!=0);
# if CIO_WIN_WC_XLATE
if( pstReachesConsole(ppst) ){
int rv;
maybeSetupAsConsole(ppst, 1);
rv = conZstrEmit(ppst, z, (int)strlen(z));
if( 0 == isKnownWritable(ppst->pf) ) restoreConsoleArb(ppst);
return rv;
}else {
# endif
return (fputs(z, pfO)<0)? 0 : (int)strlen(z);
# if CIO_WIN_WC_XLATE
}
# endif
}
SQLITE_INTERNAL_LINKAGE int ePutsUtf8(const char *z){
FILE *pfErr;
PerStreamTags pst = PST_INITIALIZER; /* for unknown streams */
# if CIO_WIN_WC_XLATE
PerStreamTags *ppst = getEmitStreamInfo(2, &pst, &pfErr);
# else
getEmitStreamInfo(2, &pst, &pfErr);
# endif
assert(z!=0);
# if CIO_WIN_WC_XLATE
if( pstReachesConsole(ppst) ) return conZstrEmit(ppst, z, (int)strlen(z));
else {
# endif
return (fputs(z, pfErr)<0)? 0 : (int)strlen(z);
# if CIO_WIN_WC_XLATE
}
# endif
}
SQLITE_INTERNAL_LINKAGE int oPutsUtf8(const char *z){
FILE *pfOut;
PerStreamTags pst = PST_INITIALIZER; /* for unknown streams */
# if CIO_WIN_WC_XLATE
PerStreamTags *ppst = getEmitStreamInfo(1, &pst, &pfOut);
# else
getEmitStreamInfo(1, &pst, &pfOut);
# endif
assert(z!=0);
# if CIO_WIN_WC_XLATE
if( pstReachesConsole(ppst) ) return conZstrEmit(ppst, z, (int)strlen(z));
else {
# endif
return (fputs(z, pfOut)<0)? 0 : (int)strlen(z);
# if CIO_WIN_WC_XLATE
}
# endif
}
#endif /* !defined(SQLITE_CIO_NO_TRANSLATE) */
#if !(defined(SQLITE_CIO_NO_UTF8SCAN) && defined(SQLITE_CIO_NO_TRANSLATE))
/* Skip over as much z[] input char sequence as is valid UTF-8,
** limited per nAccept char's or whole characters and containing
** no char cn such that ((1<<cn) & ccm)!=0. On return, the
** sequence z:return (inclusive:exclusive) is validated UTF-8.
** Limit: nAccept>=0 => char count, nAccept<0 => character
*/
SQLITE_INTERNAL_LINKAGE const char*
zSkipValidUtf8(const char *z, int nAccept, long ccm){
int ng = (nAccept<0)? -nAccept : 0;
const char *pcLimit = (nAccept>=0)? z+nAccept : 0;
assert(z!=0);
while( (pcLimit)? (z<pcLimit) : (ng-- != 0) ){
char c = *z;
if( (c & 0x80) == 0 ){
if( ccm != 0L && c < 0x20 && ((1L<<c) & ccm) != 0 ) return z;
++z; /* ASCII */
}else if( (c & 0xC0) != 0xC0 ) return z; /* not a lead byte */
else{
const char *zt = z+1; /* Got lead byte, look at trail bytes.*/
do{
if( pcLimit && zt >= pcLimit ) return z;
else{
char ct = *zt++;
if( ct==0 || (zt-z)>4 || (ct & 0xC0)!=0x80 ){
/* Trailing bytes are too few, too many, or invalid. */
return z;
}
}
} while( ((c <<= 1) & 0x40) == 0x40 ); /* Eat lead byte's count. */
z = zt;
}
}
return z;
}
#endif /*!(defined(SQLITE_CIO_NO_UTF8SCAN)&&defined(SQLITE_CIO_NO_TRANSLATE))*/
#ifndef SQLITE_CIO_NO_TRANSLATE
# ifdef CONSIO_SPUTB
SQLITE_INTERNAL_LINKAGE int
fPutbUtf8(FILE *pfO, const char *cBuf, int nAccept){
assert(pfO!=0);
# if CIO_WIN_WC_XLATE
PerStreamTags pst = PST_INITIALIZER; /* for unknown streams */
PerStreamTags *ppst = getEmitStreamInfo(0, &pst, &pfO);
if( pstReachesConsole(ppst) ){
int rv;
maybeSetupAsConsole(ppst, 1);
rv = conZstrEmit(ppst, cBuf, nAccept);
if( 0 == isKnownWritable(ppst->pf) ) restoreConsoleArb(ppst);
return rv;
}else {
# endif
return (int)fwrite(cBuf, 1, nAccept, pfO);
# if CIO_WIN_WC_XLATE
}
# endif
}
# endif
SQLITE_INTERNAL_LINKAGE int
oPutbUtf8(const char *cBuf, int nAccept){
FILE *pfOut;
PerStreamTags pst = PST_INITIALIZER; /* for unknown streams */
# if CIO_WIN_WC_XLATE
PerStreamTags *ppst = getEmitStreamInfo(1, &pst, &pfOut);
# else
getEmitStreamInfo(1, &pst, &pfOut);
# endif
# if CIO_WIN_WC_XLATE
if( pstReachesConsole(ppst) ){
return conZstrEmit(ppst, cBuf, nAccept);
}else {
# endif
return (int)fwrite(cBuf, 1, nAccept, pfOut);
# if CIO_WIN_WC_XLATE
}
# endif
}
/*
** Flush the given output stream. Return non-zero for success, else 0.
*/
#if !defined(SQLITE_CIO_NO_FLUSH) && !defined(SQLITE_CIO_NO_SETMODE)
SQLITE_INTERNAL_LINKAGE int
fFlushBuffer(FILE *pfOut){
# if CIO_WIN_WC_XLATE && !defined(SHELL_OMIT_FIO_DUPE)
return FlushFileBuffers(handleOfFile(pfOut))? 1 : 0;
# else
return fflush(pfOut);
# endif
}
#endif
#if CIO_WIN_WC_XLATE \
&& !defined(SHELL_OMIT_FIO_DUPE) \
&& defined(SQLITE_USE_ONLY_WIN32)
static struct FileAltIds {
int fd;
HANDLE fh;
} altIdsOfFile(FILE *pf){
struct FileAltIds rv = { _fileno(pf) };
union { intptr_t osfh; HANDLE fh; } fid = {
(rv.fd>=0)? _get_osfhandle(rv.fd) : (intptr_t)INVALID_HANDLE_VALUE
};
rv.fh = fid.fh;
return rv;
}
SQLITE_INTERNAL_LINKAGE size_t
cfWrite(const void *buf, size_t osz, size_t ocnt, FILE *pf){
size_t rv = 0;
struct FileAltIds fai = altIdsOfFile(pf);
int fmode = _setmode(fai.fd, _O_BINARY);
_setmode(fai.fd, fmode);
while( rv < ocnt ){
size_t nbo = osz;
while( nbo > 0 ){
DWORD dwno = (nbo>(1L<<24))? 1L<<24 : (DWORD)nbo;
BOOL wrc = TRUE;
BOOL genCR = (fmode & _O_TEXT)!=0;
if( genCR ){
const char *pnl = (const char*)memchr(buf, '\n', nbo);
if( pnl ) nbo = pnl - (const char*)buf;
else genCR = 0;
}
if( dwno>0 ) wrc = WriteFile(fai.fh, buf, dwno, 0,0);
if( genCR && wrc ){
wrc = WriteFile(fai.fh, "\r\n", 2, 0,0);
++dwno; /* Skip over the LF */
}
if( !wrc ) return rv;
buf = (const char*)buf + dwno;
nbo += dwno;
}
++rv;
}
return rv;
}
/* An fgets() equivalent, using Win32 file API for actual input.
** Input ends when given buffer is filled or a newline is read.
** If the FILE object is in text mode, swallows 0x0D. (ASCII CR)
*/
SQLITE_INTERNAL_LINKAGE char *
cfGets(char *cBuf, int n, FILE *pf){
int nci = 0;
struct FileAltIds fai = altIdsOfFile(pf);
int fmode = _setmode(fai.fd, _O_BINARY);
BOOL eatCR = (fmode & _O_TEXT)!=0;
_setmode(fai.fd, fmode);
while( nci < n-1 ){
DWORD nr;
if( !ReadFile(fai.fh, cBuf+nci, 1, &nr, 0) || nr==0 ) break;
if( nr>0 && (!eatCR || cBuf[nci]!='\r') ){
nci += nr;
if( cBuf[nci-nr]=='\n' ) break;
}
}
if( nci < n ) cBuf[nci] = 0;
return (nci>0)? cBuf : 0;
}
# else
# define cfWrite(b,os,no,f) fwrite(b,os,no,f)
# define cfGets(b,n,f) fgets(b,n,f)
# endif
# ifdef CONSIO_EPUTB
SQLITE_INTERNAL_LINKAGE int
ePutbUtf8(const char *cBuf, int nAccept){
FILE *pfErr;
PerStreamTags pst = PST_INITIALIZER; /* for unknown streams */
PerStreamTags *ppst = getEmitStreamInfo(2, &pst, &pfErr);
# if CIO_WIN_WC_XLATE
if( pstReachesConsole(ppst) ){
return conZstrEmit(ppst, cBuf, nAccept);
}else {
# endif
return (int)cfWrite(cBuf, 1, nAccept, pfErr);
# if CIO_WIN_WC_XLATE
}
# endif
}
# endif /* defined(CONSIO_EPUTB) */
SQLITE_INTERNAL_LINKAGE char* fGetsUtf8(char *cBuf, int ncMax, FILE *pfIn){
if( pfIn==0 ) pfIn = stdin;
# if CIO_WIN_WC_XLATE
if( pfIn == consoleInfo.pstSetup[0].pf
&& (consoleInfo.sacSetup & SAC_InConsole)!=0 ){
# if CIO_WIN_WC_XLATE==1
# define SHELL_GULP 150 /* Count of WCHARS to be gulped at a time */
WCHAR wcBuf[SHELL_GULP+1];
int lend = 0, noc = 0;
if( ncMax > 0 ) cBuf[0] = 0;
while( noc < ncMax-8-1 && !lend ){
/* There is room for at least 2 more characters and a 0-terminator. */
int na = (ncMax > SHELL_GULP*4+1 + noc)? SHELL_GULP : (ncMax-1 - noc)/4;
# undef SHELL_GULP
DWORD nbr = 0;
BOOL bRC = ReadConsoleW(consoleInfo.pstSetup[0].hx, wcBuf, na, &nbr, 0);
if( bRC && nbr>0 && (wcBuf[nbr-1]&0xF800)==0xD800 ){
/* Last WHAR read is first of a UTF-16 surrogate pair. Grab its mate. */
DWORD nbrx;
bRC &= ReadConsoleW(consoleInfo.pstSetup[0].hx, wcBuf+nbr, 1, &nbrx, 0);
if( bRC ) nbr += nbrx;
}
if( !bRC || (noc==0 && nbr==0) ) return 0;
if( nbr > 0 ){
int nmb = WideCharToMultiByte(CP_UTF8, 0, wcBuf,nbr,0,0,0,0);
if( nmb != 0 && noc+nmb <= ncMax ){
int iseg = noc;
nmb = WideCharToMultiByte(CP_UTF8, 0, wcBuf,nbr,cBuf+noc,nmb,0,0);
noc += nmb;
/* Fixup line-ends as coded by Windows for CR (or "Enter".)
** This is done without regard for any setMode{Text,Binary}()
** call that might have been done on the interactive input.
*/
if( noc > 0 ){
if( cBuf[noc-1]=='\n' ){
lend = 1;
if( noc > 1 && cBuf[noc-2]=='\r' ) cBuf[--noc-1] = '\n';
}
}
/* Check for ^Z (anywhere in line) too, to act as EOF. */
while( iseg < noc ){
if( cBuf[iseg]=='\x1a' ){
noc = iseg; /* Chop ^Z and anything following. */
lend = 1; /* Counts as end of line too. */
break;
}
++iseg;
}
}else break; /* Drop apparent garbage in. (Could assert.) */
}else break;
}
/* If got nothing, (after ^Z chop), must be at end-of-file. */
if( noc > 0 ){
cBuf[noc] = 0;
return cBuf;
}else return 0;
# endif
}else{
# endif
return cfGets(cBuf, ncMax, pfIn);
# if CIO_WIN_WC_XLATE
}
# endif
}
#endif /* !defined(SQLITE_CIO_NO_TRANSLATE) */
#if defined(_MSC_VER)
# pragma warning(default : 4204)
#endif
#undef SHELL_INVALID_FILE_PTR
+287
View File
@@ -0,0 +1,287 @@
/*
** 2023 November 1
**
** The author disclaims copyright to this source code. In place of
** a legal notice, here is a blessing:
**
** May you do good and not evil.
** May you find forgiveness for yourself and forgive others.
** May you share freely, never taking more than you give.
**
********************************************************************************
** This file exposes various interfaces used for console and other I/O
** by the SQLite project command-line tools. These interfaces are used
** at either source conglomeration time, compilation time, or run time.
** This source provides for either inclusion into conglomerated,
** "single-source" forms or separate compilation then linking.
**
** Platform dependencies are "hidden" here by various stratagems so
** that, provided certain conditions are met, the programs using this
** source or object code compiled from it need no explicit conditional
** compilation in their source for their console and stream I/O.
**
** The symbols and functionality exposed here are not a public API.
** This code may change in tandem with other project code as needed.
**
** When this .h file and its companion .c are directly incorporated into
** a source conglomeration (such as shell.c), the preprocessor symbol
** CIO_WIN_WC_XLATE is defined as 0 or 1, reflecting whether console I/O
** translation for Windows is effected for the build.
*/
#define HAVE_CONSOLE_IO_H 1
#ifndef SQLITE_INTERNAL_LINKAGE
# define SQLITE_INTERNAL_LINKAGE extern /* external to translation unit */
# include <stdio.h>
#else
# define SHELL_NO_SYSINC /* Better yet, modify mkshellc.tcl for this. */
#endif
#ifndef SQLITE3_H
# include "sqlite3.h"
#endif
#ifndef SQLITE_CIO_NO_CLASSIFY
/* Define enum for use with following function. */
typedef enum StreamsAreConsole {
SAC_NoConsole = 0,
SAC_InConsole = 1, SAC_OutConsole = 2, SAC_ErrConsole = 4,
SAC_AnyConsole = 0x7
} StreamsAreConsole;
/*
** Classify the three standard I/O streams according to whether
** they are connected to a console attached to the process.
**
** Returns the bit-wise OR of SAC_{In,Out,Err}Console values,
** or SAC_NoConsole if none of the streams reaches a console.
**
** This function should be called before any I/O is done with
** the given streams. As a side-effect, the given inputs are
** recorded so that later I/O operations on them may be done
** differently than the C library FILE* I/O would be done,
** iff the stream is used for the I/O functions that follow,
** and to support the ones that use an implicit stream.
**
** On some platforms, stream or console mode alteration (aka
** "Setup") may be made which is undone by consoleRestore().
*/
SQLITE_INTERNAL_LINKAGE StreamsAreConsole
consoleClassifySetup( FILE *pfIn, FILE *pfOut, FILE *pfErr );
/* A usual call for convenience: */
#define SQLITE_STD_CONSOLE_INIT() consoleClassifySetup(stdin,stdout,stderr)
/*
** After an initial call to consoleClassifySetup(...), renew
** the same setup it effected. (A call not after is an error.)
** This will restore state altered by consoleRestore();
**
** Applications which run an inferior (child) process which
** inherits the same I/O streams may call this function after
** such a process exits to guard against console mode changes.
*/
SQLITE_INTERNAL_LINKAGE void consoleRenewSetup(void);
/*
** Undo any side-effects left by consoleClassifySetup(...).
**
** This should be called after consoleClassifySetup() and
** before the process terminates normally. It is suitable
** for use with the atexit() C library procedure. After
** this call, no console I/O should be done until one of
** console{Classify or Renew}Setup(...) is called again.
**
** Applications which run an inferior (child) process that
** inherits the same I/O streams might call this procedure
** before so that said process will have a console setup
** however users have configured it or come to expect.
*/
SQLITE_INTERNAL_LINKAGE void SQLITE_CDECL consoleRestore( void );
#else /* defined(SQLITE_CIO_NO_CLASSIFY) */
# define consoleClassifySetup(i,o,e)
# define consoleRenewSetup()
# define consoleRestore()
#endif /* defined(SQLITE_CIO_NO_CLASSIFY) */
#ifndef SQLITE_CIO_NO_REDIRECT
/*
** Set stream to be used for the functions below which write
** to "the designated X stream", where X is Output or Error.
** Returns the previous value.
**
** Alternatively, pass the special value, invalidFileStream,
** to get the designated stream value without setting it.
**
** Before the designated streams are set, they default to
** those passed to consoleClassifySetup(...), and before
** that is called they default to stdout and stderr.
**
** It is error to close a stream so designated, then, without
** designating another, use the corresponding {o,e}Emit(...).
*/
SQLITE_INTERNAL_LINKAGE FILE *invalidFileStream;
SQLITE_INTERNAL_LINKAGE FILE *setOutputStream(FILE *pf);
# ifdef CONSIO_SET_ERROR_STREAM
SQLITE_INTERNAL_LINKAGE FILE *setErrorStream(FILE *pf);
# endif
#else
# define setOutputStream(pf)
# define setErrorStream(pf)
#endif /* !defined(SQLITE_CIO_NO_REDIRECT) */
#ifndef SQLITE_CIO_NO_TRANSLATE
/*
** Emit output like fprintf(). If the output is going to the
** console and translation from UTF-8 is necessary, perform
** the needed translation. Otherwise, write formatted output
** to the provided stream almost as-is, possibly with newline
** translation as specified by set{Binary,Text}Mode().
*/
SQLITE_INTERNAL_LINKAGE int fPrintfUtf8(FILE *pfO, const char *zFormat, ...);
/* Like fPrintfUtf8 except stream is always the designated output. */
SQLITE_INTERNAL_LINKAGE int oPrintfUtf8(const char *zFormat, ...);
/* Like fPrintfUtf8 except stream is always the designated error. */
SQLITE_INTERNAL_LINKAGE int ePrintfUtf8(const char *zFormat, ...);
/*
** Emit output like fputs(). If the output is going to the
** console and translation from UTF-8 is necessary, perform
** the needed translation. Otherwise, write given text to the
** provided stream almost as-is, possibly with newline
** translation as specified by set{Binary,Text}Mode().
*/
SQLITE_INTERNAL_LINKAGE int fPutsUtf8(const char *z, FILE *pfO);
/* Like fPutsUtf8 except stream is always the designated output. */
SQLITE_INTERNAL_LINKAGE int oPutsUtf8(const char *z);
/* Like fPutsUtf8 except stream is always the designated error. */
SQLITE_INTERNAL_LINKAGE int ePutsUtf8(const char *z);
/*
** Emit output like fPutsUtf8(), except that the length of the
** accepted char or character sequence is limited by nAccept.
**
** Returns the number of accepted char values.
*/
#ifdef CONSIO_SPUTB
SQLITE_INTERNAL_LINKAGE int
fPutbUtf8(FILE *pfOut, const char *cBuf, int nAccept);
/* Like fPutbUtf8 except stream is always the designated output. */
#endif
SQLITE_INTERNAL_LINKAGE int
oPutbUtf8(const char *cBuf, int nAccept);
/* Like fPutbUtf8 except stream is always the designated error. */
#ifdef CONSIO_EPUTB
SQLITE_INTERNAL_LINKAGE int
ePutbUtf8(const char *cBuf, int nAccept);
#endif
/*
** Flush the given output stream. Return non-zero for success, else 0.
*/
#if !defined(SQLITE_CIO_NO_FLUSH) && !defined(SQLITE_CIO_NO_SETMODE)
SQLITE_INTERNAL_LINKAGE int
fFlushBuffer(FILE *pfOut);
#endif
/*
** Collect input like fgets(...) with special provisions for input
** from the console on such platforms as require same. Newline
** translation may be done as set by set{Binary,Text}Mode().
** As a convenience, pfIn==NULL is treated as stdin.
*/
SQLITE_INTERNAL_LINKAGE char* fGetsUtf8(char *cBuf, int ncMax, FILE *pfIn);
/* Like fGetsUtf8 except stream is always the designated input. */
/* SQLITE_INTERNAL_LINKAGE char* iGetsUtf8(char *cBuf, int ncMax); */
#endif /* !defined(SQLITE_CIO_NO_TRANSLATE) */
#ifndef SQLITE_CIO_NO_SETMODE
/*
** Set given stream for binary mode, where newline translation is
** not done, or for text mode where, for some platforms, newlines
** are translated to the platform's conventional char sequence.
** If bFlush true, flush the stream.
**
** An additional side-effect is that if the stream is one passed
** to consoleClassifySetup() as an output, it is flushed first.
**
** Note that binary/text mode has no effect on console I/O
** translation. On all platforms, newline to the console starts
** a new line and CR,LF chars from the console become a newline.
*/
SQLITE_INTERNAL_LINKAGE void setBinaryMode(FILE *, short bFlush);
SQLITE_INTERNAL_LINKAGE void setTextMode(FILE *, short bFlush);
#endif
#ifdef SQLITE_CIO_PROMPTED_IN
typedef struct Prompts {
int numPrompts;
const char **azPrompts;
} Prompts;
/*
** Macros for use of a line editor.
**
** The following macros define operations involving use of a
** line-editing library or simple console interaction.
** A "T" argument is a text (char *) buffer or filename.
** A "N" argument is an integer.
**
** SHELL_ADD_HISTORY(T) // Record text as line(s) of history.
** SHELL_READ_HISTORY(T) // Read history from file named by T.
** SHELL_WRITE_HISTORY(T) // Write history to file named by T.
** SHELL_STIFLE_HISTORY(N) // Limit history to N entries.
**
** A console program which does interactive console input is
** expected to call:
** SHELL_READ_HISTORY(T) before collecting such input;
** SHELL_ADD_HISTORY(T) as record-worthy input is taken;
** SHELL_STIFLE_HISTORY(N) after console input ceases; then
** SHELL_WRITE_HISTORY(T) before the program exits.
*/
/*
** Retrieve a single line of input text from an input stream.
**
** If pfIn is the input stream passed to consoleClassifySetup(),
** and azPrompt is not NULL, then a prompt is issued before the
** line is collected, as selected by the isContinuation flag.
** Array azPrompt[{0,1}] holds the {main,continuation} prompt.
**
** If zBufPrior is not NULL then it is a buffer from a prior
** call to this routine that can be reused, or will be freed.
**
** The result is stored in space obtained from malloc() and
** must either be freed by the caller or else passed back to
** this function as zBufPrior for reuse.
**
** This function may call upon services of a line-editing
** library to interactively collect line edited input.
*/
SQLITE_INTERNAL_LINKAGE char *
shellGetLine(FILE *pfIn, char *zBufPrior, int nLen,
short isContinuation, Prompts azPrompt);
#endif /* defined(SQLITE_CIO_PROMPTED_IN) */
/*
** TBD: Define an interface for application(s) to generate
** completion candidates for use by the line-editor.
**
** This may be premature; the CLI is the only application
** that does this. Yet, getting line-editing melded into
** console I/O is desirable because a line-editing library
** may have to establish console operating mode, possibly
** in a way that interferes with the above functionality.
*/
#if !(defined(SQLITE_CIO_NO_UTF8SCAN)&&defined(SQLITE_CIO_NO_TRANSLATE))
/* Skip over as much z[] input char sequence as is valid UTF-8,
** limited per nAccept char's or whole characters and containing
** no char cn such that ((1<<cn) & ccm)!=0. On return, the
** sequence z:return (inclusive:exclusive) is validated UTF-8.
** Limit: nAccept>=0 => char count, nAccept<0 => character
*/
SQLITE_INTERNAL_LINKAGE const char*
zSkipValidUtf8(const char *z, int nAccept, long ccm);
#endif
+53
View File
@@ -8,6 +8,7 @@
# May you share freely, never taking more than you give.
#
#***********************************************************************
# TESTRUNNER: shell
#
# The focus of this file is testing the CLI shell tool. Specifically,
# the ".recommend" command.
@@ -391,6 +392,16 @@ do_setup_rec_test $tn.18.1 {
SEARCH SomeObject USING COVERING INDEX SomeObject_idx_00000078 (x=?)
}
do_setup_rec_test $tn.19.0 {
CREATE TABLE t1("index");
} {
SELECT * FROM t1 ORDER BY "index";
} {
CREATE INDEX t1_idx_01a7214e ON t1('index');
SCAN t1 USING COVERING INDEX t1_idx_01a7214e
}
}
proc do_candidates_test {tn sql res} {
@@ -417,6 +428,8 @@ do_execsql_test 5.0 {
WITH s(i) AS ( VALUES(1) UNION ALL SELECT i+1 FROM s WHERE i<100)
INSERT INTO t2 SELECT (i-1)/20, (i-1)/5 FROM s;
CREATE INDEX i1 ON t1( lower(a) );
}
do_candidates_test 5.1 {
SELECT * FROM t1,t2 WHERE (b=? OR a=?) AND (c=? OR d=?)
@@ -446,6 +459,7 @@ do_execsql_test 5.3 {
ANALYZE;
SELECT * FROM sqlite_stat1 ORDER BY 1, 2;
} {
t1 i1 {100 50}
t1 t1_idx_00000061 {100 50}
t1 t1_idx_00000062 {100 20}
t1 t1_idx_000123a7 {100 50 17}
@@ -454,4 +468,43 @@ do_execsql_test 5.3 {
t2 t2_idx_0001295b {100 20 5}
}
do_catchsql_test 5.4 {
SELECT sqlite_expert_rem(123, 123);
} {1 {no such function: sqlite_expert_rem}}
do_catchsql_test 5.5 {
SELECT sqlite_expert_sample();
} {1 {no such function: sqlite_expert_sample}}
if 0 {
do_test expert1-6.0 {
catchcmd :memory: {
.expert
select base64('');
.expert
select name from pragma_collation_list order by name collate uint;
}
} {0 {(no new indexes)
SCAN CONSTANT ROW
(no new indexes)
SCAN pragma_collation_list VIRTUAL TABLE INDEX 0:
USE TEMP B-TREE FOR ORDER BY
}}
}
do_execsql_test 6.0 {
CREATE TABLE x1(a, b, c, d);
CREATE INDEX x1ab ON x1(a, lower(b));
CREATE INDEX x1dcba ON x1(d, b+c, a);
}
do_candidates_test 6.1 {
SELECT * FROM x1 WHERE b=? ORDER BY a;
} {
CREATE INDEX x1_idx_0001267f ON x1(b, a);
CREATE INDEX x1_idx_00000062 ON x1(b);
}
finish_test
+126 -11
View File
@@ -32,7 +32,7 @@
#endif /* !defined(SQLITE_AMALGAMATION) */
#ifndef SQLITE_OMIT_VIRTUALTABLE
#ifndef SQLITE_OMIT_VIRTUALTABLE
typedef sqlite3_int64 i64;
typedef sqlite3_uint64 u64;
@@ -626,7 +626,7 @@ static int expertFilter(
pCsr->pData = 0;
if( rc==SQLITE_OK ){
rc = idxPrintfPrepareStmt(pExpert->db, &pCsr->pData, &pVtab->base.zErrMsg,
"SELECT * FROM main.%Q WHERE sample()", pVtab->pTab->zName
"SELECT * FROM main.%Q WHERE sqlite_expert_sample()", pVtab->pTab->zName
);
}
@@ -662,6 +662,7 @@ static int idxRegisterVtab(sqlite3expert *p){
0, /* xRelease */
0, /* xRollbackTo */
0, /* xShadowName */
0, /* xIntegrity */
};
return sqlite3_create_module(p->dbv, "expert", &expertModule, (void*)p);
@@ -820,6 +821,10 @@ static char *idxAppendText(int *pRc, char *zIn, const char *zFmt, ...){
*/
static int idxIdentifierRequiresQuotes(const char *zId){
int i;
int nId = STRLEN(zId);
if( sqlite3_keyword_check(zId, nId) ) return 1;
for(i=0; zId[i]; i++){
if( !(zId[i]=='_')
&& !(zId[i]>='0' && zId[i]<='9')
@@ -1495,7 +1500,7 @@ struct IdxRemCtx {
};
/*
** Implementation of scalar function rem().
** Implementation of scalar function sqlite_expert_rem().
*/
static void idxRemFunc(
sqlite3_context *pCtx,
@@ -1508,7 +1513,7 @@ static void idxRemFunc(
assert( argc==2 );
iSlot = sqlite3_value_int(argv[0]);
assert( iSlot<=p->nSlot );
assert( iSlot<p->nSlot );
pSlot = &p->aSlot[iSlot];
switch( pSlot->eType ){
@@ -1618,8 +1623,15 @@ static int idxPopulateOneStat1(
const char *zComma = zCols==0 ? "" : ", ";
const char *zName = (const char*)sqlite3_column_text(pIndexXInfo, 0);
const char *zColl = (const char*)sqlite3_column_text(pIndexXInfo, 1);
if( zName==0 ){
/* This index contains an expression. Ignore it. */
sqlite3_free(zCols);
sqlite3_free(zOrder);
return sqlite3_reset(pIndexXInfo);
}
zCols = idxAppendText(&rc, zCols,
"%sx.%Q IS rem(%d, x.%Q) COLLATE %s", zComma, zName, nCol, zName, zColl
"%sx.%Q IS sqlite_expert_rem(%d, x.%Q) COLLATE %s",
zComma, zName, nCol, zName, zColl
);
zOrder = idxAppendText(&rc, zOrder, "%s%d", zComma, ++nCol);
}
@@ -1752,13 +1764,13 @@ static int idxPopulateStat1(sqlite3expert *p, char **pzErr){
if( rc==SQLITE_OK ){
sqlite3 *dbrem = (p->iSample==100 ? p->db : p->dbv);
rc = sqlite3_create_function(
dbrem, "rem", 2, SQLITE_UTF8, (void*)pCtx, idxRemFunc, 0, 0
rc = sqlite3_create_function(dbrem, "sqlite_expert_rem",
2, SQLITE_UTF8, (void*)pCtx, idxRemFunc, 0, 0
);
}
if( rc==SQLITE_OK ){
rc = sqlite3_create_function(
p->db, "sample", 0, SQLITE_UTF8, (void*)&samplectx, idxSampleFunc, 0, 0
rc = sqlite3_create_function(p->db, "sqlite_expert_sample",
0, SQLITE_UTF8, (void*)&samplectx, idxSampleFunc, 0, 0
);
}
@@ -1810,10 +1822,95 @@ static int idxPopulateStat1(sqlite3expert *p, char **pzErr){
rc = sqlite3_exec(p->dbm, "ANALYZE sqlite_schema", 0, 0, 0);
}
sqlite3_create_function(p->db, "sqlite_expert_rem", 2, SQLITE_UTF8, 0,0,0,0);
sqlite3_create_function(p->db, "sqlite_expert_sample", 0,SQLITE_UTF8,0,0,0,0);
sqlite3_exec(p->db, "DROP TABLE IF EXISTS temp."UNIQUE_TABLE_NAME,0,0,0);
return rc;
}
/*
** Define and possibly pretend to use a useless collation sequence.
** This pretense allows expert to accept SQL using custom collations.
*/
int dummyCompare(void *up1, int up2, const void *up3, int up4, const void *up5){
(void)up1;
(void)up2;
(void)up3;
(void)up4;
(void)up5;
assert(0); /* VDBE should never be run. */
return 0;
}
/* And a callback to register above upon actual need */
void useDummyCS(void *up1, sqlite3 *db, int etr, const char *zName){
(void)up1;
sqlite3_create_collation_v2(db, zName, etr, 0, dummyCompare, 0);
}
#if !defined(SQLITE_OMIT_SCHEMA_PRAGMAS) \
&& !defined(SQLITE_OMIT_INTROSPECTION_PRAGMAS)
/*
** dummy functions for no-op implementation of UDFs during expert's work
*/
void dummyUDF(sqlite3_context *up1, int up2, sqlite3_value **up3){
(void)up1;
(void)up2;
(void)up3;
assert(0); /* VDBE should never be run. */
}
void dummyUDFvalue(sqlite3_context *up1){
(void)up1;
assert(0); /* VDBE should never be run. */
}
/*
** Register UDFs from user database with another.
*/
int registerUDFs(sqlite3 *dbSrc, sqlite3 *dbDst){
sqlite3_stmt *pStmt;
int rc = sqlite3_prepare_v2(dbSrc,
"SELECT name,type,enc,narg,flags "
"FROM pragma_function_list() "
"WHERE builtin==0", -1, &pStmt, 0);
if( rc==SQLITE_OK ){
while( SQLITE_ROW==(rc = sqlite3_step(pStmt)) ){
int nargs = sqlite3_column_int(pStmt,3);
int flags = sqlite3_column_int(pStmt,4);
const char *name = (char*)sqlite3_column_text(pStmt,0);
const char *type = (char*)sqlite3_column_text(pStmt,1);
const char *enc = (char*)sqlite3_column_text(pStmt,2);
if( name==0 || type==0 || enc==0 ){
/* no-op. Only happens on OOM */
}else{
int ienc = SQLITE_UTF8;
int rcf = SQLITE_ERROR;
if( strcmp(enc,"utf16le")==0 ) ienc = SQLITE_UTF16LE;
else if( strcmp(enc,"utf16be")==0 ) ienc = SQLITE_UTF16BE;
ienc |= (flags & (SQLITE_DETERMINISTIC|SQLITE_DIRECTONLY));
if( strcmp(type,"w")==0 ){
rcf = sqlite3_create_window_function(dbDst,name,nargs,ienc,0,
dummyUDF,dummyUDFvalue,0,0,0);
}else if( strcmp(type,"a")==0 ){
rcf = sqlite3_create_function(dbDst,name,nargs,ienc,0,
0,dummyUDF,dummyUDFvalue);
}else if( strcmp(type,"s")==0 ){
rcf = sqlite3_create_function(dbDst,name,nargs,ienc,0,
dummyUDF,0,0);
}
if( rcf!=SQLITE_OK ){
rc = rcf;
break;
}
}
}
sqlite3_finalize(pStmt);
if( rc==SQLITE_DONE ) rc = SQLITE_OK;
}
return rc;
}
#endif
/*
** Allocate a new sqlite3expert object.
*/
@@ -1840,14 +1937,28 @@ sqlite3expert *sqlite3_expert_new(sqlite3 *db, char **pzErrmsg){
sqlite3_db_config(pNew->dbm, SQLITE_DBCONFIG_TRIGGER_EQP, 1, (int*)0);
}
}
/* Allow custom collations to be dealt with through prepare. */
if( rc==SQLITE_OK ) rc = sqlite3_collation_needed(pNew->dbm,0,useDummyCS);
if( rc==SQLITE_OK ) rc = sqlite3_collation_needed(pNew->dbv,0,useDummyCS);
#if !defined(SQLITE_OMIT_SCHEMA_PRAGMAS) \
&& !defined(SQLITE_OMIT_INTROSPECTION_PRAGMAS)
/* Register UDFs from database [db] with [dbm] and [dbv]. */
if( rc==SQLITE_OK ){
rc = registerUDFs(pNew->db, pNew->dbm);
}
if( rc==SQLITE_OK ){
rc = registerUDFs(pNew->db, pNew->dbv);
}
#endif
/* Copy the entire schema of database [db] into [dbm]. */
if( rc==SQLITE_OK ){
sqlite3_stmt *pSql = 0;
rc = idxPrintfPrepareStmt(pNew->db, &pSql, pzErrmsg,
"SELECT sql FROM sqlite_schema WHERE name NOT LIKE 'sqlite_%%'"
" AND sql NOT LIKE 'CREATE VIRTUAL %%'"
" AND sql NOT LIKE 'CREATE VIRTUAL %%' ORDER BY rowid"
);
while( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pSql) ){
const char *zSql = (const char*)sqlite3_column_text(pSql, 0);
@@ -1916,6 +2027,10 @@ int sqlite3_expert_sql(
while( rc==SQLITE_OK && zStmt && zStmt[0] ){
sqlite3_stmt *pStmt = 0;
/* Ensure that the provided statement compiles against user's DB. */
rc = idxPrepareStmt(p->db, &pStmt, pzErr, zStmt);
if( rc!=SQLITE_OK ) break;
sqlite3_finalize(pStmt);
rc = sqlite3_prepare_v2(p->dbv, zStmt, -1, &pStmt, &zStmt);
if( rc==SQLITE_OK ){
if( pStmt ){
+1 -9
View File
@@ -16,15 +16,7 @@
#include "sqlite3expert.h"
#include <assert.h>
#include <string.h>
#if defined(INCLUDE_SQLITE_TCL_H)
# include "sqlite_tcl.h"
#else
# include "tcl.h"
# ifndef SQLITE_TCLAPI
# define SQLITE_TCLAPI
# endif
#endif
#include "tclsqlite.h"
#ifndef SQLITE_OMIT_VIRTUALTABLE
-14
View File
@@ -1,14 +0,0 @@
ALLOC_NORMAL
FS
UTF8ToString
addFunction
allocate
allocateUTF8OnStack
ccall
cwrap
getValue
removeFunction
setValue
stackAlloc
stackRestore
stackSave
-34
View File
@@ -1,34 +0,0 @@
# This GNU makefile exists primarily to simplify/speed up development
# from emacs. It is not part of the canonical build process.
default:
$(MAKE) -C ../.. wasm -e emcc_opt=-O0
clean:
$(MAKE) -C ../../ clean-wasm
fiddle_files = emscripten.css fiddle.html \
fiddle.js fiddle-module.js \
fiddle-module.wasm fiddle-worker.js \
$(wildcard *.wasm.gz) $(wildcard *.js.gz)
# fiddle_remote is the remote destination for the fiddle app. It
# must be a [user@]HOST:/path for rsync.
# Note that the target "should probably" contain a symlink of
# index.html -> fiddle.html.
fiddle_remote ?=
ifeq (,$(fiddle_remote))
ifneq (,$(wildcard /home/stephan))
fiddle_remote = wh2:www/wh/sqlite3/.
else ifneq (,$(wildcard /home/drh))
#fiddle_remote = if appropriate, add that user@host:/path here
endif
endif
$(fiddle_files): default
push-fiddle: $(fiddle_files)
@if [ x = "x$(fiddle_remote)" ]; then \
echo "fiddle_remote must be a [user@]HOST:/path for rsync"; \
exit 1; \
fi
rsync -va $(fiddle_files) $(fiddle_remote)
-144
View File
@@ -1,144 +0,0 @@
/*
2022-05-22
The author disclaims copyright to this source code. In place of a
legal notice, here is a blessing:
* May you do good and not evil.
* May you find forgiveness for yourself and forgive others.
* May you share freely, never taking more than you give.
***********************************************************************
This file contains bootstrapping code used by various test scripts
which live in this file's directory.
*/
(function(){
/* querySelectorAll() proxy */
const EAll = function(/*[element=document,] cssSelector*/){
return (arguments.length>1 ? arguments[0] : document)
.querySelectorAll(arguments[arguments.length-1]);
};
/* querySelector() proxy */
const E = function(/*[element=document,] cssSelector*/){
return (arguments.length>1 ? arguments[0] : document)
.querySelector(arguments[arguments.length-1]);
};
/**
Helpers for writing sqlite3-specific tests.
*/
self/*window or worker*/.SqliteTestUtil = {
/** Running total of the number of tests run via
this API. */
counter: 0,
/**
If expr is a function, it is called and its result
is returned, coerced to a bool, else expr, coerced to
a bool, is returned.
*/
toBool: function(expr){
return (expr instanceof Function) ? !!expr() : !!expr;
},
/** abort() if expr is false. If expr is a function, it
is called and its result is evaluated.
*/
assert: function f(expr, msg){
if(!f._){
f._ = ('undefined'===typeof abort
? (msg)=>{throw new Error(msg)}
: abort);
}
++this.counter;
if(!this.toBool(expr)){
f._(msg || "Assertion failed.");
}
return this;
},
/** Identical to assert() but throws instead of calling
abort(). */
affirm: function(expr, msg){
++this.counter;
if(!this.toBool(expr)) throw new Error(msg || "Affirmation failed.");
return this;
},
/** Calls f() and squelches any exception it throws. If it
does not throw, this function throws. */
mustThrow: function(f, msg){
++this.counter;
let err;
try{ f(); } catch(e){err=e;}
if(!err) throw new Error(msg || "Expected exception.");
return this;
},
/** Throws if expr is truthy or expr is a function and expr()
returns truthy. */
throwIf: function(expr, msg){
++this.counter;
if(this.toBool(expr)) throw new Error(msg || "throwIf() failed");
return this;
},
/** Throws if expr is falsy or expr is a function and expr()
returns falsy. */
throwUnless: function(expr, msg){
++this.counter;
if(!this.toBool(expr)) throw new Error(msg || "throwUnless() failed");
return this;
}
};
/**
This is a module object for use with the emscripten-installed
initSqlite3Module() factory function.
*/
self.sqlite3TestModule = {
postRun: [
/* function(theModule){...} */
],
//onRuntimeInitialized: function(){},
/* Proxy for C-side stdout output. */
print: function(){
console.log.apply(console, Array.prototype.slice.call(arguments));
},
/* Proxy for C-side stderr output. */
printErr: function(){
console.error.apply(console, Array.prototype.slice.call(arguments));
},
/**
Called by the module init bits to report loading
progress. It gets passed an empty argument when loading is
done (after onRuntimeInitialized() and any this.postRun
callbacks have been run).
*/
setStatus: function f(text){
if(!f.last){
f.last = { text: '', step: 0 };
f.ui = {
status: E('#module-status'),
progress: E('#module-progress'),
spinner: E('#module-spinner')
};
}
if(text === f.last.text) return;
f.last.text = text;
if(f.ui.progress){
f.ui.progress.value = f.last.step;
f.ui.progress.max = f.last.step + 1;
}
++f.last.step;
if(text) {
f.ui.status.classList.remove('hidden');
f.ui.status.innerText = text;
}else{
if(f.ui.progress){
f.ui.progress.remove();
f.ui.spinner.remove();
delete f.ui.progress;
delete f.ui.spinner;
}
f.ui.status.classList.add('hidden');
}
}
};
})(self/*window or worker*/);
-301
View File
@@ -1,301 +0,0 @@
/*
2022-05-20
The author disclaims copyright to this source code. In place of a
legal notice, here is a blessing:
* May you do good and not evil.
* May you find forgiveness for yourself and forgive others.
* May you share freely, never taking more than you give.
***********************************************************************
This is the JS Worker file for the sqlite3 fiddle app. It loads the
sqlite3 wasm module and offers access to the db via the Worker
message-passing interface.
Forewarning: this API is still very much Under Construction and
subject to any number of changes as experience reveals what those
need to be.
Because we can have only a single message handler, as opposed to an
arbitrary number of discrete event listeners like with DOM elements,
we have to define a lower-level message API. Messages abstractly
look like:
{ type: string, data: type-specific value }
Where 'type' is used for dispatching and 'data' is a
'type'-dependent value.
The 'type' values expected by each side of the main/worker
connection vary. The types are described below but subject to
change at any time as this experiment evolves.
Workers-to-Main types
- stdout, stderr: indicate stdout/stderr output from the wasm
layer. The data property is the string of the output, noting
that the emscripten binding emits these one line at a time. Thus,
if a C-side puts() emits multiple lines in a single call, the JS
side will see that as multiple calls. Example:
{type:'stdout', data: 'Hi, world.'}
- module: Status text. This is intended to alert the main thread
about module loading status so that, e.g., the main thread can
update a progress widget and DTRT when the module is finished
loading and available for work. Status messages come in the form
{type:'module', data:{
type:'status',
data: {text:string|null, step:1-based-integer}
}
with an incrementing step value for each subsequent message. When
the module loading is complete, a message with a text value of
null is posted.
- working: data='start'|'end'. Indicates that work is about to be
sent to the module or has just completed. This can be used, e.g.,
to disable UI elements which should not be activated while work
is pending. Example:
{type:'working', data:'start'}
Main-to-Worker types:
- shellExec: data=text to execute as if it had been entered in the
sqlite3 CLI shell app (as opposed to sqlite3_exec()). This event
causes the worker to emit a 'working' event (data='start') before
it starts and a 'working' event (data='end') when it finished. If
called while work is currently being executed it emits stderr
message instead of doing actual work, as the underlying db cannot
handle concurrent tasks. Example:
{type:'shellExec', data: 'select * from sqlite_master'}
- More TBD as the higher-level db layer develops.
*/
/*
Apparent browser(s) bug: console messages emitted may be duplicated
in the console, even though they're provably only run once. See:
https://stackoverflow.com/questions/49659464
Noting that it happens in Firefox as well as Chrome. Harmless but
annoying.
*/
"use strict";
(function(){
/**
Posts a message in the form {type,data} unless passed more than 2
args, in which case it posts {type, data:[arg1...argN]}.
*/
const wMsg = function(type,data){
postMessage({
type,
data: arguments.length<3
? data
: Array.prototype.slice.call(arguments,1)
});
};
const stdout = function(){wMsg('stdout', Array.prototype.slice.call(arguments));};
const stderr = function(){wMsg('stderr', Array.prototype.slice.call(arguments));};
self.onerror = function(/*message, source, lineno, colno, error*/) {
const err = arguments[4];
if(err && 'ExitStatus'==err.name){
/* This is relevant for the sqlite3 shell binding but not the
lower-level binding. */
fiddleModule.isDead = true;
stderr("FATAL ERROR:", err.message);
stderr("Restarting the app requires reloading the page.");
wMsg('error', err);
}
console.error(err);
fiddleModule.setStatus('Exception thrown, see JavaScript console: '+err);
};
const Sqlite3Shell = {
/** Returns the name of the currently-opened db. */
dbFilename: function f(){
if(!f._) f._ = fiddleModule.cwrap('fiddle_db_filename', "string", ['string']);
return f._();
},
/**
Runs the given text through the shell as if it had been typed
in by a user. Fires a working/start event before it starts and
working/end event when it finishes.
*/
exec: function f(sql){
if(!f._) f._ = fiddleModule.cwrap('fiddle_exec', null, ['string']);
if(fiddleModule.isDead){
stderr("shell module has exit()ed. Cannot run SQL.");
return;
}
wMsg('working','start');
try {
if(f._running){
stderr('Cannot run multiple commands concurrently.');
}else{
f._running = true;
f._(sql);
}
} finally {
delete f._running;
wMsg('working','end');
}
},
resetDb: function f(){
if(!f._) f._ = fiddleModule.cwrap('fiddle_reset_db', null);
stdout("Resetting database.");
f._();
stdout("Reset",this.dbFilename());
},
/* Interrupt can't work: this Worker is tied up working, so won't get the
interrupt event which would be needed to perform the interrupt. */
interrupt: function f(){
if(!f._) f._ = fiddleModule.cwrap('fiddle_interrupt', null);
stdout("Requesting interrupt.");
f._();
}
};
self.onmessage = function f(ev){
ev = ev.data;
if(!f.cache){
f.cache = {
prevFilename: null
};
}
//console.debug("worker: onmessage.data",ev);
switch(ev.type){
case 'shellExec': Sqlite3Shell.exec(ev.data); return;
case 'db-reset': Sqlite3Shell.resetDb(); return;
case 'interrupt': Sqlite3Shell.interrupt(); return;
/** Triggers the export of the current db. Fires an
event in the form:
{type:'db-export',
data:{
filename: name of db,
buffer: contents of the db file (Uint8Array),
error: on error, a message string and no buffer property.
}
}
*/
case 'db-export': {
const fn = Sqlite3Shell.dbFilename();
stdout("Exporting",fn+".");
const fn2 = fn ? fn.split(/[/\\]/).pop() : null;
try{
if(!fn2) throw new Error("DB appears to be closed.");
wMsg('db-export',{
filename: fn2,
buffer: fiddleModule.FS.readFile(fn, {encoding:"binary"})
});
}catch(e){
/* Post a failure message so that UI elements disabled
during the export can be re-enabled. */
wMsg('db-export',{
filename: fn,
error: e.message
});
}
return;
}
case 'open': {
/* Expects: {
buffer: ArrayBuffer | Uint8Array,
filename: for logging/informational purposes only
} */
const opt = ev.data;
let buffer = opt.buffer;
if(buffer instanceof Uint8Array){
}else if(buffer instanceof ArrayBuffer){
buffer = new Uint8Array(buffer);
}else{
stderr("'open' expects {buffer:Uint8Array} containing an uploaded db.");
return;
}
const fn = (
opt.filename
? opt.filename.split(/[/\\]/).pop().replace('"','_')
: ("db-"+((Math.random() * 10000000) | 0)+
"-"+((Math.random() * 10000000) | 0)+".sqlite3")
);
/* We cannot delete the existing db file until the new one
is installed, which means that we risk overflowing our
quota (if any) by having both the previous and current
db briefly installed in the virtual filesystem. */
fiddleModule.FS.createDataFile("/", fn, buffer, true, true);
const oldName = Sqlite3Shell.dbFilename();
Sqlite3Shell.exec('.open "/'+fn+'"');
if(oldName && oldName !== fn){
try{fiddleModule.FS.unlink(oldName);}
catch(e){/*ignored*/}
}
stdout("Replaced DB with",fn+".");
return;
}
};
console.warn("Unknown fiddle-worker message type:",ev);
};
/**
emscripten module for use with build mode -sMODULARIZE.
*/
const fiddleModule = {
print: stdout,
printErr: stderr,
/**
Intercepts status updates from the emscripting module init
and fires worker events with a type of 'status' and a
payload of:
{
text: string | null, // null at end of load process
step: integer // starts at 1, increments 1 per call
}
We have no way of knowing in advance how many steps will
be processed/posted, so creating a "percentage done" view is
not really practical. One can be approximated by giving it a
current value of message.step and max value of message.step+1,
though.
When work is finished, a message with a text value of null is
submitted.
After a message with text==null is posted, the module may later
post messages about fatal problems, e.g. an exit() being
triggered, so it is recommended that UI elements for posting
status messages not be outright removed from the DOM when
text==null, and that they instead be hidden until/unless
text!=null.
*/
setStatus: function f(text){
if(!f.last) f.last = { step: 0, text: '' };
else if(text === f.last.text) return;
f.last.text = text;
wMsg('module',{
type:'status',
data:{step: ++f.last.step, text: text||null}
});
}
};
importScripts('fiddle-module.js');
/**
initFiddleModule() is installed via fiddle-module.js due to
building with:
emcc ... -sMODULARIZE=1 -sEXPORT_NAME=initFiddleModule
*/
initFiddleModule(fiddleModule).then(function(thisModule){
wMsg('fiddle-ready');
});
})();
-809
View File
@@ -1,809 +0,0 @@
/*
2022-05-20
The author disclaims copyright to this source code. In place of a
legal notice, here is a blessing:
* May you do good and not evil.
* May you find forgiveness for yourself and forgive others.
* May you share freely, never taking more than you give.
***********************************************************************
This is the main entry point for the sqlite3 fiddle app. It sets up the
various UI bits, loads a Worker for the db connection, and manages the
communication between the UI and worker.
*/
(function(){
'use strict';
/* Recall that the 'self' symbol, except where locally
overwritten, refers to the global window or worker object. */
const storage = (function(NS/*namespace object in which to store this module*/){
/* Pedantic licensing note: this code originated in the Fossil SCM
source tree, where it has a different license, but the person who
ported it into sqlite is the same one who wrote it for fossil. */
'use strict';
NS = NS||{};
/**
This module provides a basic wrapper around localStorage
or sessionStorage or a dummy proxy object if neither
of those are available.
*/
const tryStorage = function f(obj){
if(!f.key) f.key = 'storage.access.check';
try{
obj.setItem(f.key, 'f');
const x = obj.getItem(f.key);
obj.removeItem(f.key);
if(x!=='f') throw new Error(f.key+" failed")
return obj;
}catch(e){
return undefined;
}
};
/** Internal storage impl for this module. */
const $storage =
tryStorage(window.localStorage)
|| tryStorage(window.sessionStorage)
|| tryStorage({
// A basic dummy xyzStorage stand-in
$$$:{},
setItem: function(k,v){this.$$$[k]=v},
getItem: function(k){
return this.$$$.hasOwnProperty(k) ? this.$$$[k] : undefined;
},
removeItem: function(k){delete this.$$$[k]},
clear: function(){this.$$$={}}
});
/**
For the dummy storage we need to differentiate between
$storage and its real property storage for hasOwnProperty()
to work properly...
*/
const $storageHolder = $storage.hasOwnProperty('$$$') ? $storage.$$$ : $storage;
/**
A prefix which gets internally applied to all storage module
property keys so that localStorage and sessionStorage across the
same browser profile instance do not "leak" across multiple apps
being hosted by the same origin server. Such cross-polination is
still there but, with this key prefix applied, it won't be
immediately visible via the storage API.
With this in place we can justify using localStorage instead of
sessionStorage.
One implication of using localStorage and sessionStorage is that
their scope (the same "origin" and client application/profile)
allows multiple apps on the same origin to use the same
storage. Thus /appA/foo could then see changes made via
/appB/foo. The data do not cross user- or browser boundaries,
though, so it "might" arguably be called a
feature. storageKeyPrefix was added so that we can sandbox that
state for each separate app which shares an origin.
See: https://fossil-scm.org/forum/forumpost/4afc4d34de
Sidebar: it might seem odd to provide a key prefix and stick all
properties in the topmost level of the storage object. We do that
because adding a layer of object to sandbox each app would mean
(de)serializing that whole tree on every storage property change.
e.g. instead of storageObject.projectName.foo we have
storageObject[storageKeyPrefix+'foo']. That's soley for
efficiency's sake (in terms of battery life and
environment-internal storage-level effort).
*/
const storageKeyPrefix = (
$storageHolder===$storage/*localStorage or sessionStorage*/
? (
(NS.config ?
(NS.config.projectCode || NS.config.projectName
|| NS.config.shortProjectName)
: false)
|| window.location.pathname
)+'::' : (
'' /* transient storage */
)
);
/**
A proxy for localStorage or sessionStorage or a
page-instance-local proxy, if neither one is availble.
Which exact storage implementation is uses is unspecified, and
apps must not rely on it.
*/
NS.storage = {
storageKeyPrefix: storageKeyPrefix,
/** Sets the storage key k to value v, implicitly converting
it to a string. */
set: (k,v)=>$storage.setItem(storageKeyPrefix+k,v),
/** Sets storage key k to JSON.stringify(v). */
setJSON: (k,v)=>$storage.setItem(storageKeyPrefix+k,JSON.stringify(v)),
/** Returns the value for the given storage key, or
dflt if the key is not found in the storage. */
get: (k,dflt)=>$storageHolder.hasOwnProperty(
storageKeyPrefix+k
) ? $storage.getItem(storageKeyPrefix+k) : dflt,
/** Returns true if the given key has a value of "true". If the
key is not found, it returns true if the boolean value of dflt
is "true". (Remember that JS persistent storage values are all
strings.) */
getBool: function(k,dflt){
return 'true'===this.get(k,''+(!!dflt));
},
/** Returns the JSON.parse()'d value of the given
storage key's value, or dflt is the key is not
found or JSON.parse() fails. */
getJSON: function f(k,dflt){
try {
const x = this.get(k,f);
return x===f ? dflt : JSON.parse(x);
}
catch(e){return dflt}
},
/** Returns true if the storage contains the given key,
else false. */
contains: (k)=>$storageHolder.hasOwnProperty(storageKeyPrefix+k),
/** Removes the given key from the storage. Returns this. */
remove: function(k){
$storage.removeItem(storageKeyPrefix+k);
return this;
},
/** Clears ALL keys from the storage. Returns this. */
clear: function(){
this.keys().forEach((k)=>$storage.removeItem(/*w/o prefix*/k));
return this;
},
/** Returns an array of all keys currently in the storage. */
keys: ()=>Object.keys($storageHolder).filter((v)=>(v||'').startsWith(storageKeyPrefix)),
/** Returns true if this storage is transient (only available
until the page is reloaded), indicating that fileStorage
and sessionStorage are unavailable. */
isTransient: ()=>$storageHolder!==$storage,
/** Returns a symbolic name for the current storage mechanism. */
storageImplName: function(){
if($storage===window.localStorage) return 'localStorage';
else if($storage===window.sessionStorage) return 'sessionStorage';
else return 'transient';
},
/**
Returns a brief help text string for the currently-selected
storage type.
*/
storageHelpDescription: function(){
return {
localStorage: "Browser-local persistent storage with an "+
"unspecified long-term lifetime (survives closing the browser, "+
"but maybe not a browser upgrade).",
sessionStorage: "Storage local to this browser tab, "+
"lost if this tab is closed.",
"transient": "Transient storage local to this invocation of this page."
}[this.storageImplName()];
}
};
return NS.storage;
})({})/*storage API setup*/;
/** Name of the stored copy of SqliteFiddle.config. */
const configStorageKey = 'sqlite3-fiddle-config';
/**
The SqliteFiddle object is intended to be the primary
app-level object for the main-thread side of the sqlite
fiddle application. It uses a worker thread to load the
sqlite WASM module and communicate with it.
*/
const SF/*local convenience alias*/
= window.SqliteFiddle/*canonical name*/ = {
/* Config options. */
config: {
/* If true, SqliteFiddle.echo() will auto-scroll the
output widget to the bottom when it receives output,
else it won't. */
autoScrollOutput: true,
/* If true, the output area will be cleared before each
command is run, else it will not. */
autoClearOutput: false,
/* If true, SqliteFiddle.echo() will echo its output to
the console, in addition to its normal output widget.
That slows it down but is useful for testing. */
echoToConsole: false,
/* If true, display input/output areas side-by-side. */
sideBySide: true,
/* If true, swap positions of the input/output areas. */
swapInOut: false
},
/**
Emits the given text, followed by a line break, to the
output widget. If given more than one argument, they are
join()'d together with a space between each. As a special
case, if passed a single array, that array is used in place
of the arguments array (this is to facilitate receiving
lists of arguments via worker events).
*/
echo: function f(text) {
/* Maintenance reminder: we currently require/expect a textarea
output element. It might be nice to extend this to behave
differently if the output element is a non-textarea element,
in which case it would need to append the given text as a TEXT
node and add a line break. */
if(!f._){
f._ = document.getElementById('output');
f._.value = ''; // clear browser cache
}
if(arguments.length > 1) text = Array.prototype.slice.call(arguments).join(' ');
else if(1===arguments.length && Array.isArray(text)) text = text.join(' ');
// These replacements are necessary if you render to raw HTML
//text = text.replace(/&/g, "&amp;");
//text = text.replace(/</g, "&lt;");
//text = text.replace(/>/g, "&gt;");
//text = text.replace('\n', '<br>', 'g');
if(null===text){/*special case: clear output*/
f._.value = '';
return;
}else if(this.echo._clearPending){
delete this.echo._clearPending;
f._.value = '';
}
if(this.config.echoToConsole) console.log(text);
if(this.jqTerm) this.jqTerm.echo(text);
f._.value += text + "\n";
if(this.config.autoScrollOutput){
f._.scrollTop = f._.scrollHeight;
}
},
_msgMap: {},
/** Adds a worker message handler for messages of the given
type. */
addMsgHandler: function f(type,callback){
if(Array.isArray(type)){
type.forEach((t)=>this.addMsgHandler(t, callback));
return this;
}
(this._msgMap.hasOwnProperty(type)
? this._msgMap[type]
: (this._msgMap[type] = [])).push(callback);
return this;
},
/** Given a worker message, runs all handlers for msg.type. */
runMsgHandlers: function(msg){
const list = (this._msgMap.hasOwnProperty(msg.type)
? this._msgMap[msg.type] : false);
if(!list){
console.warn("No handlers found for message type:",msg);
return false;
}
//console.debug("runMsgHandlers",msg);
list.forEach((f)=>f(msg));
return true;
},
/** Removes all message handlers for the given message type. */
clearMsgHandlers: function(type){
delete this._msgMap[type];
return this;
},
/* Posts a message in the form {type, data} to the db worker. Returns this. */
wMsg: function(type,data){
this.worker.postMessage({type, data});
return this;
},
/**
Prompts for confirmation and, if accepted, deletes
all content and tables in the (transient) database.
*/
resetDb: function(){
if(window.confirm("Really destroy all content and tables "
+"in the (transient) db?")){
this.wMsg('db-reset');
}
return this;
},
/** Stores this object's config in the browser's storage. */
storeConfig: function(){
storage.setJSON(configStorageKey,this.config);
}
};
if(1){ /* Restore SF.config */
const storedConfig = storage.getJSON(configStorageKey);
if(storedConfig){
/* Copy all properties to SF.config which are currently in
storedConfig. We don't bother copying any other
properties: those have been removed from the app in the
meantime. */
Object.keys(SF.config).forEach(function(k){
if(storedConfig.hasOwnProperty(k)){
SF.config[k] = storedConfig[k];
}
});
}
}
SF.worker = new Worker('fiddle-worker.js');
SF.worker.onmessage = (ev)=>SF.runMsgHandlers(ev.data);
SF.addMsgHandler(['stdout', 'stderr'], (ev)=>SF.echo(ev.data));
/* querySelectorAll() proxy */
const EAll = function(/*[element=document,] cssSelector*/){
return (arguments.length>1 ? arguments[0] : document)
.querySelectorAll(arguments[arguments.length-1]);
};
/* querySelector() proxy */
const E = function(/*[element=document,] cssSelector*/){
return (arguments.length>1 ? arguments[0] : document)
.querySelector(arguments[arguments.length-1]);
};
/** Handles status updates from the Module object. */
SF.addMsgHandler('module', function f(ev){
ev = ev.data;
if('status'!==ev.type){
console.warn("Unexpected module-type message:",ev);
return;
}
if(!f.ui){
f.ui = {
status: E('#module-status'),
progress: E('#module-progress'),
spinner: E('#module-spinner')
};
}
const msg = ev.data;
if(f.ui.progres){
progress.value = msg.step;
progress.max = msg.step + 1/*we don't know how many steps to expect*/;
}
if(1==msg.step){
f.ui.progress.classList.remove('hidden');
f.ui.spinner.classList.remove('hidden');
}
if(msg.text){
f.ui.status.classList.remove('hidden');
f.ui.status.innerText = msg.text;
}else{
if(f.ui.progress){
f.ui.progress.remove();
f.ui.spinner.remove();
delete f.ui.progress;
delete f.ui.spinner;
}
f.ui.status.classList.add('hidden');
/* The module can post messages about fatal problems,
e.g. an exit() being triggered or assertion failure,
after the last "load" message has arrived, so
leave f.ui.status and message listener intact. */
}
});
/**
The 'fiddle-ready' event is fired (with no payload) when the
wasm module has finished loading. Interestingly, that happens
_before_ the final module:status event */
SF.addMsgHandler('fiddle-ready', function(){
SF.clearMsgHandlers('fiddle-ready');
self.onSFLoaded();
});
/**
Performs all app initialization which must wait until after the
worker module is loaded. This function removes itself when it's
called.
*/
self.onSFLoaded = function(){
delete this.onSFLoaded;
// Unhide all elements which start out hidden
EAll('.initially-hidden').forEach((e)=>e.classList.remove('initially-hidden'));
E('#btn-reset').addEventListener('click',()=>SF.resetDb());
const taInput = E('#input');
const btnClearIn = E('#btn-clear');
btnClearIn.addEventListener('click',function(){
taInput.value = '';
},false);
// Ctrl-enter and shift-enter both run the current SQL.
taInput.addEventListener('keydown',function(ev){
if((ev.ctrlKey || ev.shiftKey) && 13 === ev.keyCode){
ev.preventDefault();
ev.stopPropagation();
btnShellExec.click();
}
}, false);
const taOutput = E('#output');
const btnClearOut = E('#btn-clear-output');
btnClearOut.addEventListener('click',function(){
taOutput.value = '';
if(SF.jqTerm) SF.jqTerm.clear();
},false);
const btnShellExec = E('#btn-shell-exec');
btnShellExec.addEventListener('click',function(ev){
let sql;
ev.preventDefault();
if(taInput.selectionStart<taInput.selectionEnd){
sql = taInput.value.substring(taInput.selectionStart,taInput.selectionEnd).trim();
}else{
sql = taInput.value.trim();
}
if(sql) SF.dbExec(sql);
},false);
const btnInterrupt = E("#btn-interrupt");
//btnInterrupt.classList.add('hidden');
/** To be called immediately before work is sent to the
worker. Updates some UI elements. The 'working'/'end'
event will apply the inverse, undoing the bits this
function does. This impl is not in the 'working'/'start'
event handler because that event is given to us
asynchronously _after_ we need to have performed this
work.
*/
const preStartWork = function f(){
if(!f._){
const title = E('title');
f._ = {
btnLabel: btnShellExec.innerText,
pageTitle: title,
pageTitleOrig: title.innerText
};
}
f._.pageTitle.innerText = "[working...] "+f._.pageTitleOrig;
btnShellExec.setAttribute('disabled','disabled');
btnInterrupt.removeAttribute('disabled','disabled');
};
/* Sends the given text to the db module to evaluate as if it
had been entered in the sqlite3 CLI shell. If it's null or
empty, this is a no-op except that the very first call will
initialize the db and output an informational header. */
SF.dbExec = function f(sql){
if(this.config.autoClearOutput){
this.echo._clearPending = true;
}
preStartWork();
this.wMsg('shellExec',sql);
};
SF.addMsgHandler('working',function f(ev){
switch(ev.data){
case 'start': /* See notes in preStartWork(). */; return;
case 'end':
preStartWork._.pageTitle.innerText = preStartWork._.pageTitleOrig;
btnShellExec.innerText = preStartWork._.btnLabel;
btnShellExec.removeAttribute('disabled');
btnInterrupt.setAttribute('disabled','disabled');
return;
}
console.warn("Unhandled 'working' event:",ev.data);
});
/* For each checkbox with data-csstgt, set up a handler which
toggles the given CSS class on the element matching
E(data-csstgt). */
EAll('input[type=checkbox][data-csstgt]')
.forEach(function(e){
const tgt = E(e.dataset.csstgt);
const cssClass = e.dataset.cssclass || 'error';
e.checked = tgt.classList.contains(cssClass);
e.addEventListener('change', function(){
tgt.classList[
this.checked ? 'add' : 'remove'
](cssClass)
}, false);
});
/* For each checkbox with data-config=X, set up a binding to
SF.config[X]. These must be set up AFTER data-csstgt
checkboxes so that those two states can be synced properly. */
EAll('input[type=checkbox][data-config]')
.forEach(function(e){
const confVal = !!SF.config[e.dataset.config];
if(e.checked !== confVal){
/* Ensure that data-csstgt mappings (if any) get
synced properly. */
e.checked = confVal;
e.dispatchEvent(new Event('change'));
}
e.addEventListener('change', function(){
SF.config[this.dataset.config] = this.checked;
SF.storeConfig();
}, false);
});
/* For each button with data-cmd=X, map a click handler which
calls SF.dbExec(X). */
const cmdClick = function(){SF.dbExec(this.dataset.cmd);};
EAll('button[data-cmd]').forEach(
e => e.addEventListener('click', cmdClick, false)
);
btnInterrupt.addEventListener('click',function(){
SF.wMsg('interrupt');
});
/** Initiate a download of the db. */
const btnExport = E('#btn-export');
const eLoadDb = E('#load-db');
const btnLoadDb = E('#btn-load-db');
btnLoadDb.addEventListener('click', ()=>eLoadDb.click());
/**
Enables (if passed true) or disables all UI elements which
"might," if timed "just right," interfere with an
in-progress db import/export/exec operation.
*/
const enableMutatingElements = function f(enable){
if(!f._elems){
f._elems = [
/* UI elements to disable while import/export are
running. Normally the export is fast enough
that this won't matter, but we really don't
want to be reading (from outside of sqlite) the
db when the user taps btnShellExec. */
btnShellExec, btnExport, eLoadDb
];
}
f._elems.forEach( enable
? (e)=>e.removeAttribute('disabled')
: (e)=>e.setAttribute('disabled','disabled') );
};
btnExport.addEventListener('click',function(){
enableMutatingElements(false);
SF.wMsg('db-export');
});
SF.addMsgHandler('db-export', function(ev){
enableMutatingElements(true);
ev = ev.data;
if(ev.error){
SF.echo("Export failed:",ev.error);
return;
}
const blob = new Blob([ev.buffer], {type:"application/x-sqlite3"});
const a = document.createElement('a');
document.body.appendChild(a);
a.href = window.URL.createObjectURL(blob);
a.download = ev.filename;
a.addEventListener('click',function(){
setTimeout(function(){
SF.echo("Exported (possibly auto-downloaded):",ev.filename);
window.URL.revokeObjectURL(a.href);
a.remove();
},500);
});
a.click();
});
/**
Handle load/import of an external db file.
*/
eLoadDb.addEventListener('change',function(){
const f = this.files[0];
const r = new FileReader();
const status = {loaded: 0, total: 0};
enableMutatingElements(false);
r.addEventListener('loadstart', function(){
SF.echo("Loading",f.name,"...");
});
r.addEventListener('progress', function(ev){
SF.echo("Loading progress:",ev.loaded,"of",ev.total,"bytes.");
});
const that = this;
r.addEventListener('load', function(){
enableMutatingElements(true);
SF.echo("Loaded",f.name+". Opening db...");
SF.wMsg('open',{
filename: f.name,
buffer: this.result
});
});
r.addEventListener('error',function(){
enableMutatingElements(true);
SF.echo("Loading",f.name,"failed for unknown reasons.");
});
r.addEventListener('abort',function(){
enableMutatingElements(true);
SF.echo("Cancelled loading of",f.name+".");
});
r.readAsArrayBuffer(f);
});
EAll('fieldset.collapsible').forEach(function(fs){
const btnToggle = E(fs,'legend > .fieldset-toggle'),
content = EAll(fs,':scope > div');
btnToggle.addEventListener('click', function(){
fs.classList.toggle('collapsed');
content.forEach((d)=>d.classList.toggle('hidden'));
}, false);
});
/**
Given a DOM element, this routine measures its "effective
height", which is the bounding top/bottom range of this element
and all of its children, recursively. For some DOM structure
cases, a parent may have a reported height of 0 even though
children have non-0 sizes.
Returns 0 if !e or if the element really has no height.
*/
const effectiveHeight = function f(e){
if(!e) return 0;
if(!f.measure){
f.measure = function callee(e, depth){
if(!e) return;
const m = e.getBoundingClientRect();
if(0===depth){
callee.top = m.top;
callee.bottom = m.bottom;
}else{
callee.top = m.top ? Math.min(callee.top, m.top) : callee.top;
callee.bottom = Math.max(callee.bottom, m.bottom);
}
Array.prototype.forEach.call(e.children,(e)=>callee(e,depth+1));
if(0===depth){
//console.debug("measure() height:",e.className, callee.top, callee.bottom, (callee.bottom - callee.top));
f.extra += callee.bottom - callee.top;
}
return f.extra;
};
}
f.extra = 0;
f.measure(e,0);
return f.extra;
};
/**
Returns a function, that, as long as it continues to be invoked,
will not be triggered. The function will be called after it stops
being called for N milliseconds. If `immediate` is passed, call
the callback immediately and hinder future invocations until at
least the given time has passed.
If passed only 1 argument, or passed a falsy 2nd argument,
the default wait time set in this function's $defaultDelay
property is used.
Source: underscore.js, by way of https://davidwalsh.name/javascript-debounce-function
*/
const debounce = function f(func, wait, immediate) {
var timeout;
if(!wait) wait = f.$defaultDelay;
return function() {
const context = this, args = Array.prototype.slice.call(arguments);
const later = function() {
timeout = undefined;
if(!immediate) func.apply(context, args);
};
const callNow = immediate && !timeout;
clearTimeout(timeout);
timeout = setTimeout(later, wait);
if(callNow) func.apply(context, args);
};
};
debounce.$defaultDelay = 500 /*arbitrary*/;
const ForceResizeKludge = (function(){
/* Workaround for Safari mayhem regarding use of vh CSS
units.... We cannot use vh units to set the main view
size because Safari chokes on that, so we calculate
that height here. Larger than ~95% is too big for
Firefox on Android, causing the input area to move
off-screen. */
const appViews = EAll('.app-view');
const elemsToCount = [
/* Elements which we need to always count in the
visible body size. */
E('body > header'),
E('body > footer')
];
const resized = function f(){
if(f.$disabled) return;
const wh = window.innerHeight;
var ht;
var extra = 0;
elemsToCount.forEach((e)=>e ? extra += effectiveHeight(e) : false);
ht = wh - extra;
appViews.forEach(function(e){
e.style.height =
e.style.maxHeight = [
"calc(", (ht>=100 ? ht : 100), "px",
" - 2em"/*fudge value*/,")"
/* ^^^^ hypothetically not needed, but both
Chrome/FF on Linux will force scrollbars on the
body if this value is too small. */
].join('');
});
};
resized.$disabled = true/*gets deleted when setup is finished*/;
window.addEventListener('resize', debounce(resized, 250), false);
return resized;
})();
/** Set up a selection list of examples */
(function(){
const xElem = E('#select-examples');
const examples = [
{name: "Help", sql:
`-- ================================================
-- Use ctrl-enter or shift-enter to execute sqlite3
-- shell commands and SQL.
-- If a subset of the text is currently selected,
-- only that part is executed.
-- ================================================
.help`},
{name: "Timer on", sql: ".timer on"},
{name: "Setup table T", sql:`.nullvalue NULL
CREATE TABLE t(a,b);
INSERT INTO t(a,b) VALUES('abc',123),('def',456),(NULL,789),('ghi',012);
SELECT * FROM t;`},
{name: "Table list", sql: ".tables"},
{name: "Box Mode", sql: ".mode box"},
{name: "JSON Mode", sql: ".mode json"},
{name: "Mandlebrot", sql: `WITH RECURSIVE
xaxis(x) AS (VALUES(-2.0) UNION ALL SELECT x+0.05 FROM xaxis WHERE x<1.2),
yaxis(y) AS (VALUES(-1.0) UNION ALL SELECT y+0.1 FROM yaxis WHERE y<1.0),
m(iter, cx, cy, x, y) AS (
SELECT 0, x, y, 0.0, 0.0 FROM xaxis, yaxis
UNION ALL
SELECT iter+1, cx, cy, x*x-y*y + cx, 2.0*x*y + cy FROM m
WHERE (x*x + y*y) < 4.0 AND iter<28
),
m2(iter, cx, cy) AS (
SELECT max(iter), cx, cy FROM m GROUP BY cx, cy
),
a(t) AS (
SELECT group_concat( substr(' .+*#', 1+min(iter/7,4), 1), '')
FROM m2 GROUP BY cy
)
SELECT group_concat(rtrim(t),x'0a') as Mandelbrot FROM a;`}
];
const newOpt = function(lbl,val){
const o = document.createElement('option');
o.value = val;
if(!val) o.setAttribute('disabled',true);
o.appendChild(document.createTextNode(lbl));
xElem.appendChild(o);
};
newOpt("Examples (replaces input!)");
examples.forEach((o)=>newOpt(o.name, o.sql));
//xElem.setAttribute('disabled',true);
xElem.selectedIndex = 0;
xElem.addEventListener('change', function(){
taInput.value = '-- ' +
this.selectedOptions[0].innerText +
'\n' + this.value;
SF.dbExec(this.value);
});
})()/* example queries */;
SF.echo(null/*clear any output generated by the init process*/);
if(window.jQuery && window.jQuery.terminal){
/* Set up the terminal-style view... */
const eTerm = window.jQuery('#view-terminal').empty();
SF.jqTerm = eTerm.terminal(SF.dbExec.bind(SF),{
prompt: 'sqlite> ',
greetings: false /* note that the docs incorrectly call this 'greeting' */
});
/* Set up a button to toggle the views... */
const head = E('header#titlebar');
const btnToggleView = document.createElement('button');
btnToggleView.appendChild(document.createTextNode("Toggle View"));
head.appendChild(btnToggleView);
btnToggleView.addEventListener('click',function f(){
EAll('.app-view').forEach(e=>e.classList.toggle('hidden'));
if(document.body.classList.toggle('terminal-mode')){
ForceResizeKludge();
}
}, false);
btnToggleView.click()/*default to terminal view*/;
}
SF.dbExec(null/*init the db and output the header*/);
SF.echo('This experimental app is provided in the hope that it',
'may prove interesting or useful but is not an officially',
'supported deliverable of the sqlite project. It is subject to',
'any number of changes or outright removal at any time.\n');
delete ForceResizeKludge.$disabled;
ForceResizeKludge();
btnShellExec.click();
}/*onSFLoaded()*/;
})();
-94
View File
@@ -1,94 +0,0 @@
This directory houses a "fiddle"-style application which embeds a
[Web Assembly (WASM)](https://en.wikipedia.org/wiki/WebAssembly)
build of the sqlite3 shell app into an HTML page, effectively running
the shell in a client-side browser.
It requires [emscripten][] and that the build environment be set up for
emscripten. A mini-HOWTO for setting that up follows...
First, install the Emscripten SDK, as documented
[here](https://emscripten.org/docs/getting_started/downloads.html) and summarized
below for Linux environments:
```
# Clone the emscripten repository:
$ git clone https://github.com/emscripten-core/emsdk.git
$ cd emsdk
# Download and install the latest SDK tools:
$ ./emsdk install latest
# Make the "latest" SDK "active" for the current user:
$ ./emsdk activate latest
```
Those parts only need to be run once. The following needs to be run for each
shell instance which needs the `emcc` compiler:
```
# Activate PATH and other environment variables in the current terminal:
$ source ./emsdk_env.sh
$ which emcc
/path/to/emsdk/upstream/emscripten/emcc
```
That `env` script needs to be sourced for building this application from the
top of the sqlite3 build tree:
```
$ make fiddle
```
Or:
```
$ cd ext/fiddle
$ make
```
That will generate the fiddle application under
[ext/fiddle](/dir/ext/fiddle), as `fiddle.html`. That application
cannot, due to XMLHttpRequest security limitations, run if the HTML
file is opened directly in the browser (i.e. if it is opened using a
`file://` URL), so it needs to be served via an HTTP server. For
example, using [althttpd][]:
```
$ cd ext/fiddle
$ althttpd -debug 1 -jail 0 -port 9090 -root .
```
Then browse to `http://localhost:9090/fiddle.html`.
Note that when serving this app via [althttpd][], it must be a version
from 2022-05-17 or newer so that it recognizes the `.wasm` file
extension and responds with the mimetype `application/wasm`, as the
WASM loader is pedantic about that detail.
# Known Quirks and Limitations
Some "impedence mismatch" between C and WASM/JavaScript is to be
expected.
## No I/O
sqlite3 shell commands which require file I/O or pipes are disabled in
the WASM build.
## `exit()` Triggered from C
When C code calls `exit()`, as happens (for example) when running an
"unsafe" command when safe mode is active, WASM's connection to the
sqlite3 shell environment has no sensible choice but to shut down
because `exit()` leaves it in a state we can no longer recover
from. The JavaScript-side application attempts to recognize this and
warn the user that restarting the application is necessary. Currently
the only way to restart it is to reload the page. Restructuring the
shell code such that it could be "rebooted" without restarting the
JS app would require some invasive changes which are not currently
on any TODO list but have not been entirely ruled out long-term.
[emscripten]: https://emscripten.org
[althttpd]: https://sqlite.org/althttpd
File diff suppressed because it is too large Load Diff
-44
View File
@@ -1,44 +0,0 @@
/*
2022-05-23
The author disclaims copyright to this source code. In place of a
legal notice, here is a blessing:
* May you do good and not evil.
* May you find forgiveness for yourself and forgive others.
* May you share freely, never taking more than you give.
***********************************************************************
This is a JS Worker file for the main sqlite3 api. It loads
sqlite3.js, initializes the module, and postMessage()'s a message
after the module is initialized:
{type: 'sqlite3-api', data: 'ready'}
This seemingly superfluous level of indirection is necessary when
loading sqlite3.js via a Worker. Loading sqlite3.js from the main
window thread elides the Worker-specific API. Instantiating a worker
with new Worker("sqlite.js") will not (cannot) call
initSqlite3Module() to initialize the module due to a
timing/order-of-operations conflict (and that symbol is not exported
in a way that a Worker loading it that way can see it). Thus JS
code wanting to load the sqlite3 Worker-specific API needs to pass
_this_ file (or equivalent) to the Worker constructor and then
listen for an event in the form shown above in order to know when
the module has completed initialization. sqlite3.js will fire a
similar event, with data:'loaded' as the final step in its loading
process. Whether or not we _really_ need both 'loaded' and 'ready'
events is unclear, but they are currently separate events primarily
for the sake of clarity in the timing of when it's okay to use the
loaded module. At the time the 'loaded' event is fired, it's
possible (but unknown and unknowable) that the emscripten-generated
module-setup infrastructure still has work to do. Thus it is
hypothesized that client code is better off waiting for the 'ready'
even before using the API.
*/
"use strict";
importScripts('sqlite3.js');
initSqlite3Module().then(function(){
setTimeout(()=>self.postMessage({type:'sqlite3-api',data:'ready'}), 0);
});
-31
View File
@@ -1,31 +0,0 @@
textarea {
font-family: monospace;
}
header {
font-size: 130%;
font-weight: bold;
}
.hidden, .initially-hidden {
position: absolute !important;
opacity: 0 !important;
pointer-events: none !important;
display: none !important;
}
fieldset.options {
font-size: 75%;
}
fieldset > legend {
padding: 0 0.5em;
}
span.labeled-input {
padding: 0.25em;
margin: 0.25em 0.5em;
border-radius: 0.25em;
white-space: nowrap;
background: #0002;
}
.center { text-align: center; }
.error {
color: red;
background-color: yellow;
}
-185
View File
@@ -1,185 +0,0 @@
/*
2022-05-22
The author disclaims copyright to this source code. In place of a
legal notice, here is a blessing:
* May you do good and not evil.
* May you find forgiveness for yourself and forgive others.
* May you share freely, never taking more than you give.
***********************************************************************
A basic test script for sqlite3-api.js. This file must be run in
main JS thread and sqlite3.js must have been loaded before it.
*/
(function(){
const T = self.SqliteTestUtil;
const log = console.log.bind(console);
const assert = function(condition, text) {
if (!condition) {
throw new Error('Assertion failed' + (text ? ': ' + text : ''));
}
};
const test1 = function(db,sqlite3){
const api = sqlite3.api;
log("Basic sanity tests...");
T.assert(db._pDb);
let st = db.prepare("select 3 as a");
//log("statement =",st);
T.assert(st._pStmt)
.assert(!st._mayGet)
.assert('a' === st.getColumnName(0))
.assert(st === db._statements[st._pStmt])
.assert(1===st.columnCount)
.assert(0===st.parameterCount)
.mustThrow(()=>st.bind(1,null))
.assert(true===st.step())
.assert(3 === st.get(0))
.mustThrow(()=>st.get(1))
.mustThrow(()=>st.get(0,~api.SQLITE_INTEGER))
.assert(3 === st.get(0,api.SQLITE_INTEGER))
.assert(3 === st.getInt(0))
.assert('3' === st.get(0,api.SQLITE_TEXT))
.assert('3' === st.getString(0))
.assert(3.0 === st.get(0,api.SQLITE_FLOAT))
.assert(3.0 === st.getFloat(0))
.assert(st.get(0,api.SQLITE_BLOB) instanceof Uint8Array)
.assert(st.getBlob(0) instanceof Uint8Array)
.assert(3 === st.get([])[0])
.assert(3 === st.get({}).a)
.assert(3 === st.getJSON(0))
.assert(st._mayGet)
.assert(false===st.step())
.assert(!st._mayGet)
;
let pId = st._pStmt;
st.finalize();
T.assert(!st._pStmt)
.assert(!db._statements[pId]);
let list = [];
db.exec({
sql:`CREATE TABLE t(a,b);
INSERT INTO t(a,b) VALUES(1,2),(3,4),(?,?);`,
multi: true,
saveSql: list,
bind: [5,6]
});
T.assert(2 === list.length);
//log("Exec'd SQL:", list);
let counter = 0, colNames = [];
list.length = 0;
db.exec("SELECT a a, b b FROM t",{
rowMode: 'object',
resultRows: list,
columnNames: colNames,
callback: function(row,stmt){
++counter;
T.assert(row.a%2 && row.a<6);
}
});
T.assert(2 === colNames.length)
.assert('a' === colNames[0])
.assert(3 === counter)
.assert(3 === list.length);
list.length = 0;
db.exec("SELECT a a, b b FROM t",{
rowMode: 'array',
callback: function(row,stmt){
++counter;
T.assert(Array.isArray(row))
.assert(0===row[1]%2 && row[1]<7);
}
});
T.assert(6 === counter);
};
const testUDF = function(db){
log("Testing UDF...");
db.createFunction("foo",function(a,b){return a+b});
T.assert(7===db.selectValue("select foo(3,4)")).
assert(5===db.selectValue("select foo(3,?)",2)).
assert(5===db.selectValue("select foo(?,?)",[1,4])).
assert(5===db.selectValue("select foo($a,$b)",{$a:0,$b:5}));
db.createFunction("bar", {
arity: -1,
callback: function(){
var rc = 0;
for(let i = 0; i < arguments.length; ++i) rc += arguments[i];
return rc;
}
});
log("Testing DB::selectValue() w/ UDF...");
T.assert(0===db.selectValue("select bar()")).
assert(1===db.selectValue("select bar(1)")).
assert(3===db.selectValue("select bar(1,2)")).
assert(-1===db.selectValue("select bar(1,2,-4)"));
const eqApprox = function(v1,v2,factor=0.05){
return v1>=(v2-factor) && v1<=(v2+factor);
};
T.assert('hi' === db.selectValue("select ?",'hi')).
assert(null===db.selectValue("select null")).
assert(null === db.selectValue("select ?",null)).
assert(null === db.selectValue("select ?",[null])).
assert(null === db.selectValue("select $a",{$a:null})).
assert(eqApprox(3.1,db.selectValue("select 3.0 + 0.1")))
;
};
const testAttach = function(db){
log("Testing ATTACH...");
db.exec({
sql:[
"attach 'foo.db' as foo",
"create table foo.bar(a)",
"insert into foo.bar(a) values(1),(2),(3)"
].join(';'),
multi: true
});
T.assert(2===db.selectValue('select a from foo.bar where a>1 order by a'));
db.exec("detach foo");
T.mustThrow(()=>db.exec("select * from foo.bar"));
};
const runTests = function(Module){
T.assert(Module._free instanceof Function).
assert(Module.allocate instanceof Function).
assert(Module.addFunction instanceof Function).
assert(Module.removeFunction instanceof Function);
const sqlite3 = Module.sqlite3;
const api = sqlite3.api;
const oo = sqlite3.SQLite3;
console.log("Loaded module:",api.sqlite3_libversion(),
api.sqlite3_sourceid());
log("Build options:",oo.compileOptionUsed());
const db = new oo.DB();
try {
log("DB:",db.filename);
[
test1, testUDF, testAttach
].forEach((f)=>{
const t = T.counter;
f(db, sqlite3);
log("Test count:",T.counter - t);
});
}finally{
db.close();
}
log("Total Test count:",T.counter);
};
initSqlite3Module(self.sqlite3TestModule).then(function(theModule){
/** Use a timeout so that we are (hopefully) out from
under the module init stack when our setup gets
run. Just on principle, not because we _need_ to
be. */
//console.debug("theModule =",theModule);
setTimeout(()=>runTests(theModule), 0);
});
})();
-235
View File
@@ -1,235 +0,0 @@
/*
2022-05-22
The author disclaims copyright to this source code. In place of a
legal notice, here is a blessing:
* May you do good and not evil.
* May you find forgiveness for yourself and forgive others.
* May you share freely, never taking more than you give.
***********************************************************************
A basic test script for sqlite3-worker.js.
*/
(function(){
const T = self.SqliteTestUtil;
const SW = new Worker("sqlite3-worker.js");
/** Posts a worker message as {type:type, data:data}. */
const wMsg = function(type,data){
SW.postMessage({type, data});
return SW;
};
const log = console.log.bind(console);
const warn = console.warn.bind(console);
const error = console.error.bind(console);
SW.onerror = function(event){
error("onerror",event);
};
/**
A queue for callbacks which are to be run in response to async
DB commands. See the notes in runTests() for why we need
this. The event-handling plumbing of this file requires that
any DB command which includes a `messageId` property also have
a queued callback entry, as the existence of that property in
response payloads is how it knows whether or not to shift an
entry off of the queue.
*/
const MsgHandlerQueue = {
queue: [],
id: 0,
push: function(type,callback){
this.queue.push(callback);
return type + '-' + (++this.id);
},
shift: function(){
return this.queue.shift();
}
};
const testCount = ()=>log("Total test count:",T.counter);
const runOneTest = function(eventType, eventData, callback){
T.assert(eventData && 'object'===typeof eventData);
/* ^^^ that is for the testing and messageId-related code, not
a hard requirement of all of the Worker-exposed APIs. */
eventData.messageId = MsgHandlerQueue.push(eventType,function(ev){
log("runOneTest",eventType,"result",ev.data);
if(callback instanceof Function){
callback(ev);
testCount();
}
});
wMsg(eventType, eventData);
};
/** Methods which map directly to onmessage() event.type keys.
They get passed the inbound event.data. */
const dbMsgHandler = {
open: function(ev){
log("open result",ev.data);
},
exec: function(ev){
log("exec result",ev.data);
},
export: function(ev){
log("exec result",ev.data);
},
error: function(ev){
error("ERROR from the worker:",ev.data);
},
resultRowTest1: function f(ev){
if(undefined === f.counter) f.counter = 0;
if(ev.data) ++f.counter;
//log("exec() result row:",ev.data);
T.assert(null===ev.data || 'number' === typeof ev.data.b);
}
};
const runTests = function(){
const mustNotReach = ()=>{
throw new Error("This is not supposed to be reached.");
};
/**
"The problem" now is that the test results are async. We
know, however, that the messages posted to the worker will
be processed in the order they are passed to it, so we can
create a queue of callbacks to handle them. The problem
with that approach is that it's not error-handling
friendly, in that an error can cause us to bypass a result
handler queue entry. We have to perform some extra
acrobatics to account for that.
*/
runOneTest('open', {filename:'testing2.sqlite3'}, function(ev){
//log("open result",ev);
T.assert('testing2.sqlite3'===ev.data.filename)
.assert(ev.data.messageId);
});
runOneTest('exec',{
sql: ["create table t(a,b)",
"insert into t(a,b) values(1,2),(3,4),(5,6)"
].join(';'),
multi: true,
resultRows: [], columnNames: []
}, function(ev){
ev = ev.data;
T.assert(0===ev.resultRows.length)
.assert(0===ev.columnNames.length);
});
runOneTest('exec',{
sql: 'select a a, b b from t order by a',
resultRows: [], columnNames: [],
}, function(ev){
ev = ev.data;
T.assert(3===ev.resultRows.length)
.assert(1===ev.resultRows[0][0])
.assert(6===ev.resultRows[2][1])
.assert(2===ev.columnNames.length)
.assert('b'===ev.columnNames[1]);
});
runOneTest('exec',{
sql: 'select a a, b b from t order by a',
resultRows: [], columnNames: [],
rowMode: 'object'
}, function(ev){
ev = ev.data;
T.assert(3===ev.resultRows.length)
.assert(1===ev.resultRows[0].a)
.assert(6===ev.resultRows[2].b)
});
runOneTest('exec',{sql:'intentional_error'}, mustNotReach);
// Ensure that the message-handler queue survives ^^^ that error...
runOneTest('exec',{
sql:'select 1',
resultRows: [],
//rowMode: 'array', // array is the default in the Worker interface
}, function(ev){
ev = ev.data;
T.assert(1 === ev.resultRows.length)
.assert(1 === ev.resultRows[0][0]);
});
runOneTest('exec',{
sql: 'select a a, b b from t order by a',
callback: 'resultRowTest1',
rowMode: 'object'
}, function(ev){
T.assert(3===dbMsgHandler.resultRowTest1.counter);
dbMsgHandler.resultRowTest1.counter = 0;
});
runOneTest('exec',{sql: 'delete from t where a>3'});
runOneTest('exec',{
sql: 'select count(a) from t',
resultRows: []
},function(ev){
ev = ev.data;
T.assert(1===ev.resultRows.length)
.assert(2===ev.resultRows[0][0]);
});
runOneTest('export',{}, function(ev){
ev = ev.data;
T.assert('string' === typeof ev.filename)
.assert(ev.buffer instanceof Uint8Array)
.assert(ev.buffer.length > 1024)
.assert('application/x-sqlite3' === ev.mimetype);
});
/***** close() tests must come last. *****/
runOneTest('close',{unlink:true},function(ev){
ev = ev.data;
T.assert('string' === typeof ev.filename);
});
runOneTest('close',{unlink:true},function(ev){
ev = ev.data;
T.assert(undefined === ev.filename);
});
};
SW.onmessage = function(ev){
if(!ev.data || 'object'!==typeof ev.data){
warn("Unknown sqlite3-worker message type:",ev);
return;
}
ev = ev.data/*expecting a nested object*/;
//log("main window onmessage:",ev);
if(ev.data && ev.data.messageId){
/* We're expecting a queued-up callback handler. */
const f = MsgHandlerQueue.shift();
if('error'===ev.type){
dbMsgHandler.error(ev);
return;
}
T.assert(f instanceof Function);
f(ev);
return;
}
switch(ev.type){
case 'sqlite3-api':
switch(ev.data){
case 'loaded':
log("Message:",ev); return;
case 'ready':
log("Message:",ev);
self.sqlite3TestModule.setStatus(null);
setTimeout(runTests, 0);
return;
default:
warn("Unknown sqlite3-api message type:",ev);
return;
}
default:
if(dbMsgHandler.hasOwnProperty(ev.type)){
try{dbMsgHandler[ev.type](ev);}
catch(err){
error("Exception while handling db result message",
ev,":",err);
}
return;
}
warn("Unknown sqlite3-api message type:",ev);
}
};
log("Init complete, but async init bits may still be running.");
})();
-2
View File
@@ -1,2 +0,0 @@
This folder contains source code to the first full-text search
extension for SQLite.
-404
View File
@@ -1,404 +0,0 @@
/*
** 2001 September 22
**
** The author disclaims copyright to this source code. In place of
** a legal notice, here is a blessing:
**
** May you do good and not evil.
** May you find forgiveness for yourself and forgive others.
** May you share freely, never taking more than you give.
**
*************************************************************************
** This is the implementation of generic hash-tables used in SQLite.
** We've modified it slightly to serve as a standalone hash table
** implementation for the full-text indexing module.
*/
#include <assert.h>
#include <stdlib.h>
#include <string.h>
#include "ft_hash.h"
void *malloc_and_zero(int n){
void *p = malloc(n);
if( p ){
memset(p, 0, n);
}
return p;
}
/* Turn bulk memory into a hash table object by initializing the
** fields of the Hash structure.
**
** "pNew" is a pointer to the hash table that is to be initialized.
** keyClass is one of the constants HASH_INT, HASH_POINTER,
** HASH_BINARY, or HASH_STRING. The value of keyClass
** determines what kind of key the hash table will use. "copyKey" is
** true if the hash table should make its own private copy of keys and
** false if it should just use the supplied pointer. CopyKey only makes
** sense for HASH_STRING and HASH_BINARY and is ignored
** for other key classes.
*/
void HashInit(Hash *pNew, int keyClass, int copyKey){
assert( pNew!=0 );
assert( keyClass>=HASH_STRING && keyClass<=HASH_BINARY );
pNew->keyClass = keyClass;
#if 0
if( keyClass==HASH_POINTER || keyClass==HASH_INT ) copyKey = 0;
#endif
pNew->copyKey = copyKey;
pNew->first = 0;
pNew->count = 0;
pNew->htsize = 0;
pNew->ht = 0;
pNew->xMalloc = malloc_and_zero;
pNew->xFree = free;
}
/* Remove all entries from a hash table. Reclaim all memory.
** Call this routine to delete a hash table or to reset a hash table
** to the empty state.
*/
void HashClear(Hash *pH){
HashElem *elem; /* For looping over all elements of the table */
assert( pH!=0 );
elem = pH->first;
pH->first = 0;
if( pH->ht ) pH->xFree(pH->ht);
pH->ht = 0;
pH->htsize = 0;
while( elem ){
HashElem *next_elem = elem->next;
if( pH->copyKey && elem->pKey ){
pH->xFree(elem->pKey);
}
pH->xFree(elem);
elem = next_elem;
}
pH->count = 0;
}
#if 0 /* NOT USED */
/*
** Hash and comparison functions when the mode is HASH_INT
*/
static int intHash(const void *pKey, int nKey){
return nKey ^ (nKey<<8) ^ (nKey>>8);
}
static int intCompare(const void *pKey1, int n1, const void *pKey2, int n2){
return n2 - n1;
}
#endif
#if 0 /* NOT USED */
/*
** Hash and comparison functions when the mode is HASH_POINTER
*/
static int ptrHash(const void *pKey, int nKey){
uptr x = Addr(pKey);
return x ^ (x<<8) ^ (x>>8);
}
static int ptrCompare(const void *pKey1, int n1, const void *pKey2, int n2){
if( pKey1==pKey2 ) return 0;
if( pKey1<pKey2 ) return -1;
return 1;
}
#endif
/*
** Hash and comparison functions when the mode is HASH_STRING
*/
static int strHash(const void *pKey, int nKey){
const char *z = (const char *)pKey;
int h = 0;
if( nKey<=0 ) nKey = (int) strlen(z);
while( nKey > 0 ){
h = (h<<3) ^ h ^ *z++;
nKey--;
}
return h & 0x7fffffff;
}
static int strCompare(const void *pKey1, int n1, const void *pKey2, int n2){
if( n1!=n2 ) return 1;
return strncmp((const char*)pKey1,(const char*)pKey2,n1);
}
/*
** Hash and comparison functions when the mode is HASH_BINARY
*/
static int binHash(const void *pKey, int nKey){
int h = 0;
const char *z = (const char *)pKey;
while( nKey-- > 0 ){
h = (h<<3) ^ h ^ *(z++);
}
return h & 0x7fffffff;
}
static int binCompare(const void *pKey1, int n1, const void *pKey2, int n2){
if( n1!=n2 ) return 1;
return memcmp(pKey1,pKey2,n1);
}
/*
** Return a pointer to the appropriate hash function given the key class.
**
** The C syntax in this function definition may be unfamilar to some
** programmers, so we provide the following additional explanation:
**
** The name of the function is "hashFunction". The function takes a
** single parameter "keyClass". The return value of hashFunction()
** is a pointer to another function. Specifically, the return value
** of hashFunction() is a pointer to a function that takes two parameters
** with types "const void*" and "int" and returns an "int".
*/
static int (*hashFunction(int keyClass))(const void*,int){
#if 0 /* HASH_INT and HASH_POINTER are never used */
switch( keyClass ){
case HASH_INT: return &intHash;
case HASH_POINTER: return &ptrHash;
case HASH_STRING: return &strHash;
case HASH_BINARY: return &binHash;;
default: break;
}
return 0;
#else
if( keyClass==HASH_STRING ){
return &strHash;
}else{
assert( keyClass==HASH_BINARY );
return &binHash;
}
#endif
}
/*
** Return a pointer to the appropriate hash function given the key class.
**
** For help in interpreted the obscure C code in the function definition,
** see the header comment on the previous function.
*/
static int (*compareFunction(int keyClass))(const void*,int,const void*,int){
#if 0 /* HASH_INT and HASH_POINTER are never used */
switch( keyClass ){
case HASH_INT: return &intCompare;
case HASH_POINTER: return &ptrCompare;
case HASH_STRING: return &strCompare;
case HASH_BINARY: return &binCompare;
default: break;
}
return 0;
#else
if( keyClass==HASH_STRING ){
return &strCompare;
}else{
assert( keyClass==HASH_BINARY );
return &binCompare;
}
#endif
}
/* Link an element into the hash table
*/
static void insertElement(
Hash *pH, /* The complete hash table */
struct _ht *pEntry, /* The entry into which pNew is inserted */
HashElem *pNew /* The element to be inserted */
){
HashElem *pHead; /* First element already in pEntry */
pHead = pEntry->chain;
if( pHead ){
pNew->next = pHead;
pNew->prev = pHead->prev;
if( pHead->prev ){ pHead->prev->next = pNew; }
else { pH->first = pNew; }
pHead->prev = pNew;
}else{
pNew->next = pH->first;
if( pH->first ){ pH->first->prev = pNew; }
pNew->prev = 0;
pH->first = pNew;
}
pEntry->count++;
pEntry->chain = pNew;
}
/* Resize the hash table so that it cantains "new_size" buckets.
** "new_size" must be a power of 2. The hash table might fail
** to resize if sqliteMalloc() fails.
*/
static void rehash(Hash *pH, int new_size){
struct _ht *new_ht; /* The new hash table */
HashElem *elem, *next_elem; /* For looping over existing elements */
int (*xHash)(const void*,int); /* The hash function */
assert( (new_size & (new_size-1))==0 );
new_ht = (struct _ht *)pH->xMalloc( new_size*sizeof(struct _ht) );
if( new_ht==0 ) return;
if( pH->ht ) pH->xFree(pH->ht);
pH->ht = new_ht;
pH->htsize = new_size;
xHash = hashFunction(pH->keyClass);
for(elem=pH->first, pH->first=0; elem; elem = next_elem){
int h = (*xHash)(elem->pKey, elem->nKey) & (new_size-1);
next_elem = elem->next;
insertElement(pH, &new_ht[h], elem);
}
}
/* This function (for internal use only) locates an element in an
** hash table that matches the given key. The hash for this key has
** already been computed and is passed as the 4th parameter.
*/
static HashElem *findElementGivenHash(
const Hash *pH, /* The pH to be searched */
const void *pKey, /* The key we are searching for */
int nKey,
int h /* The hash for this key. */
){
HashElem *elem; /* Used to loop thru the element list */
int count; /* Number of elements left to test */
int (*xCompare)(const void*,int,const void*,int); /* comparison function */
if( pH->ht ){
struct _ht *pEntry = &pH->ht[h];
elem = pEntry->chain;
count = pEntry->count;
xCompare = compareFunction(pH->keyClass);
while( count-- && elem ){
if( (*xCompare)(elem->pKey,elem->nKey,pKey,nKey)==0 ){
return elem;
}
elem = elem->next;
}
}
return 0;
}
/* Remove a single entry from the hash table given a pointer to that
** element and a hash on the element's key.
*/
static void removeElementGivenHash(
Hash *pH, /* The pH containing "elem" */
HashElem* elem, /* The element to be removed from the pH */
int h /* Hash value for the element */
){
struct _ht *pEntry;
if( elem->prev ){
elem->prev->next = elem->next;
}else{
pH->first = elem->next;
}
if( elem->next ){
elem->next->prev = elem->prev;
}
pEntry = &pH->ht[h];
if( pEntry->chain==elem ){
pEntry->chain = elem->next;
}
pEntry->count--;
if( pEntry->count<=0 ){
pEntry->chain = 0;
}
if( pH->copyKey && elem->pKey ){
pH->xFree(elem->pKey);
}
pH->xFree( elem );
pH->count--;
if( pH->count<=0 ){
assert( pH->first==0 );
assert( pH->count==0 );
HashClear(pH);
}
}
/* Attempt to locate an element of the hash table pH with a key
** that matches pKey,nKey. Return the data for this element if it is
** found, or NULL if there is no match.
*/
void *HashFind(const Hash *pH, const void *pKey, int nKey){
int h; /* A hash on key */
HashElem *elem; /* The element that matches key */
int (*xHash)(const void*,int); /* The hash function */
if( pH==0 || pH->ht==0 ) return 0;
xHash = hashFunction(pH->keyClass);
assert( xHash!=0 );
h = (*xHash)(pKey,nKey);
assert( (pH->htsize & (pH->htsize-1))==0 );
elem = findElementGivenHash(pH,pKey,nKey, h & (pH->htsize-1));
return elem ? elem->data : 0;
}
/* Insert an element into the hash table pH. The key is pKey,nKey
** and the data is "data".
**
** If no element exists with a matching key, then a new
** element is created. A copy of the key is made if the copyKey
** flag is set. NULL is returned.
**
** If another element already exists with the same key, then the
** new data replaces the old data and the old data is returned.
** The key is not copied in this instance. If a malloc fails, then
** the new data is returned and the hash table is unchanged.
**
** If the "data" parameter to this function is NULL, then the
** element corresponding to "key" is removed from the hash table.
*/
void *HashInsert(Hash *pH, const void *pKey, int nKey, void *data){
int hraw; /* Raw hash value of the key */
int h; /* the hash of the key modulo hash table size */
HashElem *elem; /* Used to loop thru the element list */
HashElem *new_elem; /* New element added to the pH */
int (*xHash)(const void*,int); /* The hash function */
assert( pH!=0 );
xHash = hashFunction(pH->keyClass);
assert( xHash!=0 );
hraw = (*xHash)(pKey, nKey);
assert( (pH->htsize & (pH->htsize-1))==0 );
h = hraw & (pH->htsize-1);
elem = findElementGivenHash(pH,pKey,nKey,h);
if( elem ){
void *old_data = elem->data;
if( data==0 ){
removeElementGivenHash(pH,elem,h);
}else{
elem->data = data;
}
return old_data;
}
if( data==0 ) return 0;
new_elem = (HashElem*)pH->xMalloc( sizeof(HashElem) );
if( new_elem==0 ) return data;
if( pH->copyKey && pKey!=0 ){
new_elem->pKey = pH->xMalloc( nKey );
if( new_elem->pKey==0 ){
pH->xFree(new_elem);
return data;
}
memcpy((void*)new_elem->pKey, pKey, nKey);
}else{
new_elem->pKey = (void*)pKey;
}
new_elem->nKey = nKey;
pH->count++;
if( pH->htsize==0 ){
rehash(pH,8);
if( pH->htsize==0 ){
pH->count = 0;
pH->xFree(new_elem);
return data;
}
}
if( pH->count > pH->htsize ){
rehash(pH,pH->htsize*2);
}
assert( pH->htsize>0 );
assert( (pH->htsize & (pH->htsize-1))==0 );
h = hraw & (pH->htsize-1);
insertElement(pH, &pH->ht[h], new_elem);
new_elem->data = data;
return 0;
}
-111
View File
@@ -1,111 +0,0 @@
/*
** 2001 September 22
**
** The author disclaims copyright to this source code. In place of
** a legal notice, here is a blessing:
**
** May you do good and not evil.
** May you find forgiveness for yourself and forgive others.
** May you share freely, never taking more than you give.
**
*************************************************************************
** This is the header file for the generic hash-table implementation
** used in SQLite. We've modified it slightly to serve as a standalone
** hash table implementation for the full-text indexing module.
**
*/
#ifndef _HASH_H_
#define _HASH_H_
/* Forward declarations of structures. */
typedef struct Hash Hash;
typedef struct HashElem HashElem;
/* A complete hash table is an instance of the following structure.
** The internals of this structure are intended to be opaque -- client
** code should not attempt to access or modify the fields of this structure
** directly. Change this structure only by using the routines below.
** However, many of the "procedures" and "functions" for modifying and
** accessing this structure are really macros, so we can't really make
** this structure opaque.
*/
struct Hash {
char keyClass; /* HASH_INT, _POINTER, _STRING, _BINARY */
char copyKey; /* True if copy of key made on insert */
int count; /* Number of entries in this table */
HashElem *first; /* The first element of the array */
void *(*xMalloc)(int); /* malloc() function to use */
void (*xFree)(void *); /* free() function to use */
int htsize; /* Number of buckets in the hash table */
struct _ht { /* the hash table */
int count; /* Number of entries with this hash */
HashElem *chain; /* Pointer to first entry with this hash */
} *ht;
};
/* Each element in the hash table is an instance of the following
** structure. All elements are stored on a single doubly-linked list.
**
** Again, this structure is intended to be opaque, but it can't really
** be opaque because it is used by macros.
*/
struct HashElem {
HashElem *next, *prev; /* Next and previous elements in the table */
void *data; /* Data associated with this element */
void *pKey; int nKey; /* Key associated with this element */
};
/*
** There are 4 different modes of operation for a hash table:
**
** HASH_INT nKey is used as the key and pKey is ignored.
**
** HASH_POINTER pKey is used as the key and nKey is ignored.
**
** HASH_STRING pKey points to a string that is nKey bytes long
** (including the null-terminator, if any). Case
** is respected in comparisons.
**
** HASH_BINARY pKey points to binary data nKey bytes long.
** memcmp() is used to compare keys.
**
** A copy of the key is made for HASH_STRING and HASH_BINARY
** if the copyKey parameter to HashInit is 1.
*/
/* #define HASH_INT 1 // NOT USED */
/* #define HASH_POINTER 2 // NOT USED */
#define HASH_STRING 3
#define HASH_BINARY 4
/*
** Access routines. To delete, insert a NULL pointer.
*/
void HashInit(Hash*, int keytype, int copyKey);
void *HashInsert(Hash*, const void *pKey, int nKey, void *pData);
void *HashFind(const Hash*, const void *pKey, int nKey);
void HashClear(Hash*);
/*
** Macros for looping over all elements of a hash table. The idiom is
** like this:
**
** Hash h;
** HashElem *p;
** ...
** for(p=HashFirst(&h); p; p=HashNext(p)){
** SomeStructure *pData = HashData(p);
** // do something with pData
** }
*/
#define HashFirst(H) ((H)->first)
#define HashNext(E) ((E)->next)
#define HashData(E) ((E)->data)
#define HashKey(E) ((E)->pKey)
#define HashKeysize(E) ((E)->nKey)
/*
** Number of entries in a hash table
*/
#define HashCount(H) ((H)->count)
#endif /* _HASH_H_ */
-3348
View File
File diff suppressed because it is too large Load Diff
-11
View File
@@ -1,11 +0,0 @@
#include "sqlite3.h"
#ifdef __cplusplus
extern "C" {
#endif /* __cplusplus */
int sqlite3Fts1Init(sqlite3 *db);
#ifdef __cplusplus
} /* extern "C" */
#endif /* __cplusplus */
-369
View File
@@ -1,369 +0,0 @@
/*
** 2001 September 22
**
** The author disclaims copyright to this source code. In place of
** a legal notice, here is a blessing:
**
** May you do good and not evil.
** May you find forgiveness for yourself and forgive others.
** May you share freely, never taking more than you give.
**
*************************************************************************
** This is the implementation of generic hash-tables used in SQLite.
** We've modified it slightly to serve as a standalone hash table
** implementation for the full-text indexing module.
*/
#include <assert.h>
#include <stdlib.h>
#include <string.h>
/*
** The code in this file is only compiled if:
**
** * The FTS1 module is being built as an extension
** (in which case SQLITE_CORE is not defined), or
**
** * The FTS1 module is being built into the core of
** SQLite (in which case SQLITE_ENABLE_FTS1 is defined).
*/
#if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS1)
#include "fts1_hash.h"
static void *malloc_and_zero(int n){
void *p = malloc(n);
if( p ){
memset(p, 0, n);
}
return p;
}
/* Turn bulk memory into a hash table object by initializing the
** fields of the Hash structure.
**
** "pNew" is a pointer to the hash table that is to be initialized.
** keyClass is one of the constants
** FTS1_HASH_BINARY or FTS1_HASH_STRING. The value of keyClass
** determines what kind of key the hash table will use. "copyKey" is
** true if the hash table should make its own private copy of keys and
** false if it should just use the supplied pointer.
*/
void sqlite3Fts1HashInit(fts1Hash *pNew, int keyClass, int copyKey){
assert( pNew!=0 );
assert( keyClass>=FTS1_HASH_STRING && keyClass<=FTS1_HASH_BINARY );
pNew->keyClass = keyClass;
pNew->copyKey = copyKey;
pNew->first = 0;
pNew->count = 0;
pNew->htsize = 0;
pNew->ht = 0;
pNew->xMalloc = malloc_and_zero;
pNew->xFree = free;
}
/* Remove all entries from a hash table. Reclaim all memory.
** Call this routine to delete a hash table or to reset a hash table
** to the empty state.
*/
void sqlite3Fts1HashClear(fts1Hash *pH){
fts1HashElem *elem; /* For looping over all elements of the table */
assert( pH!=0 );
elem = pH->first;
pH->first = 0;
if( pH->ht ) pH->xFree(pH->ht);
pH->ht = 0;
pH->htsize = 0;
while( elem ){
fts1HashElem *next_elem = elem->next;
if( pH->copyKey && elem->pKey ){
pH->xFree(elem->pKey);
}
pH->xFree(elem);
elem = next_elem;
}
pH->count = 0;
}
/*
** Hash and comparison functions when the mode is FTS1_HASH_STRING
*/
static int strHash(const void *pKey, int nKey){
const char *z = (const char *)pKey;
int h = 0;
if( nKey<=0 ) nKey = (int) strlen(z);
while( nKey > 0 ){
h = (h<<3) ^ h ^ *z++;
nKey--;
}
return h & 0x7fffffff;
}
static int strCompare(const void *pKey1, int n1, const void *pKey2, int n2){
if( n1!=n2 ) return 1;
return strncmp((const char*)pKey1,(const char*)pKey2,n1);
}
/*
** Hash and comparison functions when the mode is FTS1_HASH_BINARY
*/
static int binHash(const void *pKey, int nKey){
int h = 0;
const char *z = (const char *)pKey;
while( nKey-- > 0 ){
h = (h<<3) ^ h ^ *(z++);
}
return h & 0x7fffffff;
}
static int binCompare(const void *pKey1, int n1, const void *pKey2, int n2){
if( n1!=n2 ) return 1;
return memcmp(pKey1,pKey2,n1);
}
/*
** Return a pointer to the appropriate hash function given the key class.
**
** The C syntax in this function definition may be unfamilar to some
** programmers, so we provide the following additional explanation:
**
** The name of the function is "hashFunction". The function takes a
** single parameter "keyClass". The return value of hashFunction()
** is a pointer to another function. Specifically, the return value
** of hashFunction() is a pointer to a function that takes two parameters
** with types "const void*" and "int" and returns an "int".
*/
static int (*hashFunction(int keyClass))(const void*,int){
if( keyClass==FTS1_HASH_STRING ){
return &strHash;
}else{
assert( keyClass==FTS1_HASH_BINARY );
return &binHash;
}
}
/*
** Return a pointer to the appropriate hash function given the key class.
**
** For help in interpreted the obscure C code in the function definition,
** see the header comment on the previous function.
*/
static int (*compareFunction(int keyClass))(const void*,int,const void*,int){
if( keyClass==FTS1_HASH_STRING ){
return &strCompare;
}else{
assert( keyClass==FTS1_HASH_BINARY );
return &binCompare;
}
}
/* Link an element into the hash table
*/
static void insertElement(
fts1Hash *pH, /* The complete hash table */
struct _fts1ht *pEntry, /* The entry into which pNew is inserted */
fts1HashElem *pNew /* The element to be inserted */
){
fts1HashElem *pHead; /* First element already in pEntry */
pHead = pEntry->chain;
if( pHead ){
pNew->next = pHead;
pNew->prev = pHead->prev;
if( pHead->prev ){ pHead->prev->next = pNew; }
else { pH->first = pNew; }
pHead->prev = pNew;
}else{
pNew->next = pH->first;
if( pH->first ){ pH->first->prev = pNew; }
pNew->prev = 0;
pH->first = pNew;
}
pEntry->count++;
pEntry->chain = pNew;
}
/* Resize the hash table so that it cantains "new_size" buckets.
** "new_size" must be a power of 2. The hash table might fail
** to resize if sqliteMalloc() fails.
*/
static void rehash(fts1Hash *pH, int new_size){
struct _fts1ht *new_ht; /* The new hash table */
fts1HashElem *elem, *next_elem; /* For looping over existing elements */
int (*xHash)(const void*,int); /* The hash function */
assert( (new_size & (new_size-1))==0 );
new_ht = (struct _fts1ht *)pH->xMalloc( new_size*sizeof(struct _fts1ht) );
if( new_ht==0 ) return;
if( pH->ht ) pH->xFree(pH->ht);
pH->ht = new_ht;
pH->htsize = new_size;
xHash = hashFunction(pH->keyClass);
for(elem=pH->first, pH->first=0; elem; elem = next_elem){
int h = (*xHash)(elem->pKey, elem->nKey) & (new_size-1);
next_elem = elem->next;
insertElement(pH, &new_ht[h], elem);
}
}
/* This function (for internal use only) locates an element in an
** hash table that matches the given key. The hash for this key has
** already been computed and is passed as the 4th parameter.
*/
static fts1HashElem *findElementGivenHash(
const fts1Hash *pH, /* The pH to be searched */
const void *pKey, /* The key we are searching for */
int nKey,
int h /* The hash for this key. */
){
fts1HashElem *elem; /* Used to loop thru the element list */
int count; /* Number of elements left to test */
int (*xCompare)(const void*,int,const void*,int); /* comparison function */
if( pH->ht ){
struct _fts1ht *pEntry = &pH->ht[h];
elem = pEntry->chain;
count = pEntry->count;
xCompare = compareFunction(pH->keyClass);
while( count-- && elem ){
if( (*xCompare)(elem->pKey,elem->nKey,pKey,nKey)==0 ){
return elem;
}
elem = elem->next;
}
}
return 0;
}
/* Remove a single entry from the hash table given a pointer to that
** element and a hash on the element's key.
*/
static void removeElementGivenHash(
fts1Hash *pH, /* The pH containing "elem" */
fts1HashElem* elem, /* The element to be removed from the pH */
int h /* Hash value for the element */
){
struct _fts1ht *pEntry;
if( elem->prev ){
elem->prev->next = elem->next;
}else{
pH->first = elem->next;
}
if( elem->next ){
elem->next->prev = elem->prev;
}
pEntry = &pH->ht[h];
if( pEntry->chain==elem ){
pEntry->chain = elem->next;
}
pEntry->count--;
if( pEntry->count<=0 ){
pEntry->chain = 0;
}
if( pH->copyKey && elem->pKey ){
pH->xFree(elem->pKey);
}
pH->xFree( elem );
pH->count--;
if( pH->count<=0 ){
assert( pH->first==0 );
assert( pH->count==0 );
fts1HashClear(pH);
}
}
/* Attempt to locate an element of the hash table pH with a key
** that matches pKey,nKey. Return the data for this element if it is
** found, or NULL if there is no match.
*/
void *sqlite3Fts1HashFind(const fts1Hash *pH, const void *pKey, int nKey){
int h; /* A hash on key */
fts1HashElem *elem; /* The element that matches key */
int (*xHash)(const void*,int); /* The hash function */
if( pH==0 || pH->ht==0 ) return 0;
xHash = hashFunction(pH->keyClass);
assert( xHash!=0 );
h = (*xHash)(pKey,nKey);
assert( (pH->htsize & (pH->htsize-1))==0 );
elem = findElementGivenHash(pH,pKey,nKey, h & (pH->htsize-1));
return elem ? elem->data : 0;
}
/* Insert an element into the hash table pH. The key is pKey,nKey
** and the data is "data".
**
** If no element exists with a matching key, then a new
** element is created. A copy of the key is made if the copyKey
** flag is set. NULL is returned.
**
** If another element already exists with the same key, then the
** new data replaces the old data and the old data is returned.
** The key is not copied in this instance. If a malloc fails, then
** the new data is returned and the hash table is unchanged.
**
** If the "data" parameter to this function is NULL, then the
** element corresponding to "key" is removed from the hash table.
*/
void *sqlite3Fts1HashInsert(
fts1Hash *pH, /* The hash table to insert into */
const void *pKey, /* The key */
int nKey, /* Number of bytes in the key */
void *data /* The data */
){
int hraw; /* Raw hash value of the key */
int h; /* the hash of the key modulo hash table size */
fts1HashElem *elem; /* Used to loop thru the element list */
fts1HashElem *new_elem; /* New element added to the pH */
int (*xHash)(const void*,int); /* The hash function */
assert( pH!=0 );
xHash = hashFunction(pH->keyClass);
assert( xHash!=0 );
hraw = (*xHash)(pKey, nKey);
assert( (pH->htsize & (pH->htsize-1))==0 );
h = hraw & (pH->htsize-1);
elem = findElementGivenHash(pH,pKey,nKey,h);
if( elem ){
void *old_data = elem->data;
if( data==0 ){
removeElementGivenHash(pH,elem,h);
}else{
elem->data = data;
}
return old_data;
}
if( data==0 ) return 0;
new_elem = (fts1HashElem*)pH->xMalloc( sizeof(fts1HashElem) );
if( new_elem==0 ) return data;
if( pH->copyKey && pKey!=0 ){
new_elem->pKey = pH->xMalloc( nKey );
if( new_elem->pKey==0 ){
pH->xFree(new_elem);
return data;
}
memcpy((void*)new_elem->pKey, pKey, nKey);
}else{
new_elem->pKey = (void*)pKey;
}
new_elem->nKey = nKey;
pH->count++;
if( pH->htsize==0 ){
rehash(pH,8);
if( pH->htsize==0 ){
pH->count = 0;
pH->xFree(new_elem);
return data;
}
}
if( pH->count > pH->htsize ){
rehash(pH,pH->htsize*2);
}
assert( pH->htsize>0 );
assert( (pH->htsize & (pH->htsize-1))==0 );
h = hraw & (pH->htsize-1);
insertElement(pH, &pH->ht[h], new_elem);
new_elem->data = data;
return 0;
}
#endif /* !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS1) */
-112
View File
@@ -1,112 +0,0 @@
/*
** 2001 September 22
**
** The author disclaims copyright to this source code. In place of
** a legal notice, here is a blessing:
**
** May you do good and not evil.
** May you find forgiveness for yourself and forgive others.
** May you share freely, never taking more than you give.
**
*************************************************************************
** This is the header file for the generic hash-table implementation
** used in SQLite. We've modified it slightly to serve as a standalone
** hash table implementation for the full-text indexing module.
**
*/
#ifndef _FTS1_HASH_H_
#define _FTS1_HASH_H_
/* Forward declarations of structures. */
typedef struct fts1Hash fts1Hash;
typedef struct fts1HashElem fts1HashElem;
/* A complete hash table is an instance of the following structure.
** The internals of this structure are intended to be opaque -- client
** code should not attempt to access or modify the fields of this structure
** directly. Change this structure only by using the routines below.
** However, many of the "procedures" and "functions" for modifying and
** accessing this structure are really macros, so we can't really make
** this structure opaque.
*/
struct fts1Hash {
char keyClass; /* HASH_INT, _POINTER, _STRING, _BINARY */
char copyKey; /* True if copy of key made on insert */
int count; /* Number of entries in this table */
fts1HashElem *first; /* The first element of the array */
void *(*xMalloc)(int); /* malloc() function to use */
void (*xFree)(void *); /* free() function to use */
int htsize; /* Number of buckets in the hash table */
struct _fts1ht { /* the hash table */
int count; /* Number of entries with this hash */
fts1HashElem *chain; /* Pointer to first entry with this hash */
} *ht;
};
/* Each element in the hash table is an instance of the following
** structure. All elements are stored on a single doubly-linked list.
**
** Again, this structure is intended to be opaque, but it can't really
** be opaque because it is used by macros.
*/
struct fts1HashElem {
fts1HashElem *next, *prev; /* Next and previous elements in the table */
void *data; /* Data associated with this element */
void *pKey; int nKey; /* Key associated with this element */
};
/*
** There are 2 different modes of operation for a hash table:
**
** FTS1_HASH_STRING pKey points to a string that is nKey bytes long
** (including the null-terminator, if any). Case
** is respected in comparisons.
**
** FTS1_HASH_BINARY pKey points to binary data nKey bytes long.
** memcmp() is used to compare keys.
**
** A copy of the key is made if the copyKey parameter to fts1HashInit is 1.
*/
#define FTS1_HASH_STRING 1
#define FTS1_HASH_BINARY 2
/*
** Access routines. To delete, insert a NULL pointer.
*/
void sqlite3Fts1HashInit(fts1Hash*, int keytype, int copyKey);
void *sqlite3Fts1HashInsert(fts1Hash*, const void *pKey, int nKey, void *pData);
void *sqlite3Fts1HashFind(const fts1Hash*, const void *pKey, int nKey);
void sqlite3Fts1HashClear(fts1Hash*);
/*
** Shorthand for the functions above
*/
#define fts1HashInit sqlite3Fts1HashInit
#define fts1HashInsert sqlite3Fts1HashInsert
#define fts1HashFind sqlite3Fts1HashFind
#define fts1HashClear sqlite3Fts1HashClear
/*
** Macros for looping over all elements of a hash table. The idiom is
** like this:
**
** fts1Hash h;
** fts1HashElem *p;
** ...
** for(p=fts1HashFirst(&h); p; p=fts1HashNext(p)){
** SomeStructure *pData = fts1HashData(p);
** // do something with pData
** }
*/
#define fts1HashFirst(H) ((H)->first)
#define fts1HashNext(E) ((E)->next)
#define fts1HashData(E) ((E)->data)
#define fts1HashKey(E) ((E)->pKey)
#define fts1HashKeysize(E) ((E)->nKey)
/*
** Number of entries in a hash table
*/
#define fts1HashCount(H) ((H)->count)
#endif /* _FTS1_HASH_H_ */
-643
View File
@@ -1,643 +0,0 @@
/*
** 2006 September 30
**
** The author disclaims copyright to this source code. In place of
** a legal notice, here is a blessing:
**
** May you do good and not evil.
** May you find forgiveness for yourself and forgive others.
** May you share freely, never taking more than you give.
**
*************************************************************************
** Implementation of the full-text-search tokenizer that implements
** a Porter stemmer.
*/
/*
** The code in this file is only compiled if:
**
** * The FTS1 module is being built as an extension
** (in which case SQLITE_CORE is not defined), or
**
** * The FTS1 module is being built into the core of
** SQLite (in which case SQLITE_ENABLE_FTS1 is defined).
*/
#if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS1)
#include <assert.h>
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <ctype.h>
#include "fts1_tokenizer.h"
/*
** Class derived from sqlite3_tokenizer
*/
typedef struct porter_tokenizer {
sqlite3_tokenizer base; /* Base class */
} porter_tokenizer;
/*
** Class derived from sqlit3_tokenizer_cursor
*/
typedef struct porter_tokenizer_cursor {
sqlite3_tokenizer_cursor base;
const char *zInput; /* input we are tokenizing */
int nInput; /* size of the input */
int iOffset; /* current position in zInput */
int iToken; /* index of next token to be returned */
char *zToken; /* storage for current token */
int nAllocated; /* space allocated to zToken buffer */
} porter_tokenizer_cursor;
/* Forward declaration */
static const sqlite3_tokenizer_module porterTokenizerModule;
/*
** Create a new tokenizer instance.
*/
static int porterCreate(
int argc, const char * const *argv,
sqlite3_tokenizer **ppTokenizer
){
porter_tokenizer *t;
t = (porter_tokenizer *) calloc(sizeof(*t), 1);
if( t==NULL ) return SQLITE_NOMEM;
*ppTokenizer = &t->base;
return SQLITE_OK;
}
/*
** Destroy a tokenizer
*/
static int porterDestroy(sqlite3_tokenizer *pTokenizer){
free(pTokenizer);
return SQLITE_OK;
}
/*
** Prepare to begin tokenizing a particular string. The input
** string to be tokenized is zInput[0..nInput-1]. A cursor
** used to incrementally tokenize this string is returned in
** *ppCursor.
*/
static int porterOpen(
sqlite3_tokenizer *pTokenizer, /* The tokenizer */
const char *zInput, int nInput, /* String to be tokenized */
sqlite3_tokenizer_cursor **ppCursor /* OUT: Tokenization cursor */
){
porter_tokenizer_cursor *c;
c = (porter_tokenizer_cursor *) malloc(sizeof(*c));
if( c==NULL ) return SQLITE_NOMEM;
c->zInput = zInput;
if( zInput==0 ){
c->nInput = 0;
}else if( nInput<0 ){
c->nInput = (int)strlen(zInput);
}else{
c->nInput = nInput;
}
c->iOffset = 0; /* start tokenizing at the beginning */
c->iToken = 0;
c->zToken = NULL; /* no space allocated, yet. */
c->nAllocated = 0;
*ppCursor = &c->base;
return SQLITE_OK;
}
/*
** Close a tokenization cursor previously opened by a call to
** porterOpen() above.
*/
static int porterClose(sqlite3_tokenizer_cursor *pCursor){
porter_tokenizer_cursor *c = (porter_tokenizer_cursor *) pCursor;
free(c->zToken);
free(c);
return SQLITE_OK;
}
/*
** Vowel or consonant
*/
static const char cType[] = {
0, 1, 1, 1, 0, 1, 1, 1, 0, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 0,
1, 1, 1, 2, 1
};
/*
** isConsonant() and isVowel() determine if their first character in
** the string they point to is a consonant or a vowel, according
** to Porter ruls.
**
** A consonate is any letter other than 'a', 'e', 'i', 'o', or 'u'.
** 'Y' is a consonant unless it follows another consonant,
** in which case it is a vowel.
**
** In these routine, the letters are in reverse order. So the 'y' rule
** is that 'y' is a consonant unless it is followed by another
** consonent.
*/
static int isVowel(const char*);
static int isConsonant(const char *z){
int j;
char x = *z;
if( x==0 ) return 0;
assert( x>='a' && x<='z' );
j = cType[x-'a'];
if( j<2 ) return j;
return z[1]==0 || isVowel(z + 1);
}
static int isVowel(const char *z){
int j;
char x = *z;
if( x==0 ) return 0;
assert( x>='a' && x<='z' );
j = cType[x-'a'];
if( j<2 ) return 1-j;
return isConsonant(z + 1);
}
/*
** Let any sequence of one or more vowels be represented by V and let
** C be sequence of one or more consonants. Then every word can be
** represented as:
**
** [C] (VC){m} [V]
**
** In prose: A word is an optional consonant followed by zero or
** vowel-consonant pairs followed by an optional vowel. "m" is the
** number of vowel consonant pairs. This routine computes the value
** of m for the first i bytes of a word.
**
** Return true if the m-value for z is 1 or more. In other words,
** return true if z contains at least one vowel that is followed
** by a consonant.
**
** In this routine z[] is in reverse order. So we are really looking
** for an instance of of a consonant followed by a vowel.
*/
static int m_gt_0(const char *z){
while( isVowel(z) ){ z++; }
if( *z==0 ) return 0;
while( isConsonant(z) ){ z++; }
return *z!=0;
}
/* Like mgt0 above except we are looking for a value of m which is
** exactly 1
*/
static int m_eq_1(const char *z){
while( isVowel(z) ){ z++; }
if( *z==0 ) return 0;
while( isConsonant(z) ){ z++; }
if( *z==0 ) return 0;
while( isVowel(z) ){ z++; }
if( *z==0 ) return 1;
while( isConsonant(z) ){ z++; }
return *z==0;
}
/* Like mgt0 above except we are looking for a value of m>1 instead
** or m>0
*/
static int m_gt_1(const char *z){
while( isVowel(z) ){ z++; }
if( *z==0 ) return 0;
while( isConsonant(z) ){ z++; }
if( *z==0 ) return 0;
while( isVowel(z) ){ z++; }
if( *z==0 ) return 0;
while( isConsonant(z) ){ z++; }
return *z!=0;
}
/*
** Return TRUE if there is a vowel anywhere within z[0..n-1]
*/
static int hasVowel(const char *z){
while( isConsonant(z) ){ z++; }
return *z!=0;
}
/*
** Return TRUE if the word ends in a double consonant.
**
** The text is reversed here. So we are really looking at
** the first two characters of z[].
*/
static int doubleConsonant(const char *z){
return isConsonant(z) && z[0]==z[1] && isConsonant(z+1);
}
/*
** Return TRUE if the word ends with three letters which
** are consonant-vowel-consonent and where the final consonant
** is not 'w', 'x', or 'y'.
**
** The word is reversed here. So we are really checking the
** first three letters and the first one cannot be in [wxy].
*/
static int star_oh(const char *z){
return
z[0]!=0 && isConsonant(z) &&
z[0]!='w' && z[0]!='x' && z[0]!='y' &&
z[1]!=0 && isVowel(z+1) &&
z[2]!=0 && isConsonant(z+2);
}
/*
** If the word ends with zFrom and xCond() is true for the stem
** of the word that preceeds the zFrom ending, then change the
** ending to zTo.
**
** The input word *pz and zFrom are both in reverse order. zTo
** is in normal order.
**
** Return TRUE if zFrom matches. Return FALSE if zFrom does not
** match. Not that TRUE is returned even if xCond() fails and
** no substitution occurs.
*/
static int stem(
char **pz, /* The word being stemmed (Reversed) */
const char *zFrom, /* If the ending matches this... (Reversed) */
const char *zTo, /* ... change the ending to this (not reversed) */
int (*xCond)(const char*) /* Condition that must be true */
){
char *z = *pz;
while( *zFrom && *zFrom==*z ){ z++; zFrom++; }
if( *zFrom!=0 ) return 0;
if( xCond && !xCond(z) ) return 1;
while( *zTo ){
*(--z) = *(zTo++);
}
*pz = z;
return 1;
}
/*
** This is the fallback stemmer used when the porter stemmer is
** inappropriate. The input word is copied into the output with
** US-ASCII case folding. If the input word is too long (more
** than 20 bytes if it contains no digits or more than 6 bytes if
** it contains digits) then word is truncated to 20 or 6 bytes
** by taking 10 or 3 bytes from the beginning and end.
*/
static void copy_stemmer(const char *zIn, int nIn, char *zOut, int *pnOut){
int i, mx, j;
int hasDigit = 0;
for(i=0; i<nIn; i++){
int c = zIn[i];
if( c>='A' && c<='Z' ){
zOut[i] = c - 'A' + 'a';
}else{
if( c>='0' && c<='9' ) hasDigit = 1;
zOut[i] = c;
}
}
mx = hasDigit ? 3 : 10;
if( nIn>mx*2 ){
for(j=mx, i=nIn-mx; i<nIn; i++, j++){
zOut[j] = zOut[i];
}
i = j;
}
zOut[i] = 0;
*pnOut = i;
}
/*
** Stem the input word zIn[0..nIn-1]. Store the output in zOut.
** zOut is at least big enough to hold nIn bytes. Write the actual
** size of the output word (exclusive of the '\0' terminator) into *pnOut.
**
** Any upper-case characters in the US-ASCII character set ([A-Z])
** are converted to lower case. Upper-case UTF characters are
** unchanged.
**
** Words that are longer than about 20 bytes are stemmed by retaining
** a few bytes from the beginning and the end of the word. If the
** word contains digits, 3 bytes are taken from the beginning and
** 3 bytes from the end. For long words without digits, 10 bytes
** are taken from each end. US-ASCII case folding still applies.
**
** If the input word contains not digits but does characters not
** in [a-zA-Z] then no stemming is attempted and this routine just
** copies the input into the input into the output with US-ASCII
** case folding.
**
** Stemming never increases the length of the word. So there is
** no chance of overflowing the zOut buffer.
*/
static void porter_stemmer(const char *zIn, int nIn, char *zOut, int *pnOut){
int i, j, c;
char zReverse[28];
char *z, *z2;
if( nIn<3 || nIn>=sizeof(zReverse)-7 ){
/* The word is too big or too small for the porter stemmer.
** Fallback to the copy stemmer */
copy_stemmer(zIn, nIn, zOut, pnOut);
return;
}
for(i=0, j=sizeof(zReverse)-6; i<nIn; i++, j--){
c = zIn[i];
if( c>='A' && c<='Z' ){
zReverse[j] = c + 'a' - 'A';
}else if( c>='a' && c<='z' ){
zReverse[j] = c;
}else{
/* The use of a character not in [a-zA-Z] means that we fallback
** to the copy stemmer */
copy_stemmer(zIn, nIn, zOut, pnOut);
return;
}
}
memset(&zReverse[sizeof(zReverse)-5], 0, 5);
z = &zReverse[j+1];
/* Step 1a */
if( z[0]=='s' ){
if(
!stem(&z, "sess", "ss", 0) &&
!stem(&z, "sei", "i", 0) &&
!stem(&z, "ss", "ss", 0)
){
z++;
}
}
/* Step 1b */
z2 = z;
if( stem(&z, "dee", "ee", m_gt_0) ){
/* Do nothing. The work was all in the test */
}else if(
(stem(&z, "gni", "", hasVowel) || stem(&z, "de", "", hasVowel))
&& z!=z2
){
if( stem(&z, "ta", "ate", 0) ||
stem(&z, "lb", "ble", 0) ||
stem(&z, "zi", "ize", 0) ){
/* Do nothing. The work was all in the test */
}else if( doubleConsonant(z) && (*z!='l' && *z!='s' && *z!='z') ){
z++;
}else if( m_eq_1(z) && star_oh(z) ){
*(--z) = 'e';
}
}
/* Step 1c */
if( z[0]=='y' && hasVowel(z+1) ){
z[0] = 'i';
}
/* Step 2 */
switch( z[1] ){
case 'a':
stem(&z, "lanoita", "ate", m_gt_0) ||
stem(&z, "lanoit", "tion", m_gt_0);
break;
case 'c':
stem(&z, "icne", "ence", m_gt_0) ||
stem(&z, "icna", "ance", m_gt_0);
break;
case 'e':
stem(&z, "rezi", "ize", m_gt_0);
break;
case 'g':
stem(&z, "igol", "log", m_gt_0);
break;
case 'l':
stem(&z, "ilb", "ble", m_gt_0) ||
stem(&z, "illa", "al", m_gt_0) ||
stem(&z, "iltne", "ent", m_gt_0) ||
stem(&z, "ile", "e", m_gt_0) ||
stem(&z, "ilsuo", "ous", m_gt_0);
break;
case 'o':
stem(&z, "noitazi", "ize", m_gt_0) ||
stem(&z, "noita", "ate", m_gt_0) ||
stem(&z, "rota", "ate", m_gt_0);
break;
case 's':
stem(&z, "msila", "al", m_gt_0) ||
stem(&z, "ssenevi", "ive", m_gt_0) ||
stem(&z, "ssenluf", "ful", m_gt_0) ||
stem(&z, "ssensuo", "ous", m_gt_0);
break;
case 't':
stem(&z, "itila", "al", m_gt_0) ||
stem(&z, "itivi", "ive", m_gt_0) ||
stem(&z, "itilib", "ble", m_gt_0);
break;
}
/* Step 3 */
switch( z[0] ){
case 'e':
stem(&z, "etaci", "ic", m_gt_0) ||
stem(&z, "evita", "", m_gt_0) ||
stem(&z, "ezila", "al", m_gt_0);
break;
case 'i':
stem(&z, "itici", "ic", m_gt_0);
break;
case 'l':
stem(&z, "laci", "ic", m_gt_0) ||
stem(&z, "luf", "", m_gt_0);
break;
case 's':
stem(&z, "ssen", "", m_gt_0);
break;
}
/* Step 4 */
switch( z[1] ){
case 'a':
if( z[0]=='l' && m_gt_1(z+2) ){
z += 2;
}
break;
case 'c':
if( z[0]=='e' && z[2]=='n' && (z[3]=='a' || z[3]=='e') && m_gt_1(z+4) ){
z += 4;
}
break;
case 'e':
if( z[0]=='r' && m_gt_1(z+2) ){
z += 2;
}
break;
case 'i':
if( z[0]=='c' && m_gt_1(z+2) ){
z += 2;
}
break;
case 'l':
if( z[0]=='e' && z[2]=='b' && (z[3]=='a' || z[3]=='i') && m_gt_1(z+4) ){
z += 4;
}
break;
case 'n':
if( z[0]=='t' ){
if( z[2]=='a' ){
if( m_gt_1(z+3) ){
z += 3;
}
}else if( z[2]=='e' ){
stem(&z, "tneme", "", m_gt_1) ||
stem(&z, "tnem", "", m_gt_1) ||
stem(&z, "tne", "", m_gt_1);
}
}
break;
case 'o':
if( z[0]=='u' ){
if( m_gt_1(z+2) ){
z += 2;
}
}else if( z[3]=='s' || z[3]=='t' ){
stem(&z, "noi", "", m_gt_1);
}
break;
case 's':
if( z[0]=='m' && z[2]=='i' && m_gt_1(z+3) ){
z += 3;
}
break;
case 't':
stem(&z, "eta", "", m_gt_1) ||
stem(&z, "iti", "", m_gt_1);
break;
case 'u':
if( z[0]=='s' && z[2]=='o' && m_gt_1(z+3) ){
z += 3;
}
break;
case 'v':
case 'z':
if( z[0]=='e' && z[2]=='i' && m_gt_1(z+3) ){
z += 3;
}
break;
}
/* Step 5a */
if( z[0]=='e' ){
if( m_gt_1(z+1) ){
z++;
}else if( m_eq_1(z+1) && !star_oh(z+1) ){
z++;
}
}
/* Step 5b */
if( m_gt_1(z) && z[0]=='l' && z[1]=='l' ){
z++;
}
/* z[] is now the stemmed word in reverse order. Flip it back
** around into forward order and return.
*/
*pnOut = i = strlen(z);
zOut[i] = 0;
while( *z ){
zOut[--i] = *(z++);
}
}
/*
** Characters that can be part of a token. We assume any character
** whose value is greater than 0x80 (any UTF character) can be
** part of a token. In other words, delimiters all must have
** values of 0x7f or lower.
*/
static const char isIdChar[] = {
/* x0 x1 x2 x3 x4 x5 x6 x7 x8 x9 xA xB xC xD xE xF */
1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, /* 3x */
0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, /* 4x */
1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, /* 5x */
0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, /* 6x */
1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, /* 7x */
};
#define idChar(C) (((ch=C)&0x80)!=0 || (ch>0x2f && isIdChar[ch-0x30]))
#define isDelim(C) (((ch=C)&0x80)==0 && (ch<0x30 || !isIdChar[ch-0x30]))
/*
** Extract the next token from a tokenization cursor. The cursor must
** have been opened by a prior call to porterOpen().
*/
static int porterNext(
sqlite3_tokenizer_cursor *pCursor, /* Cursor returned by porterOpen */
const char **pzToken, /* OUT: *pzToken is the token text */
int *pnBytes, /* OUT: Number of bytes in token */
int *piStartOffset, /* OUT: Starting offset of token */
int *piEndOffset, /* OUT: Ending offset of token */
int *piPosition /* OUT: Position integer of token */
){
porter_tokenizer_cursor *c = (porter_tokenizer_cursor *) pCursor;
const char *z = c->zInput;
while( c->iOffset<c->nInput ){
int iStartOffset, ch;
/* Scan past delimiter characters */
while( c->iOffset<c->nInput && isDelim(z[c->iOffset]) ){
c->iOffset++;
}
/* Count non-delimiter characters. */
iStartOffset = c->iOffset;
while( c->iOffset<c->nInput && !isDelim(z[c->iOffset]) ){
c->iOffset++;
}
if( c->iOffset>iStartOffset ){
int n = c->iOffset-iStartOffset;
if( n>c->nAllocated ){
c->nAllocated = n+20;
c->zToken = realloc(c->zToken, c->nAllocated);
if( c->zToken==NULL ) return SQLITE_NOMEM;
}
porter_stemmer(&z[iStartOffset], n, c->zToken, pnBytes);
*pzToken = c->zToken;
*piStartOffset = iStartOffset;
*piEndOffset = c->iOffset;
*piPosition = c->iToken++;
return SQLITE_OK;
}
}
return SQLITE_DONE;
}
/*
** The set of routines that implement the porter-stemmer tokenizer
*/
static const sqlite3_tokenizer_module porterTokenizerModule = {
0,
porterCreate,
porterDestroy,
porterOpen,
porterClose,
porterNext,
};
/*
** Allocate a new porter tokenizer. Return a pointer to the new
** tokenizer in *ppModule
*/
void sqlite3Fts1PorterTokenizerModule(
sqlite3_tokenizer_module const**ppModule
){
*ppModule = &porterTokenizerModule;
}
#endif /* !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS1) */
-90
View File
@@ -1,90 +0,0 @@
/*
** 2006 July 10
**
** The author disclaims copyright to this source code.
**
*************************************************************************
** Defines the interface to tokenizers used by fulltext-search. There
** are three basic components:
**
** sqlite3_tokenizer_module is a singleton defining the tokenizer
** interface functions. This is essentially the class structure for
** tokenizers.
**
** sqlite3_tokenizer is used to define a particular tokenizer, perhaps
** including customization information defined at creation time.
**
** sqlite3_tokenizer_cursor is generated by a tokenizer to generate
** tokens from a particular input.
*/
#ifndef _FTS1_TOKENIZER_H_
#define _FTS1_TOKENIZER_H_
/* TODO(shess) Only used for SQLITE_OK and SQLITE_DONE at this time.
** If tokenizers are to be allowed to call sqlite3_*() functions, then
** we will need a way to register the API consistently.
*/
#include "sqlite3.h"
/*
** Structures used by the tokenizer interface.
*/
typedef struct sqlite3_tokenizer sqlite3_tokenizer;
typedef struct sqlite3_tokenizer_cursor sqlite3_tokenizer_cursor;
typedef struct sqlite3_tokenizer_module sqlite3_tokenizer_module;
struct sqlite3_tokenizer_module {
int iVersion; /* currently 0 */
/*
** Create and destroy a tokenizer. argc/argv are passed down from
** the fulltext virtual table creation to allow customization.
*/
int (*xCreate)(int argc, const char *const*argv,
sqlite3_tokenizer **ppTokenizer);
int (*xDestroy)(sqlite3_tokenizer *pTokenizer);
/*
** Tokenize a particular input. Call xOpen() to prepare to
** tokenize, xNext() repeatedly until it returns SQLITE_DONE, then
** xClose() to free any internal state. The pInput passed to
** xOpen() must exist until the cursor is closed. The ppToken
** result from xNext() is only valid until the next call to xNext()
** or until xClose() is called.
*/
/* TODO(shess) current implementation requires pInput to be
** nul-terminated. This should either be fixed, or pInput/nBytes
** should be converted to zInput.
*/
int (*xOpen)(sqlite3_tokenizer *pTokenizer,
const char *pInput, int nBytes,
sqlite3_tokenizer_cursor **ppCursor);
int (*xClose)(sqlite3_tokenizer_cursor *pCursor);
int (*xNext)(sqlite3_tokenizer_cursor *pCursor,
const char **ppToken, int *pnBytes,
int *piStartOffset, int *piEndOffset, int *piPosition);
};
struct sqlite3_tokenizer {
const sqlite3_tokenizer_module *pModule; /* The module for this tokenizer */
/* Tokenizer implementations will typically add additional fields */
};
struct sqlite3_tokenizer_cursor {
sqlite3_tokenizer *pTokenizer; /* Tokenizer for this cursor. */
/* Tokenizer implementations will typically add additional fields */
};
/*
** Get the module for a tokenizer which generates tokens based on a
** set of non-token characters. The default is to break tokens at any
** non-alnum character, though the set of delimiters can also be
** specified by the first argv argument to xCreate().
*/
/* TODO(shess) This doesn't belong here. Need some sort of
** registration process.
*/
void sqlite3Fts1SimpleTokenizerModule(sqlite3_tokenizer_module const**ppModule);
void sqlite3Fts1PorterTokenizerModule(sqlite3_tokenizer_module const**ppModule);
#endif /* _FTS1_TOKENIZER_H_ */
-221
View File
@@ -1,221 +0,0 @@
/*
** The author disclaims copyright to this source code.
**
*************************************************************************
** Implementation of the "simple" full-text-search tokenizer.
*/
/*
** The code in this file is only compiled if:
**
** * The FTS1 module is being built as an extension
** (in which case SQLITE_CORE is not defined), or
**
** * The FTS1 module is being built into the core of
** SQLite (in which case SQLITE_ENABLE_FTS1 is defined).
*/
#if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS1)
#include <assert.h>
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <ctype.h>
#include "fts1_tokenizer.h"
typedef struct simple_tokenizer {
sqlite3_tokenizer base;
char delim[128]; /* flag ASCII delimiters */
} simple_tokenizer;
typedef struct simple_tokenizer_cursor {
sqlite3_tokenizer_cursor base;
const char *pInput; /* input we are tokenizing */
int nBytes; /* size of the input */
int iOffset; /* current position in pInput */
int iToken; /* index of next token to be returned */
char *pToken; /* storage for current token */
int nTokenAllocated; /* space allocated to zToken buffer */
} simple_tokenizer_cursor;
/* Forward declaration */
static const sqlite3_tokenizer_module simpleTokenizerModule;
static int isDelim(simple_tokenizer *t, unsigned char c){
return c<0x80 && t->delim[c];
}
/*
** Create a new tokenizer instance.
*/
static int simpleCreate(
int argc, const char * const *argv,
sqlite3_tokenizer **ppTokenizer
){
simple_tokenizer *t;
t = (simple_tokenizer *) calloc(sizeof(*t), 1);
if( t==NULL ) return SQLITE_NOMEM;
/* TODO(shess) Delimiters need to remain the same from run to run,
** else we need to reindex. One solution would be a meta-table to
** track such information in the database, then we'd only want this
** information on the initial create.
*/
if( argc>1 ){
int i, n = strlen(argv[1]);
for(i=0; i<n; i++){
unsigned char ch = argv[1][i];
/* We explicitly don't support UTF-8 delimiters for now. */
if( ch>=0x80 ){
free(t);
return SQLITE_ERROR;
}
t->delim[ch] = 1;
}
} else {
/* Mark non-alphanumeric ASCII characters as delimiters */
int i;
for(i=1; i<0x80; i++){
t->delim[i] = !isalnum(i);
}
}
*ppTokenizer = &t->base;
return SQLITE_OK;
}
/*
** Destroy a tokenizer
*/
static int simpleDestroy(sqlite3_tokenizer *pTokenizer){
free(pTokenizer);
return SQLITE_OK;
}
/*
** Prepare to begin tokenizing a particular string. The input
** string to be tokenized is pInput[0..nBytes-1]. A cursor
** used to incrementally tokenize this string is returned in
** *ppCursor.
*/
static int simpleOpen(
sqlite3_tokenizer *pTokenizer, /* The tokenizer */
const char *pInput, int nBytes, /* String to be tokenized */
sqlite3_tokenizer_cursor **ppCursor /* OUT: Tokenization cursor */
){
simple_tokenizer_cursor *c;
c = (simple_tokenizer_cursor *) malloc(sizeof(*c));
if( c==NULL ) return SQLITE_NOMEM;
c->pInput = pInput;
if( pInput==0 ){
c->nBytes = 0;
}else if( nBytes<0 ){
c->nBytes = (int)strlen(pInput);
}else{
c->nBytes = nBytes;
}
c->iOffset = 0; /* start tokenizing at the beginning */
c->iToken = 0;
c->pToken = NULL; /* no space allocated, yet. */
c->nTokenAllocated = 0;
*ppCursor = &c->base;
return SQLITE_OK;
}
/*
** Close a tokenization cursor previously opened by a call to
** simpleOpen() above.
*/
static int simpleClose(sqlite3_tokenizer_cursor *pCursor){
simple_tokenizer_cursor *c = (simple_tokenizer_cursor *) pCursor;
free(c->pToken);
free(c);
return SQLITE_OK;
}
/*
** Extract the next token from a tokenization cursor. The cursor must
** have been opened by a prior call to simpleOpen().
*/
static int simpleNext(
sqlite3_tokenizer_cursor *pCursor, /* Cursor returned by simpleOpen */
const char **ppToken, /* OUT: *ppToken is the token text */
int *pnBytes, /* OUT: Number of bytes in token */
int *piStartOffset, /* OUT: Starting offset of token */
int *piEndOffset, /* OUT: Ending offset of token */
int *piPosition /* OUT: Position integer of token */
){
simple_tokenizer_cursor *c = (simple_tokenizer_cursor *) pCursor;
simple_tokenizer *t = (simple_tokenizer *) pCursor->pTokenizer;
unsigned char *p = (unsigned char *)c->pInput;
while( c->iOffset<c->nBytes ){
int iStartOffset;
/* Scan past delimiter characters */
while( c->iOffset<c->nBytes && isDelim(t, p[c->iOffset]) ){
c->iOffset++;
}
/* Count non-delimiter characters. */
iStartOffset = c->iOffset;
while( c->iOffset<c->nBytes && !isDelim(t, p[c->iOffset]) ){
c->iOffset++;
}
if( c->iOffset>iStartOffset ){
int i, n = c->iOffset-iStartOffset;
if( n>c->nTokenAllocated ){
c->nTokenAllocated = n+20;
c->pToken = realloc(c->pToken, c->nTokenAllocated);
if( c->pToken==NULL ) return SQLITE_NOMEM;
}
for(i=0; i<n; i++){
/* TODO(shess) This needs expansion to handle UTF-8
** case-insensitivity.
*/
unsigned char ch = p[iStartOffset+i];
c->pToken[i] = ch<0x80 ? tolower(ch) : ch;
}
*ppToken = c->pToken;
*pnBytes = n;
*piStartOffset = iStartOffset;
*piEndOffset = c->iOffset;
*piPosition = c->iToken++;
return SQLITE_OK;
}
}
return SQLITE_DONE;
}
/*
** The set of routines that implement the simple tokenizer
*/
static const sqlite3_tokenizer_module simpleTokenizerModule = {
0,
simpleCreate,
simpleDestroy,
simpleOpen,
simpleClose,
simpleNext,
};
/*
** Allocate a new simple tokenizer. Return a pointer to the new
** tokenizer in *ppModule
*/
void sqlite3Fts1SimpleTokenizerModule(
sqlite3_tokenizer_module const**ppModule
){
*ppModule = &simpleTokenizerModule;
}
#endif /* !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS1) */
-1511
View File
File diff suppressed because it is too large Load Diff
-11
View File
@@ -1,11 +0,0 @@
#include "sqlite3.h"
#ifdef __cplusplus
extern "C" {
#endif /* __cplusplus */
int fulltext_init(sqlite3 *db);
#ifdef __cplusplus
} /* extern "C" */
#endif /* __cplusplus */
-174
View File
@@ -1,174 +0,0 @@
/*
** The author disclaims copyright to this source code.
**
*************************************************************************
** Implementation of the "simple" full-text-search tokenizer.
*/
#include <assert.h>
#if !defined(__APPLE__)
#include <malloc.h>
#else
#include <stdlib.h>
#endif
#include <stdio.h>
#include <string.h>
#include <ctype.h>
#include "tokenizer.h"
/* Duplicate a string; the caller must free() the returned string.
* (We don't use strdup() since it's not part of the standard C library and
* may not be available everywhere.) */
/* TODO(shess) Copied from fulltext.c, consider util.c for such
** things. */
static char *string_dup(const char *s){
char *str = malloc(strlen(s) + 1);
strcpy(str, s);
return str;
}
typedef struct simple_tokenizer {
sqlite3_tokenizer base;
const char *zDelim; /* token delimiters */
} simple_tokenizer;
typedef struct simple_tokenizer_cursor {
sqlite3_tokenizer_cursor base;
const char *pInput; /* input we are tokenizing */
int nBytes; /* size of the input */
const char *pCurrent; /* current position in pInput */
int iToken; /* index of next token to be returned */
char *zToken; /* storage for current token */
int nTokenBytes; /* actual size of current token */
int nTokenAllocated; /* space allocated to zToken buffer */
} simple_tokenizer_cursor;
static sqlite3_tokenizer_module simpleTokenizerModule;/* forward declaration */
static int simpleCreate(
int argc, const char **argv,
sqlite3_tokenizer **ppTokenizer
){
simple_tokenizer *t;
t = (simple_tokenizer *) malloc(sizeof(simple_tokenizer));
/* TODO(shess) Delimiters need to remain the same from run to run,
** else we need to reindex. One solution would be a meta-table to
** track such information in the database, then we'd only want this
** information on the initial create.
*/
if( argc>1 ){
t->zDelim = string_dup(argv[1]);
} else {
/* Build a string excluding alphanumeric ASCII characters */
char zDelim[0x80]; /* nul-terminated, so nul not a member */
int i, j;
for(i=1, j=0; i<0x80; i++){
if( !isalnum(i) ){
zDelim[j++] = i;
}
}
zDelim[j++] = '\0';
assert( j<=sizeof(zDelim) );
t->zDelim = string_dup(zDelim);
}
*ppTokenizer = &t->base;
return SQLITE_OK;
}
static int simpleDestroy(sqlite3_tokenizer *pTokenizer){
simple_tokenizer *t = (simple_tokenizer *) pTokenizer;
free((void *) t->zDelim);
free(t);
return SQLITE_OK;
}
static int simpleOpen(
sqlite3_tokenizer *pTokenizer,
const char *pInput, int nBytes,
sqlite3_tokenizer_cursor **ppCursor
){
simple_tokenizer_cursor *c;
c = (simple_tokenizer_cursor *) malloc(sizeof(simple_tokenizer_cursor));
c->pInput = pInput;
c->nBytes = nBytes<0 ? (int) strlen(pInput) : nBytes;
c->pCurrent = c->pInput; /* start tokenizing at the beginning */
c->iToken = 0;
c->zToken = NULL; /* no space allocated, yet. */
c->nTokenBytes = 0;
c->nTokenAllocated = 0;
*ppCursor = &c->base;
return SQLITE_OK;
}
static int simpleClose(sqlite3_tokenizer_cursor *pCursor){
simple_tokenizer_cursor *c = (simple_tokenizer_cursor *) pCursor;
if( NULL!=c->zToken ){
free(c->zToken);
}
free(c);
return SQLITE_OK;
}
static int simpleNext(
sqlite3_tokenizer_cursor *pCursor,
const char **ppToken, int *pnBytes,
int *piStartOffset, int *piEndOffset, int *piPosition
){
simple_tokenizer_cursor *c = (simple_tokenizer_cursor *) pCursor;
simple_tokenizer *t = (simple_tokenizer *) pCursor->pTokenizer;
int ii;
while( c->pCurrent-c->pInput<c->nBytes ){
int n = (int) strcspn(c->pCurrent, t->zDelim);
if( n>0 ){
if( n+1>c->nTokenAllocated ){
c->zToken = realloc(c->zToken, n+1);
}
for(ii=0; ii<n; ii++){
/* TODO(shess) This needs expansion to handle UTF-8
** case-insensitivity.
*/
char ch = c->pCurrent[ii];
c->zToken[ii] = (unsigned char)ch<0x80 ? tolower((unsigned char)ch):ch;
}
c->zToken[n] = '\0';
*ppToken = c->zToken;
*pnBytes = n;
*piStartOffset = (int) (c->pCurrent-c->pInput);
*piEndOffset = *piStartOffset+n;
*piPosition = c->iToken++;
c->pCurrent += n + 1;
return SQLITE_OK;
}
c->pCurrent += n + 1;
/* TODO(shess) could strspn() to skip delimiters en masse. Needs
** to happen in two places, though, which is annoying.
*/
}
return SQLITE_DONE;
}
static sqlite3_tokenizer_module simpleTokenizerModule = {
0,
simpleCreate,
simpleDestroy,
simpleOpen,
simpleClose,
simpleNext,
};
void get_simple_tokenizer_module(
sqlite3_tokenizer_module **ppModule
){
*ppModule = &simpleTokenizerModule;
}
-89
View File
@@ -1,89 +0,0 @@
/*
** 2006 July 10
**
** The author disclaims copyright to this source code.
**
*************************************************************************
** Defines the interface to tokenizers used by fulltext-search. There
** are three basic components:
**
** sqlite3_tokenizer_module is a singleton defining the tokenizer
** interface functions. This is essentially the class structure for
** tokenizers.
**
** sqlite3_tokenizer is used to define a particular tokenizer, perhaps
** including customization information defined at creation time.
**
** sqlite3_tokenizer_cursor is generated by a tokenizer to generate
** tokens from a particular input.
*/
#ifndef _TOKENIZER_H_
#define _TOKENIZER_H_
/* TODO(shess) Only used for SQLITE_OK and SQLITE_DONE at this time.
** If tokenizers are to be allowed to call sqlite3_*() functions, then
** we will need a way to register the API consistently.
*/
#include "sqlite3.h"
/*
** Structures used by the tokenizer interface.
*/
typedef struct sqlite3_tokenizer sqlite3_tokenizer;
typedef struct sqlite3_tokenizer_cursor sqlite3_tokenizer_cursor;
typedef struct sqlite3_tokenizer_module sqlite3_tokenizer_module;
struct sqlite3_tokenizer_module {
int iVersion; /* currently 0 */
/*
** Create and destroy a tokenizer. argc/argv are passed down from
** the fulltext virtual table creation to allow customization.
*/
int (*xCreate)(int argc, const char **argv,
sqlite3_tokenizer **ppTokenizer);
int (*xDestroy)(sqlite3_tokenizer *pTokenizer);
/*
** Tokenize a particular input. Call xOpen() to prepare to
** tokenize, xNext() repeatedly until it returns SQLITE_DONE, then
** xClose() to free any internal state. The pInput passed to
** xOpen() must exist until the cursor is closed. The ppToken
** result from xNext() is only valid until the next call to xNext()
** or until xClose() is called.
*/
/* TODO(shess) current implementation requires pInput to be
** nul-terminated. This should either be fixed, or pInput/nBytes
** should be converted to zInput.
*/
int (*xOpen)(sqlite3_tokenizer *pTokenizer,
const char *pInput, int nBytes,
sqlite3_tokenizer_cursor **ppCursor);
int (*xClose)(sqlite3_tokenizer_cursor *pCursor);
int (*xNext)(sqlite3_tokenizer_cursor *pCursor,
const char **ppToken, int *pnBytes,
int *piStartOffset, int *piEndOffset, int *piPosition);
};
struct sqlite3_tokenizer {
sqlite3_tokenizer_module *pModule; /* The module for this tokenizer */
/* Tokenizer implementations will typically add additional fields */
};
struct sqlite3_tokenizer_cursor {
sqlite3_tokenizer *pTokenizer; /* Tokenizer for this cursor. */
/* Tokenizer implementations will typically add additional fields */
};
/*
** Get the module for a tokenizer which generates tokens based on a
** set of non-token characters. The default is to break tokens at any
** non-alnum character, though the set of delimiters can also be
** specified by the first argv argument to xCreate().
*/
/* TODO(shess) This doesn't belong here. Need some sort of
** registration process.
*/
void get_simple_tokenizer_module(sqlite3_tokenizer_module **ppModule);
#endif /* _TOKENIZER_H_ */
-133
View File
@@ -1,133 +0,0 @@
1. FTS2 Tokenizers
When creating a new full-text table, FTS2 allows the user to select
the text tokenizer implementation to be used when indexing text
by specifying a "tokenizer" clause as part of the CREATE VIRTUAL TABLE
statement:
CREATE VIRTUAL TABLE <table-name> USING fts2(
<columns ...> [, tokenizer <tokenizer-name> [<tokenizer-args>]]
);
The built-in tokenizers (valid values to pass as <tokenizer name>) are
"simple" and "porter".
<tokenizer-args> should consist of zero or more white-space separated
arguments to pass to the selected tokenizer implementation. The
interpretation of the arguments, if any, depends on the individual
tokenizer.
2. Custom Tokenizers
FTS2 allows users to provide custom tokenizer implementations. The
interface used to create a new tokenizer is defined and described in
the fts2_tokenizer.h source file.
Registering a new FTS2 tokenizer is similar to registering a new
virtual table module with SQLite. The user passes a pointer to a
structure containing pointers to various callback functions that
make up the implementation of the new tokenizer type. For tokenizers,
the structure (defined in fts2_tokenizer.h) is called
"sqlite3_tokenizer_module".
FTS2 does not expose a C-function that users call to register new
tokenizer types with a database handle. Instead, the pointer must
be encoded as an SQL blob value and passed to FTS2 through the SQL
engine by evaluating a special scalar function, "fts2_tokenizer()".
The fts2_tokenizer() function may be called with one or two arguments,
as follows:
SELECT fts2_tokenizer(<tokenizer-name>);
SELECT fts2_tokenizer(<tokenizer-name>, <sqlite3_tokenizer_module ptr>);
Where <tokenizer-name> is a string identifying the tokenizer and
<sqlite3_tokenizer_module ptr> is a pointer to an sqlite3_tokenizer_module
structure encoded as an SQL blob. If the second argument is present,
it is registered as tokenizer <tokenizer-name> and a copy of it
returned. If only one argument is passed, a pointer to the tokenizer
implementation currently registered as <tokenizer-name> is returned,
encoded as a blob. Or, if no such tokenizer exists, an SQL exception
(error) is raised.
SECURITY: If the fts2 extension is used in an environment where potentially
malicious users may execute arbitrary SQL (i.e. gears), they should be
prevented from invoking the fts2_tokenizer() function, possibly using the
authorisation callback.
See "Sample code" below for an example of calling the fts2_tokenizer()
function from C code.
3. ICU Library Tokenizers
If this extension is compiled with the SQLITE_ENABLE_ICU pre-processor
symbol defined, then there exists a built-in tokenizer named "icu"
implemented using the ICU library. The first argument passed to the
xCreate() method (see fts2_tokenizer.h) of this tokenizer may be
an ICU locale identifier. For example "tr_TR" for Turkish as used
in Turkey, or "en_AU" for English as used in Australia. For example:
"CREATE VIRTUAL TABLE thai_text USING fts2(text, tokenizer icu th_TH)"
The ICU tokenizer implementation is very simple. It splits the input
text according to the ICU rules for finding word boundaries and discards
any tokens that consist entirely of white-space. This may be suitable
for some applications in some locales, but not all. If more complex
processing is required, for example to implement stemming or
discard punctuation, this can be done by creating a tokenizer
implementation that uses the ICU tokenizer as part of its implementation.
When using the ICU tokenizer this way, it is safe to overwrite the
contents of the strings returned by the xNext() method (see
fts2_tokenizer.h).
4. Sample code.
The following two code samples illustrate the way C code should invoke
the fts2_tokenizer() scalar function:
int registerTokenizer(
sqlite3 *db,
char *zName,
const sqlite3_tokenizer_module *p
){
int rc;
sqlite3_stmt *pStmt;
const char zSql[] = "SELECT fts2_tokenizer(?, ?)";
rc = sqlite3_prepare_v2(db, zSql, -1, &pStmt, 0);
if( rc!=SQLITE_OK ){
return rc;
}
sqlite3_bind_text(pStmt, 1, zName, -1, SQLITE_STATIC);
sqlite3_bind_blob(pStmt, 2, &p, sizeof(p), SQLITE_STATIC);
sqlite3_step(pStmt);
return sqlite3_finalize(pStmt);
}
int queryTokenizer(
sqlite3 *db,
char *zName,
const sqlite3_tokenizer_module **pp
){
int rc;
sqlite3_stmt *pStmt;
const char zSql[] = "SELECT fts2_tokenizer(?)";
*pp = 0;
rc = sqlite3_prepare_v2(db, zSql, -1, &pStmt, 0);
if( rc!=SQLITE_OK ){
return rc;
}
sqlite3_bind_text(pStmt, 1, zName, -1, SQLITE_STATIC);
if( SQLITE_ROW==sqlite3_step(pStmt) ){
if( sqlite3_column_type(pStmt, 0)==SQLITE_BLOB ){
memcpy(pp, sqlite3_column_blob(pStmt, 0), sizeof(*pp));
}
}
return sqlite3_finalize(pStmt);
}
-4
View File
@@ -1,4 +0,0 @@
This folder contains source code to the second full-text search
extension for SQLite. While the API is the same, this version uses a
substantially different storage schema from fts1, so tables will need
to be rebuilt.
-6860
View File
File diff suppressed because it is too large Load Diff
-26
View File
@@ -1,26 +0,0 @@
/*
** 2006 Oct 10
**
** The author disclaims copyright to this source code. In place of
** a legal notice, here is a blessing:
**
** May you do good and not evil.
** May you find forgiveness for yourself and forgive others.
** May you share freely, never taking more than you give.
**
******************************************************************************
**
** This header file is used by programs that want to link against the
** FTS2 library. All it does is declare the sqlite3Fts2Init() interface.
*/
#include "sqlite3.h"
#ifdef __cplusplus
extern "C" {
#endif /* __cplusplus */
int sqlite3Fts2Init(sqlite3 *db);
#ifdef __cplusplus
} /* extern "C" */
#endif /* __cplusplus */
-376
View File
@@ -1,376 +0,0 @@
/*
** 2001 September 22
**
** The author disclaims copyright to this source code. In place of
** a legal notice, here is a blessing:
**
** May you do good and not evil.
** May you find forgiveness for yourself and forgive others.
** May you share freely, never taking more than you give.
**
*************************************************************************
** This is the implementation of generic hash-tables used in SQLite.
** We've modified it slightly to serve as a standalone hash table
** implementation for the full-text indexing module.
*/
/*
** The code in this file is only compiled if:
**
** * The FTS2 module is being built as an extension
** (in which case SQLITE_CORE is not defined), or
**
** * The FTS2 module is being built into the core of
** SQLite (in which case SQLITE_ENABLE_FTS2 is defined).
*/
#if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS2)
#include <assert.h>
#include <stdlib.h>
#include <string.h>
#include "sqlite3.h"
#include "sqlite3ext.h"
SQLITE_EXTENSION_INIT3
#include "fts2_hash.h"
/*
** Malloc and Free functions
*/
static void *fts2HashMalloc(int n){
void *p = sqlite3_malloc(n);
if( p ){
memset(p, 0, n);
}
return p;
}
static void fts2HashFree(void *p){
sqlite3_free(p);
}
/* Turn bulk memory into a hash table object by initializing the
** fields of the Hash structure.
**
** "pNew" is a pointer to the hash table that is to be initialized.
** keyClass is one of the constants
** FTS2_HASH_BINARY or FTS2_HASH_STRING. The value of keyClass
** determines what kind of key the hash table will use. "copyKey" is
** true if the hash table should make its own private copy of keys and
** false if it should just use the supplied pointer.
*/
void sqlite3Fts2HashInit(fts2Hash *pNew, int keyClass, int copyKey){
assert( pNew!=0 );
assert( keyClass>=FTS2_HASH_STRING && keyClass<=FTS2_HASH_BINARY );
pNew->keyClass = keyClass;
pNew->copyKey = copyKey;
pNew->first = 0;
pNew->count = 0;
pNew->htsize = 0;
pNew->ht = 0;
}
/* Remove all entries from a hash table. Reclaim all memory.
** Call this routine to delete a hash table or to reset a hash table
** to the empty state.
*/
void sqlite3Fts2HashClear(fts2Hash *pH){
fts2HashElem *elem; /* For looping over all elements of the table */
assert( pH!=0 );
elem = pH->first;
pH->first = 0;
fts2HashFree(pH->ht);
pH->ht = 0;
pH->htsize = 0;
while( elem ){
fts2HashElem *next_elem = elem->next;
if( pH->copyKey && elem->pKey ){
fts2HashFree(elem->pKey);
}
fts2HashFree(elem);
elem = next_elem;
}
pH->count = 0;
}
/*
** Hash and comparison functions when the mode is FTS2_HASH_STRING
*/
static int strHash(const void *pKey, int nKey){
const char *z = (const char *)pKey;
int h = 0;
if( nKey<=0 ) nKey = (int) strlen(z);
while( nKey > 0 ){
h = (h<<3) ^ h ^ *z++;
nKey--;
}
return h & 0x7fffffff;
}
static int strCompare(const void *pKey1, int n1, const void *pKey2, int n2){
if( n1!=n2 ) return 1;
return strncmp((const char*)pKey1,(const char*)pKey2,n1);
}
/*
** Hash and comparison functions when the mode is FTS2_HASH_BINARY
*/
static int binHash(const void *pKey, int nKey){
int h = 0;
const char *z = (const char *)pKey;
while( nKey-- > 0 ){
h = (h<<3) ^ h ^ *(z++);
}
return h & 0x7fffffff;
}
static int binCompare(const void *pKey1, int n1, const void *pKey2, int n2){
if( n1!=n2 ) return 1;
return memcmp(pKey1,pKey2,n1);
}
/*
** Return a pointer to the appropriate hash function given the key class.
**
** The C syntax in this function definition may be unfamilar to some
** programmers, so we provide the following additional explanation:
**
** The name of the function is "hashFunction". The function takes a
** single parameter "keyClass". The return value of hashFunction()
** is a pointer to another function. Specifically, the return value
** of hashFunction() is a pointer to a function that takes two parameters
** with types "const void*" and "int" and returns an "int".
*/
static int (*hashFunction(int keyClass))(const void*,int){
if( keyClass==FTS2_HASH_STRING ){
return &strHash;
}else{
assert( keyClass==FTS2_HASH_BINARY );
return &binHash;
}
}
/*
** Return a pointer to the appropriate hash function given the key class.
**
** For help in interpreted the obscure C code in the function definition,
** see the header comment on the previous function.
*/
static int (*compareFunction(int keyClass))(const void*,int,const void*,int){
if( keyClass==FTS2_HASH_STRING ){
return &strCompare;
}else{
assert( keyClass==FTS2_HASH_BINARY );
return &binCompare;
}
}
/* Link an element into the hash table
*/
static void insertElement(
fts2Hash *pH, /* The complete hash table */
struct _fts2ht *pEntry, /* The entry into which pNew is inserted */
fts2HashElem *pNew /* The element to be inserted */
){
fts2HashElem *pHead; /* First element already in pEntry */
pHead = pEntry->chain;
if( pHead ){
pNew->next = pHead;
pNew->prev = pHead->prev;
if( pHead->prev ){ pHead->prev->next = pNew; }
else { pH->first = pNew; }
pHead->prev = pNew;
}else{
pNew->next = pH->first;
if( pH->first ){ pH->first->prev = pNew; }
pNew->prev = 0;
pH->first = pNew;
}
pEntry->count++;
pEntry->chain = pNew;
}
/* Resize the hash table so that it cantains "new_size" buckets.
** "new_size" must be a power of 2. The hash table might fail
** to resize if sqliteMalloc() fails.
*/
static void rehash(fts2Hash *pH, int new_size){
struct _fts2ht *new_ht; /* The new hash table */
fts2HashElem *elem, *next_elem; /* For looping over existing elements */
int (*xHash)(const void*,int); /* The hash function */
assert( (new_size & (new_size-1))==0 );
new_ht = (struct _fts2ht *)fts2HashMalloc( new_size*sizeof(struct _fts2ht) );
if( new_ht==0 ) return;
fts2HashFree(pH->ht);
pH->ht = new_ht;
pH->htsize = new_size;
xHash = hashFunction(pH->keyClass);
for(elem=pH->first, pH->first=0; elem; elem = next_elem){
int h = (*xHash)(elem->pKey, elem->nKey) & (new_size-1);
next_elem = elem->next;
insertElement(pH, &new_ht[h], elem);
}
}
/* This function (for internal use only) locates an element in an
** hash table that matches the given key. The hash for this key has
** already been computed and is passed as the 4th parameter.
*/
static fts2HashElem *findElementGivenHash(
const fts2Hash *pH, /* The pH to be searched */
const void *pKey, /* The key we are searching for */
int nKey,
int h /* The hash for this key. */
){
fts2HashElem *elem; /* Used to loop thru the element list */
int count; /* Number of elements left to test */
int (*xCompare)(const void*,int,const void*,int); /* comparison function */
if( pH->ht ){
struct _fts2ht *pEntry = &pH->ht[h];
elem = pEntry->chain;
count = pEntry->count;
xCompare = compareFunction(pH->keyClass);
while( count-- && elem ){
if( (*xCompare)(elem->pKey,elem->nKey,pKey,nKey)==0 ){
return elem;
}
elem = elem->next;
}
}
return 0;
}
/* Remove a single entry from the hash table given a pointer to that
** element and a hash on the element's key.
*/
static void removeElementGivenHash(
fts2Hash *pH, /* The pH containing "elem" */
fts2HashElem* elem, /* The element to be removed from the pH */
int h /* Hash value for the element */
){
struct _fts2ht *pEntry;
if( elem->prev ){
elem->prev->next = elem->next;
}else{
pH->first = elem->next;
}
if( elem->next ){
elem->next->prev = elem->prev;
}
pEntry = &pH->ht[h];
if( pEntry->chain==elem ){
pEntry->chain = elem->next;
}
pEntry->count--;
if( pEntry->count<=0 ){
pEntry->chain = 0;
}
if( pH->copyKey && elem->pKey ){
fts2HashFree(elem->pKey);
}
fts2HashFree( elem );
pH->count--;
if( pH->count<=0 ){
assert( pH->first==0 );
assert( pH->count==0 );
fts2HashClear(pH);
}
}
/* Attempt to locate an element of the hash table pH with a key
** that matches pKey,nKey. Return the data for this element if it is
** found, or NULL if there is no match.
*/
void *sqlite3Fts2HashFind(const fts2Hash *pH, const void *pKey, int nKey){
int h; /* A hash on key */
fts2HashElem *elem; /* The element that matches key */
int (*xHash)(const void*,int); /* The hash function */
if( pH==0 || pH->ht==0 ) return 0;
xHash = hashFunction(pH->keyClass);
assert( xHash!=0 );
h = (*xHash)(pKey,nKey);
assert( (pH->htsize & (pH->htsize-1))==0 );
elem = findElementGivenHash(pH,pKey,nKey, h & (pH->htsize-1));
return elem ? elem->data : 0;
}
/* Insert an element into the hash table pH. The key is pKey,nKey
** and the data is "data".
**
** If no element exists with a matching key, then a new
** element is created. A copy of the key is made if the copyKey
** flag is set. NULL is returned.
**
** If another element already exists with the same key, then the
** new data replaces the old data and the old data is returned.
** The key is not copied in this instance. If a malloc fails, then
** the new data is returned and the hash table is unchanged.
**
** If the "data" parameter to this function is NULL, then the
** element corresponding to "key" is removed from the hash table.
*/
void *sqlite3Fts2HashInsert(
fts2Hash *pH, /* The hash table to insert into */
const void *pKey, /* The key */
int nKey, /* Number of bytes in the key */
void *data /* The data */
){
int hraw; /* Raw hash value of the key */
int h; /* the hash of the key modulo hash table size */
fts2HashElem *elem; /* Used to loop thru the element list */
fts2HashElem *new_elem; /* New element added to the pH */
int (*xHash)(const void*,int); /* The hash function */
assert( pH!=0 );
xHash = hashFunction(pH->keyClass);
assert( xHash!=0 );
hraw = (*xHash)(pKey, nKey);
assert( (pH->htsize & (pH->htsize-1))==0 );
h = hraw & (pH->htsize-1);
elem = findElementGivenHash(pH,pKey,nKey,h);
if( elem ){
void *old_data = elem->data;
if( data==0 ){
removeElementGivenHash(pH,elem,h);
}else{
elem->data = data;
}
return old_data;
}
if( data==0 ) return 0;
new_elem = (fts2HashElem*)fts2HashMalloc( sizeof(fts2HashElem) );
if( new_elem==0 ) return data;
if( pH->copyKey && pKey!=0 ){
new_elem->pKey = fts2HashMalloc( nKey );
if( new_elem->pKey==0 ){
fts2HashFree(new_elem);
return data;
}
memcpy((void*)new_elem->pKey, pKey, nKey);
}else{
new_elem->pKey = (void*)pKey;
}
new_elem->nKey = nKey;
pH->count++;
if( pH->htsize==0 ){
rehash(pH,8);
if( pH->htsize==0 ){
pH->count = 0;
fts2HashFree(new_elem);
return data;
}
}
if( pH->count > pH->htsize ){
rehash(pH,pH->htsize*2);
}
assert( pH->htsize>0 );
assert( (pH->htsize & (pH->htsize-1))==0 );
h = hraw & (pH->htsize-1);
insertElement(pH, &pH->ht[h], new_elem);
new_elem->data = data;
return 0;
}
#endif /* !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS2) */
-110
View File
@@ -1,110 +0,0 @@
/*
** 2001 September 22
**
** The author disclaims copyright to this source code. In place of
** a legal notice, here is a blessing:
**
** May you do good and not evil.
** May you find forgiveness for yourself and forgive others.
** May you share freely, never taking more than you give.
**
*************************************************************************
** This is the header file for the generic hash-table implementation
** used in SQLite. We've modified it slightly to serve as a standalone
** hash table implementation for the full-text indexing module.
**
*/
#ifndef _FTS2_HASH_H_
#define _FTS2_HASH_H_
/* Forward declarations of structures. */
typedef struct fts2Hash fts2Hash;
typedef struct fts2HashElem fts2HashElem;
/* A complete hash table is an instance of the following structure.
** The internals of this structure are intended to be opaque -- client
** code should not attempt to access or modify the fields of this structure
** directly. Change this structure only by using the routines below.
** However, many of the "procedures" and "functions" for modifying and
** accessing this structure are really macros, so we can't really make
** this structure opaque.
*/
struct fts2Hash {
char keyClass; /* HASH_INT, _POINTER, _STRING, _BINARY */
char copyKey; /* True if copy of key made on insert */
int count; /* Number of entries in this table */
fts2HashElem *first; /* The first element of the array */
int htsize; /* Number of buckets in the hash table */
struct _fts2ht { /* the hash table */
int count; /* Number of entries with this hash */
fts2HashElem *chain; /* Pointer to first entry with this hash */
} *ht;
};
/* Each element in the hash table is an instance of the following
** structure. All elements are stored on a single doubly-linked list.
**
** Again, this structure is intended to be opaque, but it can't really
** be opaque because it is used by macros.
*/
struct fts2HashElem {
fts2HashElem *next, *prev; /* Next and previous elements in the table */
void *data; /* Data associated with this element */
void *pKey; int nKey; /* Key associated with this element */
};
/*
** There are 2 different modes of operation for a hash table:
**
** FTS2_HASH_STRING pKey points to a string that is nKey bytes long
** (including the null-terminator, if any). Case
** is respected in comparisons.
**
** FTS2_HASH_BINARY pKey points to binary data nKey bytes long.
** memcmp() is used to compare keys.
**
** A copy of the key is made if the copyKey parameter to fts2HashInit is 1.
*/
#define FTS2_HASH_STRING 1
#define FTS2_HASH_BINARY 2
/*
** Access routines. To delete, insert a NULL pointer.
*/
void sqlite3Fts2HashInit(fts2Hash*, int keytype, int copyKey);
void *sqlite3Fts2HashInsert(fts2Hash*, const void *pKey, int nKey, void *pData);
void *sqlite3Fts2HashFind(const fts2Hash*, const void *pKey, int nKey);
void sqlite3Fts2HashClear(fts2Hash*);
/*
** Shorthand for the functions above
*/
#define fts2HashInit sqlite3Fts2HashInit
#define fts2HashInsert sqlite3Fts2HashInsert
#define fts2HashFind sqlite3Fts2HashFind
#define fts2HashClear sqlite3Fts2HashClear
/*
** Macros for looping over all elements of a hash table. The idiom is
** like this:
**
** fts2Hash h;
** fts2HashElem *p;
** ...
** for(p=fts2HashFirst(&h); p; p=fts2HashNext(p)){
** SomeStructure *pData = fts2HashData(p);
** // do something with pData
** }
*/
#define fts2HashFirst(H) ((H)->first)
#define fts2HashNext(E) ((E)->next)
#define fts2HashData(E) ((E)->data)
#define fts2HashKey(E) ((E)->pKey)
#define fts2HashKeysize(E) ((E)->nKey)
/*
** Number of entries in a hash table
*/
#define fts2HashCount(H) ((H)->count)
#endif /* _FTS2_HASH_H_ */
-260
View File
@@ -1,260 +0,0 @@
/*
** 2007 June 22
**
** The author disclaims copyright to this source code. In place of
** a legal notice, here is a blessing:
**
** May you do good and not evil.
** May you find forgiveness for yourself and forgive others.
** May you share freely, never taking more than you give.
**
*************************************************************************
** This file implements a tokenizer for fts2 based on the ICU library.
**
** $Id: fts2_icu.c,v 1.3 2008/12/18 05:30:26 danielk1977 Exp $
*/
#if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS2)
#ifdef SQLITE_ENABLE_ICU
#include <assert.h>
#include <string.h>
#include "fts2_tokenizer.h"
#include <unicode/ubrk.h>
#include <unicode/ucol.h>
#include <unicode/ustring.h>
#include <unicode/utf16.h>
typedef struct IcuTokenizer IcuTokenizer;
typedef struct IcuCursor IcuCursor;
struct IcuTokenizer {
sqlite3_tokenizer base;
char *zLocale;
};
struct IcuCursor {
sqlite3_tokenizer_cursor base;
UBreakIterator *pIter; /* ICU break-iterator object */
int nChar; /* Number of UChar elements in pInput */
UChar *aChar; /* Copy of input using utf-16 encoding */
int *aOffset; /* Offsets of each character in utf-8 input */
int nBuffer;
char *zBuffer;
int iToken;
};
/*
** Create a new tokenizer instance.
*/
static int icuCreate(
int argc, /* Number of entries in argv[] */
const char * const *argv, /* Tokenizer creation arguments */
sqlite3_tokenizer **ppTokenizer /* OUT: Created tokenizer */
){
IcuTokenizer *p;
int n = 0;
if( argc>0 ){
n = strlen(argv[0])+1;
}
p = (IcuTokenizer *)sqlite3_malloc(sizeof(IcuTokenizer)+n);
if( !p ){
return SQLITE_NOMEM;
}
memset(p, 0, sizeof(IcuTokenizer));
if( n ){
p->zLocale = (char *)&p[1];
memcpy(p->zLocale, argv[0], n);
}
*ppTokenizer = (sqlite3_tokenizer *)p;
return SQLITE_OK;
}
/*
** Destroy a tokenizer
*/
static int icuDestroy(sqlite3_tokenizer *pTokenizer){
IcuTokenizer *p = (IcuTokenizer *)pTokenizer;
sqlite3_free(p);
return SQLITE_OK;
}
/*
** Prepare to begin tokenizing a particular string. The input
** string to be tokenized is pInput[0..nBytes-1]. A cursor
** used to incrementally tokenize this string is returned in
** *ppCursor.
*/
static int icuOpen(
sqlite3_tokenizer *pTokenizer, /* The tokenizer */
const char *zInput, /* Input string */
int nInput, /* Length of zInput in bytes */
sqlite3_tokenizer_cursor **ppCursor /* OUT: Tokenization cursor */
){
IcuTokenizer *p = (IcuTokenizer *)pTokenizer;
IcuCursor *pCsr;
const int32_t opt = U_FOLD_CASE_DEFAULT;
UErrorCode status = U_ZERO_ERROR;
int nChar;
UChar32 c;
int iInput = 0;
int iOut = 0;
*ppCursor = 0;
if( nInput<0 ){
nInput = strlen(zInput);
}
nChar = nInput+1;
pCsr = (IcuCursor *)sqlite3_malloc(
sizeof(IcuCursor) + /* IcuCursor */
((nChar+3)&~3) * sizeof(UChar) + /* IcuCursor.aChar[] */
(nChar+1) * sizeof(int) /* IcuCursor.aOffset[] */
);
if( !pCsr ){
return SQLITE_NOMEM;
}
memset(pCsr, 0, sizeof(IcuCursor));
pCsr->aChar = (UChar *)&pCsr[1];
pCsr->aOffset = (int *)&pCsr->aChar[(nChar+3)&~3];
pCsr->aOffset[iOut] = iInput;
U8_NEXT(zInput, iInput, nInput, c);
while( c>0 ){
int isError = 0;
c = u_foldCase(c, opt);
U16_APPEND(pCsr->aChar, iOut, nChar, c, isError);
if( isError ){
sqlite3_free(pCsr);
return SQLITE_ERROR;
}
pCsr->aOffset[iOut] = iInput;
if( iInput<nInput ){
U8_NEXT(zInput, iInput, nInput, c);
}else{
c = 0;
}
}
pCsr->pIter = ubrk_open(UBRK_WORD, p->zLocale, pCsr->aChar, iOut, &status);
if( !U_SUCCESS(status) ){
sqlite3_free(pCsr);
return SQLITE_ERROR;
}
pCsr->nChar = iOut;
ubrk_first(pCsr->pIter);
*ppCursor = (sqlite3_tokenizer_cursor *)pCsr;
return SQLITE_OK;
}
/*
** Close a tokenization cursor previously opened by a call to icuOpen().
*/
static int icuClose(sqlite3_tokenizer_cursor *pCursor){
IcuCursor *pCsr = (IcuCursor *)pCursor;
ubrk_close(pCsr->pIter);
sqlite3_free(pCsr->zBuffer);
sqlite3_free(pCsr);
return SQLITE_OK;
}
/*
** Extract the next token from a tokenization cursor.
*/
static int icuNext(
sqlite3_tokenizer_cursor *pCursor, /* Cursor returned by simpleOpen */
const char **ppToken, /* OUT: *ppToken is the token text */
int *pnBytes, /* OUT: Number of bytes in token */
int *piStartOffset, /* OUT: Starting offset of token */
int *piEndOffset, /* OUT: Ending offset of token */
int *piPosition /* OUT: Position integer of token */
){
IcuCursor *pCsr = (IcuCursor *)pCursor;
int iStart = 0;
int iEnd = 0;
int nByte = 0;
while( iStart==iEnd ){
UChar32 c;
iStart = ubrk_current(pCsr->pIter);
iEnd = ubrk_next(pCsr->pIter);
if( iEnd==UBRK_DONE ){
return SQLITE_DONE;
}
while( iStart<iEnd ){
int iWhite = iStart;
U8_NEXT(pCsr->aChar, iWhite, pCsr->nChar, c);
if( u_isspace(c) ){
iStart = iWhite;
}else{
break;
}
}
assert(iStart<=iEnd);
}
do {
UErrorCode status = U_ZERO_ERROR;
if( nByte ){
char *zNew = sqlite3_realloc(pCsr->zBuffer, nByte);
if( !zNew ){
return SQLITE_NOMEM;
}
pCsr->zBuffer = zNew;
pCsr->nBuffer = nByte;
}
u_strToUTF8(
pCsr->zBuffer, pCsr->nBuffer, &nByte, /* Output vars */
&pCsr->aChar[iStart], iEnd-iStart, /* Input vars */
&status /* Output success/failure */
);
} while( nByte>pCsr->nBuffer );
*ppToken = pCsr->zBuffer;
*pnBytes = nByte;
*piStartOffset = pCsr->aOffset[iStart];
*piEndOffset = pCsr->aOffset[iEnd];
*piPosition = pCsr->iToken++;
return SQLITE_OK;
}
/*
** The set of routines that implement the simple tokenizer
*/
static const sqlite3_tokenizer_module icuTokenizerModule = {
0, /* iVersion */
icuCreate, /* xCreate */
icuDestroy, /* xCreate */
icuOpen, /* xOpen */
icuClose, /* xClose */
icuNext, /* xNext */
};
/*
** Set *ppModule to point at the implementation of the ICU tokenizer.
*/
void sqlite3Fts2IcuTokenizerModule(
sqlite3_tokenizer_module const**ppModule
){
*ppModule = &icuTokenizerModule;
}
#endif /* defined(SQLITE_ENABLE_ICU) */
#endif /* !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS2) */
-644
View File
@@ -1,644 +0,0 @@
/*
** 2006 September 30
**
** The author disclaims copyright to this source code. In place of
** a legal notice, here is a blessing:
**
** May you do good and not evil.
** May you find forgiveness for yourself and forgive others.
** May you share freely, never taking more than you give.
**
*************************************************************************
** Implementation of the full-text-search tokenizer that implements
** a Porter stemmer.
*/
/*
** The code in this file is only compiled if:
**
** * The FTS2 module is being built as an extension
** (in which case SQLITE_CORE is not defined), or
**
** * The FTS2 module is being built into the core of
** SQLite (in which case SQLITE_ENABLE_FTS2 is defined).
*/
#if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS2)
#include <assert.h>
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include "sqlite3.h"
#include "sqlite3ext.h"
SQLITE_EXTENSION_INIT3
#include "fts2_tokenizer.h"
/*
** Class derived from sqlite3_tokenizer
*/
typedef struct porter_tokenizer {
sqlite3_tokenizer base; /* Base class */
} porter_tokenizer;
/*
** Class derived from sqlit3_tokenizer_cursor
*/
typedef struct porter_tokenizer_cursor {
sqlite3_tokenizer_cursor base;
const char *zInput; /* input we are tokenizing */
int nInput; /* size of the input */
int iOffset; /* current position in zInput */
int iToken; /* index of next token to be returned */
char *zToken; /* storage for current token */
int nAllocated; /* space allocated to zToken buffer */
} porter_tokenizer_cursor;
/* Forward declaration */
static const sqlite3_tokenizer_module porterTokenizerModule;
/*
** Create a new tokenizer instance.
*/
static int porterCreate(
int argc, const char * const *argv,
sqlite3_tokenizer **ppTokenizer
){
porter_tokenizer *t;
t = (porter_tokenizer *) sqlite3_malloc(sizeof(*t));
if( t==NULL ) return SQLITE_NOMEM;
memset(t, 0, sizeof(*t));
*ppTokenizer = &t->base;
return SQLITE_OK;
}
/*
** Destroy a tokenizer
*/
static int porterDestroy(sqlite3_tokenizer *pTokenizer){
sqlite3_free(pTokenizer);
return SQLITE_OK;
}
/*
** Prepare to begin tokenizing a particular string. The input
** string to be tokenized is zInput[0..nInput-1]. A cursor
** used to incrementally tokenize this string is returned in
** *ppCursor.
*/
static int porterOpen(
sqlite3_tokenizer *pTokenizer, /* The tokenizer */
const char *zInput, int nInput, /* String to be tokenized */
sqlite3_tokenizer_cursor **ppCursor /* OUT: Tokenization cursor */
){
porter_tokenizer_cursor *c;
c = (porter_tokenizer_cursor *) sqlite3_malloc(sizeof(*c));
if( c==NULL ) return SQLITE_NOMEM;
c->zInput = zInput;
if( zInput==0 ){
c->nInput = 0;
}else if( nInput<0 ){
c->nInput = (int)strlen(zInput);
}else{
c->nInput = nInput;
}
c->iOffset = 0; /* start tokenizing at the beginning */
c->iToken = 0;
c->zToken = NULL; /* no space allocated, yet. */
c->nAllocated = 0;
*ppCursor = &c->base;
return SQLITE_OK;
}
/*
** Close a tokenization cursor previously opened by a call to
** porterOpen() above.
*/
static int porterClose(sqlite3_tokenizer_cursor *pCursor){
porter_tokenizer_cursor *c = (porter_tokenizer_cursor *) pCursor;
sqlite3_free(c->zToken);
sqlite3_free(c);
return SQLITE_OK;
}
/*
** Vowel or consonant
*/
static const char cType[] = {
0, 1, 1, 1, 0, 1, 1, 1, 0, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 0,
1, 1, 1, 2, 1
};
/*
** isConsonant() and isVowel() determine if their first character in
** the string they point to is a consonant or a vowel, according
** to Porter ruls.
**
** A consonate is any letter other than 'a', 'e', 'i', 'o', or 'u'.
** 'Y' is a consonant unless it follows another consonant,
** in which case it is a vowel.
**
** In these routine, the letters are in reverse order. So the 'y' rule
** is that 'y' is a consonant unless it is followed by another
** consonent.
*/
static int isVowel(const char*);
static int isConsonant(const char *z){
int j;
char x = *z;
if( x==0 ) return 0;
assert( x>='a' && x<='z' );
j = cType[x-'a'];
if( j<2 ) return j;
return z[1]==0 || isVowel(z + 1);
}
static int isVowel(const char *z){
int j;
char x = *z;
if( x==0 ) return 0;
assert( x>='a' && x<='z' );
j = cType[x-'a'];
if( j<2 ) return 1-j;
return isConsonant(z + 1);
}
/*
** Let any sequence of one or more vowels be represented by V and let
** C be sequence of one or more consonants. Then every word can be
** represented as:
**
** [C] (VC){m} [V]
**
** In prose: A word is an optional consonant followed by zero or
** vowel-consonant pairs followed by an optional vowel. "m" is the
** number of vowel consonant pairs. This routine computes the value
** of m for the first i bytes of a word.
**
** Return true if the m-value for z is 1 or more. In other words,
** return true if z contains at least one vowel that is followed
** by a consonant.
**
** In this routine z[] is in reverse order. So we are really looking
** for an instance of of a consonant followed by a vowel.
*/
static int m_gt_0(const char *z){
while( isVowel(z) ){ z++; }
if( *z==0 ) return 0;
while( isConsonant(z) ){ z++; }
return *z!=0;
}
/* Like mgt0 above except we are looking for a value of m which is
** exactly 1
*/
static int m_eq_1(const char *z){
while( isVowel(z) ){ z++; }
if( *z==0 ) return 0;
while( isConsonant(z) ){ z++; }
if( *z==0 ) return 0;
while( isVowel(z) ){ z++; }
if( *z==0 ) return 1;
while( isConsonant(z) ){ z++; }
return *z==0;
}
/* Like mgt0 above except we are looking for a value of m>1 instead
** or m>0
*/
static int m_gt_1(const char *z){
while( isVowel(z) ){ z++; }
if( *z==0 ) return 0;
while( isConsonant(z) ){ z++; }
if( *z==0 ) return 0;
while( isVowel(z) ){ z++; }
if( *z==0 ) return 0;
while( isConsonant(z) ){ z++; }
return *z!=0;
}
/*
** Return TRUE if there is a vowel anywhere within z[0..n-1]
*/
static int hasVowel(const char *z){
while( isConsonant(z) ){ z++; }
return *z!=0;
}
/*
** Return TRUE if the word ends in a double consonant.
**
** The text is reversed here. So we are really looking at
** the first two characters of z[].
*/
static int doubleConsonant(const char *z){
return isConsonant(z) && z[0]==z[1] && isConsonant(z+1);
}
/*
** Return TRUE if the word ends with three letters which
** are consonant-vowel-consonent and where the final consonant
** is not 'w', 'x', or 'y'.
**
** The word is reversed here. So we are really checking the
** first three letters and the first one cannot be in [wxy].
*/
static int star_oh(const char *z){
return
z[0]!=0 && isConsonant(z) &&
z[0]!='w' && z[0]!='x' && z[0]!='y' &&
z[1]!=0 && isVowel(z+1) &&
z[2]!=0 && isConsonant(z+2);
}
/*
** If the word ends with zFrom and xCond() is true for the stem
** of the word that preceeds the zFrom ending, then change the
** ending to zTo.
**
** The input word *pz and zFrom are both in reverse order. zTo
** is in normal order.
**
** Return TRUE if zFrom matches. Return FALSE if zFrom does not
** match. Not that TRUE is returned even if xCond() fails and
** no substitution occurs.
*/
static int stem(
char **pz, /* The word being stemmed (Reversed) */
const char *zFrom, /* If the ending matches this... (Reversed) */
const char *zTo, /* ... change the ending to this (not reversed) */
int (*xCond)(const char*) /* Condition that must be true */
){
char *z = *pz;
while( *zFrom && *zFrom==*z ){ z++; zFrom++; }
if( *zFrom!=0 ) return 0;
if( xCond && !xCond(z) ) return 1;
while( *zTo ){
*(--z) = *(zTo++);
}
*pz = z;
return 1;
}
/*
** This is the fallback stemmer used when the porter stemmer is
** inappropriate. The input word is copied into the output with
** US-ASCII case folding. If the input word is too long (more
** than 20 bytes if it contains no digits or more than 6 bytes if
** it contains digits) then word is truncated to 20 or 6 bytes
** by taking 10 or 3 bytes from the beginning and end.
*/
static void copy_stemmer(const char *zIn, int nIn, char *zOut, int *pnOut){
int i, mx, j;
int hasDigit = 0;
for(i=0; i<nIn; i++){
int c = zIn[i];
if( c>='A' && c<='Z' ){
zOut[i] = c - 'A' + 'a';
}else{
if( c>='0' && c<='9' ) hasDigit = 1;
zOut[i] = c;
}
}
mx = hasDigit ? 3 : 10;
if( nIn>mx*2 ){
for(j=mx, i=nIn-mx; i<nIn; i++, j++){
zOut[j] = zOut[i];
}
i = j;
}
zOut[i] = 0;
*pnOut = i;
}
/*
** Stem the input word zIn[0..nIn-1]. Store the output in zOut.
** zOut is at least big enough to hold nIn bytes. Write the actual
** size of the output word (exclusive of the '\0' terminator) into *pnOut.
**
** Any upper-case characters in the US-ASCII character set ([A-Z])
** are converted to lower case. Upper-case UTF characters are
** unchanged.
**
** Words that are longer than about 20 bytes are stemmed by retaining
** a few bytes from the beginning and the end of the word. If the
** word contains digits, 3 bytes are taken from the beginning and
** 3 bytes from the end. For long words without digits, 10 bytes
** are taken from each end. US-ASCII case folding still applies.
**
** If the input word contains not digits but does characters not
** in [a-zA-Z] then no stemming is attempted and this routine just
** copies the input into the input into the output with US-ASCII
** case folding.
**
** Stemming never increases the length of the word. So there is
** no chance of overflowing the zOut buffer.
*/
static void porter_stemmer(const char *zIn, int nIn, char *zOut, int *pnOut){
int i, j, c;
char zReverse[28];
char *z, *z2;
if( nIn<3 || nIn>=sizeof(zReverse)-7 ){
/* The word is too big or too small for the porter stemmer.
** Fallback to the copy stemmer */
copy_stemmer(zIn, nIn, zOut, pnOut);
return;
}
for(i=0, j=sizeof(zReverse)-6; i<nIn; i++, j--){
c = zIn[i];
if( c>='A' && c<='Z' ){
zReverse[j] = c + 'a' - 'A';
}else if( c>='a' && c<='z' ){
zReverse[j] = c;
}else{
/* The use of a character not in [a-zA-Z] means that we fallback
** to the copy stemmer */
copy_stemmer(zIn, nIn, zOut, pnOut);
return;
}
}
memset(&zReverse[sizeof(zReverse)-5], 0, 5);
z = &zReverse[j+1];
/* Step 1a */
if( z[0]=='s' ){
if(
!stem(&z, "sess", "ss", 0) &&
!stem(&z, "sei", "i", 0) &&
!stem(&z, "ss", "ss", 0)
){
z++;
}
}
/* Step 1b */
z2 = z;
if( stem(&z, "dee", "ee", m_gt_0) ){
/* Do nothing. The work was all in the test */
}else if(
(stem(&z, "gni", "", hasVowel) || stem(&z, "de", "", hasVowel))
&& z!=z2
){
if( stem(&z, "ta", "ate", 0) ||
stem(&z, "lb", "ble", 0) ||
stem(&z, "zi", "ize", 0) ){
/* Do nothing. The work was all in the test */
}else if( doubleConsonant(z) && (*z!='l' && *z!='s' && *z!='z') ){
z++;
}else if( m_eq_1(z) && star_oh(z) ){
*(--z) = 'e';
}
}
/* Step 1c */
if( z[0]=='y' && hasVowel(z+1) ){
z[0] = 'i';
}
/* Step 2 */
switch( z[1] ){
case 'a':
stem(&z, "lanoita", "ate", m_gt_0) ||
stem(&z, "lanoit", "tion", m_gt_0);
break;
case 'c':
stem(&z, "icne", "ence", m_gt_0) ||
stem(&z, "icna", "ance", m_gt_0);
break;
case 'e':
stem(&z, "rezi", "ize", m_gt_0);
break;
case 'g':
stem(&z, "igol", "log", m_gt_0);
break;
case 'l':
stem(&z, "ilb", "ble", m_gt_0) ||
stem(&z, "illa", "al", m_gt_0) ||
stem(&z, "iltne", "ent", m_gt_0) ||
stem(&z, "ile", "e", m_gt_0) ||
stem(&z, "ilsuo", "ous", m_gt_0);
break;
case 'o':
stem(&z, "noitazi", "ize", m_gt_0) ||
stem(&z, "noita", "ate", m_gt_0) ||
stem(&z, "rota", "ate", m_gt_0);
break;
case 's':
stem(&z, "msila", "al", m_gt_0) ||
stem(&z, "ssenevi", "ive", m_gt_0) ||
stem(&z, "ssenluf", "ful", m_gt_0) ||
stem(&z, "ssensuo", "ous", m_gt_0);
break;
case 't':
stem(&z, "itila", "al", m_gt_0) ||
stem(&z, "itivi", "ive", m_gt_0) ||
stem(&z, "itilib", "ble", m_gt_0);
break;
}
/* Step 3 */
switch( z[0] ){
case 'e':
stem(&z, "etaci", "ic", m_gt_0) ||
stem(&z, "evita", "", m_gt_0) ||
stem(&z, "ezila", "al", m_gt_0);
break;
case 'i':
stem(&z, "itici", "ic", m_gt_0);
break;
case 'l':
stem(&z, "laci", "ic", m_gt_0) ||
stem(&z, "luf", "", m_gt_0);
break;
case 's':
stem(&z, "ssen", "", m_gt_0);
break;
}
/* Step 4 */
switch( z[1] ){
case 'a':
if( z[0]=='l' && m_gt_1(z+2) ){
z += 2;
}
break;
case 'c':
if( z[0]=='e' && z[2]=='n' && (z[3]=='a' || z[3]=='e') && m_gt_1(z+4) ){
z += 4;
}
break;
case 'e':
if( z[0]=='r' && m_gt_1(z+2) ){
z += 2;
}
break;
case 'i':
if( z[0]=='c' && m_gt_1(z+2) ){
z += 2;
}
break;
case 'l':
if( z[0]=='e' && z[2]=='b' && (z[3]=='a' || z[3]=='i') && m_gt_1(z+4) ){
z += 4;
}
break;
case 'n':
if( z[0]=='t' ){
if( z[2]=='a' ){
if( m_gt_1(z+3) ){
z += 3;
}
}else if( z[2]=='e' ){
stem(&z, "tneme", "", m_gt_1) ||
stem(&z, "tnem", "", m_gt_1) ||
stem(&z, "tne", "", m_gt_1);
}
}
break;
case 'o':
if( z[0]=='u' ){
if( m_gt_1(z+2) ){
z += 2;
}
}else if( z[3]=='s' || z[3]=='t' ){
stem(&z, "noi", "", m_gt_1);
}
break;
case 's':
if( z[0]=='m' && z[2]=='i' && m_gt_1(z+3) ){
z += 3;
}
break;
case 't':
stem(&z, "eta", "", m_gt_1) ||
stem(&z, "iti", "", m_gt_1);
break;
case 'u':
if( z[0]=='s' && z[2]=='o' && m_gt_1(z+3) ){
z += 3;
}
break;
case 'v':
case 'z':
if( z[0]=='e' && z[2]=='i' && m_gt_1(z+3) ){
z += 3;
}
break;
}
/* Step 5a */
if( z[0]=='e' ){
if( m_gt_1(z+1) ){
z++;
}else if( m_eq_1(z+1) && !star_oh(z+1) ){
z++;
}
}
/* Step 5b */
if( m_gt_1(z) && z[0]=='l' && z[1]=='l' ){
z++;
}
/* z[] is now the stemmed word in reverse order. Flip it back
** around into forward order and return.
*/
*pnOut = i = strlen(z);
zOut[i] = 0;
while( *z ){
zOut[--i] = *(z++);
}
}
/*
** Characters that can be part of a token. We assume any character
** whose value is greater than 0x80 (any UTF character) can be
** part of a token. In other words, delimiters all must have
** values of 0x7f or lower.
*/
static const char porterIdChar[] = {
/* x0 x1 x2 x3 x4 x5 x6 x7 x8 x9 xA xB xC xD xE xF */
1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, /* 3x */
0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, /* 4x */
1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, /* 5x */
0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, /* 6x */
1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, /* 7x */
};
#define isDelim(C) (((ch=C)&0x80)==0 && (ch<0x30 || !porterIdChar[ch-0x30]))
/*
** Extract the next token from a tokenization cursor. The cursor must
** have been opened by a prior call to porterOpen().
*/
static int porterNext(
sqlite3_tokenizer_cursor *pCursor, /* Cursor returned by porterOpen */
const char **pzToken, /* OUT: *pzToken is the token text */
int *pnBytes, /* OUT: Number of bytes in token */
int *piStartOffset, /* OUT: Starting offset of token */
int *piEndOffset, /* OUT: Ending offset of token */
int *piPosition /* OUT: Position integer of token */
){
porter_tokenizer_cursor *c = (porter_tokenizer_cursor *) pCursor;
const char *z = c->zInput;
while( c->iOffset<c->nInput ){
int iStartOffset, ch;
/* Scan past delimiter characters */
while( c->iOffset<c->nInput && isDelim(z[c->iOffset]) ){
c->iOffset++;
}
/* Count non-delimiter characters. */
iStartOffset = c->iOffset;
while( c->iOffset<c->nInput && !isDelim(z[c->iOffset]) ){
c->iOffset++;
}
if( c->iOffset>iStartOffset ){
int n = c->iOffset-iStartOffset;
if( n>c->nAllocated ){
c->nAllocated = n+20;
c->zToken = sqlite3_realloc(c->zToken, c->nAllocated);
if( c->zToken==NULL ) return SQLITE_NOMEM;
}
porter_stemmer(&z[iStartOffset], n, c->zToken, pnBytes);
*pzToken = c->zToken;
*piStartOffset = iStartOffset;
*piEndOffset = c->iOffset;
*piPosition = c->iToken++;
return SQLITE_OK;
}
}
return SQLITE_DONE;
}
/*
** The set of routines that implement the porter-stemmer tokenizer
*/
static const sqlite3_tokenizer_module porterTokenizerModule = {
0,
porterCreate,
porterDestroy,
porterOpen,
porterClose,
porterNext,
};
/*
** Allocate a new porter tokenizer. Return a pointer to the new
** tokenizer in *ppModule
*/
void sqlite3Fts2PorterTokenizerModule(
sqlite3_tokenizer_module const**ppModule
){
*ppModule = &porterTokenizerModule;
}
#endif /* !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS2) */
-375
View File
@@ -1,375 +0,0 @@
/*
** 2007 June 22
**
** The author disclaims copyright to this source code. In place of
** a legal notice, here is a blessing:
**
** May you do good and not evil.
** May you find forgiveness for yourself and forgive others.
** May you share freely, never taking more than you give.
**
******************************************************************************
**
** This is part of an SQLite module implementing full-text search.
** This particular file implements the generic tokenizer interface.
*/
/*
** The code in this file is only compiled if:
**
** * The FTS2 module is being built as an extension
** (in which case SQLITE_CORE is not defined), or
**
** * The FTS2 module is being built into the core of
** SQLite (in which case SQLITE_ENABLE_FTS2 is defined).
*/
#if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS2)
#include "sqlite3.h"
#include "sqlite3ext.h"
SQLITE_EXTENSION_INIT3
#include "fts2_hash.h"
#include "fts2_tokenizer.h"
#include <assert.h>
/*
** Implementation of the SQL scalar function for accessing the underlying
** hash table. This function may be called as follows:
**
** SELECT <function-name>(<key-name>);
** SELECT <function-name>(<key-name>, <pointer>);
**
** where <function-name> is the name passed as the second argument
** to the sqlite3Fts2InitHashTable() function (e.g. 'fts2_tokenizer').
**
** If the <pointer> argument is specified, it must be a blob value
** containing a pointer to be stored as the hash data corresponding
** to the string <key-name>. If <pointer> is not specified, then
** the string <key-name> must already exist in the has table. Otherwise,
** an error is returned.
**
** Whether or not the <pointer> argument is specified, the value returned
** is a blob containing the pointer stored as the hash data corresponding
** to string <key-name> (after the hash-table is updated, if applicable).
*/
static void scalarFunc(
sqlite3_context *context,
int argc,
sqlite3_value **argv
){
fts2Hash *pHash;
void *pPtr = 0;
const unsigned char *zName;
int nName;
assert( argc==1 || argc==2 );
pHash = (fts2Hash *)sqlite3_user_data(context);
zName = sqlite3_value_text(argv[0]);
nName = sqlite3_value_bytes(argv[0])+1;
if( argc==2 ){
void *pOld;
int n = sqlite3_value_bytes(argv[1]);
if( n!=sizeof(pPtr) ){
sqlite3_result_error(context, "argument type mismatch", -1);
return;
}
pPtr = *(void **)sqlite3_value_blob(argv[1]);
pOld = sqlite3Fts2HashInsert(pHash, (void *)zName, nName, pPtr);
if( pOld==pPtr ){
sqlite3_result_error(context, "out of memory", -1);
return;
}
}else{
pPtr = sqlite3Fts2HashFind(pHash, zName, nName);
if( !pPtr ){
char *zErr = sqlite3_mprintf("unknown tokenizer: %s", zName);
sqlite3_result_error(context, zErr, -1);
sqlite3_free(zErr);
return;
}
}
sqlite3_result_blob(context, (void *)&pPtr, sizeof(pPtr), SQLITE_TRANSIENT);
}
#ifdef SQLITE_TEST
#if defined(INCLUDE_SQLITE_TCL_H)
# include "sqlite_tcl.h"
#else
# include "tcl.h"
#endif
#include <string.h>
/*
** Implementation of a special SQL scalar function for testing tokenizers
** designed to be used in concert with the Tcl testing framework. This
** function must be called with two arguments:
**
** SELECT <function-name>(<key-name>, <input-string>);
** SELECT <function-name>(<key-name>, <pointer>);
**
** where <function-name> is the name passed as the second argument
** to the sqlite3Fts2InitHashTable() function (e.g. 'fts2_tokenizer')
** concatenated with the string '_test' (e.g. 'fts2_tokenizer_test').
**
** The return value is a string that may be interpreted as a Tcl
** list. For each token in the <input-string>, three elements are
** added to the returned list. The first is the token position, the
** second is the token text (folded, stemmed, etc.) and the third is the
** substring of <input-string> associated with the token. For example,
** using the built-in "simple" tokenizer:
**
** SELECT fts_tokenizer_test('simple', 'I don't see how');
**
** will return the string:
**
** "{0 i I 1 dont don't 2 see see 3 how how}"
**
*/
static void testFunc(
sqlite3_context *context,
int argc,
sqlite3_value **argv
){
fts2Hash *pHash;
sqlite3_tokenizer_module *p;
sqlite3_tokenizer *pTokenizer = 0;
sqlite3_tokenizer_cursor *pCsr = 0;
const char *zErr = 0;
const char *zName;
int nName;
const char *zInput;
int nInput;
const char *zArg = 0;
const char *zToken;
int nToken;
int iStart;
int iEnd;
int iPos;
Tcl_Obj *pRet;
assert( argc==2 || argc==3 );
nName = sqlite3_value_bytes(argv[0]);
zName = (const char *)sqlite3_value_text(argv[0]);
nInput = sqlite3_value_bytes(argv[argc-1]);
zInput = (const char *)sqlite3_value_text(argv[argc-1]);
if( argc==3 ){
zArg = (const char *)sqlite3_value_text(argv[1]);
}
pHash = (fts2Hash *)sqlite3_user_data(context);
p = (sqlite3_tokenizer_module *)sqlite3Fts2HashFind(pHash, zName, nName+1);
if( !p ){
char *zErr = sqlite3_mprintf("unknown tokenizer: %s", zName);
sqlite3_result_error(context, zErr, -1);
sqlite3_free(zErr);
return;
}
pRet = Tcl_NewObj();
Tcl_IncrRefCount(pRet);
if( SQLITE_OK!=p->xCreate(zArg ? 1 : 0, &zArg, &pTokenizer) ){
zErr = "error in xCreate()";
goto finish;
}
pTokenizer->pModule = p;
if( SQLITE_OK!=p->xOpen(pTokenizer, zInput, nInput, &pCsr) ){
zErr = "error in xOpen()";
goto finish;
}
pCsr->pTokenizer = pTokenizer;
while( SQLITE_OK==p->xNext(pCsr, &zToken, &nToken, &iStart, &iEnd, &iPos) ){
Tcl_ListObjAppendElement(0, pRet, Tcl_NewIntObj(iPos));
Tcl_ListObjAppendElement(0, pRet, Tcl_NewStringObj(zToken, nToken));
zToken = &zInput[iStart];
nToken = iEnd-iStart;
Tcl_ListObjAppendElement(0, pRet, Tcl_NewStringObj(zToken, nToken));
}
if( SQLITE_OK!=p->xClose(pCsr) ){
zErr = "error in xClose()";
goto finish;
}
if( SQLITE_OK!=p->xDestroy(pTokenizer) ){
zErr = "error in xDestroy()";
goto finish;
}
finish:
if( zErr ){
sqlite3_result_error(context, zErr, -1);
}else{
sqlite3_result_text(context, Tcl_GetString(pRet), -1, SQLITE_TRANSIENT);
}
Tcl_DecrRefCount(pRet);
}
static
int registerTokenizer(
sqlite3 *db,
char *zName,
const sqlite3_tokenizer_module *p
){
int rc;
sqlite3_stmt *pStmt;
const char zSql[] = "SELECT fts2_tokenizer(?, ?)";
rc = sqlite3_prepare_v2(db, zSql, -1, &pStmt, 0);
if( rc!=SQLITE_OK ){
return rc;
}
sqlite3_bind_text(pStmt, 1, zName, -1, SQLITE_STATIC);
sqlite3_bind_blob(pStmt, 2, &p, sizeof(p), SQLITE_STATIC);
sqlite3_step(pStmt);
return sqlite3_finalize(pStmt);
}
static
int queryFts2Tokenizer(
sqlite3 *db,
char *zName,
const sqlite3_tokenizer_module **pp
){
int rc;
sqlite3_stmt *pStmt;
const char zSql[] = "SELECT fts2_tokenizer(?)";
*pp = 0;
rc = sqlite3_prepare_v2(db, zSql, -1, &pStmt, 0);
if( rc!=SQLITE_OK ){
return rc;
}
sqlite3_bind_text(pStmt, 1, zName, -1, SQLITE_STATIC);
if( SQLITE_ROW==sqlite3_step(pStmt) ){
if( sqlite3_column_type(pStmt, 0)==SQLITE_BLOB ){
memcpy(pp, sqlite3_column_blob(pStmt, 0), sizeof(*pp));
}
}
return sqlite3_finalize(pStmt);
}
void sqlite3Fts2SimpleTokenizerModule(sqlite3_tokenizer_module const**ppModule);
/*
** Implementation of the scalar function fts2_tokenizer_internal_test().
** This function is used for testing only, it is not included in the
** build unless SQLITE_TEST is defined.
**
** The purpose of this is to test that the fts2_tokenizer() function
** can be used as designed by the C-code in the queryFts2Tokenizer and
** registerTokenizer() functions above. These two functions are repeated
** in the README.tokenizer file as an example, so it is important to
** test them.
**
** To run the tests, evaluate the fts2_tokenizer_internal_test() scalar
** function with no arguments. An assert() will fail if a problem is
** detected. i.e.:
**
** SELECT fts2_tokenizer_internal_test();
**
*/
static void intTestFunc(
sqlite3_context *context,
int argc,
sqlite3_value **argv
){
int rc;
const sqlite3_tokenizer_module *p1;
const sqlite3_tokenizer_module *p2;
sqlite3 *db = (sqlite3 *)sqlite3_user_data(context);
/* Test the query function */
sqlite3Fts2SimpleTokenizerModule(&p1);
rc = queryFts2Tokenizer(db, "simple", &p2);
assert( rc==SQLITE_OK );
assert( p1==p2 );
rc = queryFts2Tokenizer(db, "nosuchtokenizer", &p2);
assert( rc==SQLITE_ERROR );
assert( p2==0 );
assert( 0==strcmp(sqlite3_errmsg(db), "unknown tokenizer: nosuchtokenizer") );
/* Test the storage function */
rc = registerTokenizer(db, "nosuchtokenizer", p1);
assert( rc==SQLITE_OK );
rc = queryFts2Tokenizer(db, "nosuchtokenizer", &p2);
assert( rc==SQLITE_OK );
assert( p2==p1 );
sqlite3_result_text(context, "ok", -1, SQLITE_STATIC);
}
#endif
/*
** Set up SQL objects in database db used to access the contents of
** the hash table pointed to by argument pHash. The hash table must
** been initialized to use string keys, and to take a private copy
** of the key when a value is inserted. i.e. by a call similar to:
**
** sqlite3Fts2HashInit(pHash, FTS2_HASH_STRING, 1);
**
** This function adds a scalar function (see header comment above
** scalarFunc() in this file for details) and, if ENABLE_TABLE is
** defined at compilation time, a temporary virtual table (see header
** comment above struct HashTableVtab) to the database schema. Both
** provide read/write access to the contents of *pHash.
**
** The third argument to this function, zName, is used as the name
** of both the scalar and, if created, the virtual table.
*/
int sqlite3Fts2InitHashTable(
sqlite3 *db,
fts2Hash *pHash,
const char *zName
){
int rc = SQLITE_OK;
void *p = (void *)pHash;
const int any = SQLITE_ANY;
char *zTest = 0;
char *zTest2 = 0;
#ifdef SQLITE_TEST
void *pdb = (void *)db;
zTest = sqlite3_mprintf("%s_test", zName);
zTest2 = sqlite3_mprintf("%s_internal_test", zName);
if( !zTest || !zTest2 ){
rc = SQLITE_NOMEM;
}
#endif
if( rc!=SQLITE_OK
|| (rc = sqlite3_create_function(db, zName, 1, any, p, scalarFunc, 0, 0))
|| (rc = sqlite3_create_function(db, zName, 2, any, p, scalarFunc, 0, 0))
#ifdef SQLITE_TEST
|| (rc = sqlite3_create_function(db, zTest, 2, any, p, testFunc, 0, 0))
|| (rc = sqlite3_create_function(db, zTest, 3, any, p, testFunc, 0, 0))
|| (rc = sqlite3_create_function(db, zTest2, 0, any, pdb, intTestFunc, 0, 0))
#endif
);
sqlite3_free(zTest);
sqlite3_free(zTest2);
return rc;
}
#endif /* !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS2) */
-145
View File
@@ -1,145 +0,0 @@
/*
** 2006 July 10
**
** The author disclaims copyright to this source code.
**
*************************************************************************
** Defines the interface to tokenizers used by fulltext-search. There
** are three basic components:
**
** sqlite3_tokenizer_module is a singleton defining the tokenizer
** interface functions. This is essentially the class structure for
** tokenizers.
**
** sqlite3_tokenizer is used to define a particular tokenizer, perhaps
** including customization information defined at creation time.
**
** sqlite3_tokenizer_cursor is generated by a tokenizer to generate
** tokens from a particular input.
*/
#ifndef _FTS2_TOKENIZER_H_
#define _FTS2_TOKENIZER_H_
/* TODO(shess) Only used for SQLITE_OK and SQLITE_DONE at this time.
** If tokenizers are to be allowed to call sqlite3_*() functions, then
** we will need a way to register the API consistently.
*/
#include "sqlite3.h"
/*
** Structures used by the tokenizer interface. When a new tokenizer
** implementation is registered, the caller provides a pointer to
** an sqlite3_tokenizer_module containing pointers to the callback
** functions that make up an implementation.
**
** When an fts2 table is created, it passes any arguments passed to
** the tokenizer clause of the CREATE VIRTUAL TABLE statement to the
** sqlite3_tokenizer_module.xCreate() function of the requested tokenizer
** implementation. The xCreate() function in turn returns an
** sqlite3_tokenizer structure representing the specific tokenizer to
** be used for the fts2 table (customized by the tokenizer clause arguments).
**
** To tokenize an input buffer, the sqlite3_tokenizer_module.xOpen()
** method is called. It returns an sqlite3_tokenizer_cursor object
** that may be used to tokenize a specific input buffer based on
** the tokenization rules supplied by a specific sqlite3_tokenizer
** object.
*/
typedef struct sqlite3_tokenizer_module sqlite3_tokenizer_module;
typedef struct sqlite3_tokenizer sqlite3_tokenizer;
typedef struct sqlite3_tokenizer_cursor sqlite3_tokenizer_cursor;
struct sqlite3_tokenizer_module {
/*
** Structure version. Should always be set to 0.
*/
int iVersion;
/*
** Create a new tokenizer. The values in the argv[] array are the
** arguments passed to the "tokenizer" clause of the CREATE VIRTUAL
** TABLE statement that created the fts2 table. For example, if
** the following SQL is executed:
**
** CREATE .. USING fts2( ... , tokenizer <tokenizer-name> arg1 arg2)
**
** then argc is set to 2, and the argv[] array contains pointers
** to the strings "arg1" and "arg2".
**
** This method should return either SQLITE_OK (0), or an SQLite error
** code. If SQLITE_OK is returned, then *ppTokenizer should be set
** to point at the newly created tokenizer structure. The generic
** sqlite3_tokenizer.pModule variable should not be initialized by
** this callback. The caller will do so.
*/
int (*xCreate)(
int argc, /* Size of argv array */
const char *const*argv, /* Tokenizer argument strings */
sqlite3_tokenizer **ppTokenizer /* OUT: Created tokenizer */
);
/*
** Destroy an existing tokenizer. The fts2 module calls this method
** exactly once for each successful call to xCreate().
*/
int (*xDestroy)(sqlite3_tokenizer *pTokenizer);
/*
** Create a tokenizer cursor to tokenize an input buffer. The caller
** is responsible for ensuring that the input buffer remains valid
** until the cursor is closed (using the xClose() method).
*/
int (*xOpen)(
sqlite3_tokenizer *pTokenizer, /* Tokenizer object */
const char *pInput, int nBytes, /* Input buffer */
sqlite3_tokenizer_cursor **ppCursor /* OUT: Created tokenizer cursor */
);
/*
** Destroy an existing tokenizer cursor. The fts2 module calls this
** method exactly once for each successful call to xOpen().
*/
int (*xClose)(sqlite3_tokenizer_cursor *pCursor);
/*
** Retrieve the next token from the tokenizer cursor pCursor. This
** method should either return SQLITE_OK and set the values of the
** "OUT" variables identified below, or SQLITE_DONE to indicate that
** the end of the buffer has been reached, or an SQLite error code.
**
** *ppToken should be set to point at a buffer containing the
** normalized version of the token (i.e. after any case-folding and/or
** stemming has been performed). *pnBytes should be set to the length
** of this buffer in bytes. The input text that generated the token is
** identified by the byte offsets returned in *piStartOffset and
** *piEndOffset.
**
** The buffer *ppToken is set to point at is managed by the tokenizer
** implementation. It is only required to be valid until the next call
** to xNext() or xClose().
*/
/* TODO(shess) current implementation requires pInput to be
** nul-terminated. This should either be fixed, or pInput/nBytes
** should be converted to zInput.
*/
int (*xNext)(
sqlite3_tokenizer_cursor *pCursor, /* Tokenizer cursor */
const char **ppToken, int *pnBytes, /* OUT: Normalized text for token */
int *piStartOffset, /* OUT: Byte offset of token in input buffer */
int *piEndOffset, /* OUT: Byte offset of end of token in input buffer */
int *piPosition /* OUT: Number of tokens returned before this one */
);
};
struct sqlite3_tokenizer {
const sqlite3_tokenizer_module *pModule; /* The module for this tokenizer */
/* Tokenizer implementations will typically add additional fields */
};
struct sqlite3_tokenizer_cursor {
sqlite3_tokenizer *pTokenizer; /* Tokenizer for this cursor. */
/* Tokenizer implementations will typically add additional fields */
};
#endif /* _FTS2_TOKENIZER_H_ */
-233
View File
@@ -1,233 +0,0 @@
/*
** 2006 Oct 10
**
** The author disclaims copyright to this source code. In place of
** a legal notice, here is a blessing:
**
** May you do good and not evil.
** May you find forgiveness for yourself and forgive others.
** May you share freely, never taking more than you give.
**
******************************************************************************
**
** Implementation of the "simple" full-text-search tokenizer.
*/
/*
** The code in this file is only compiled if:
**
** * The FTS2 module is being built as an extension
** (in which case SQLITE_CORE is not defined), or
**
** * The FTS2 module is being built into the core of
** SQLite (in which case SQLITE_ENABLE_FTS2 is defined).
*/
#if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS2)
#include <assert.h>
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include "sqlite3.h"
#include "sqlite3ext.h"
SQLITE_EXTENSION_INIT3
#include "fts2_tokenizer.h"
typedef struct simple_tokenizer {
sqlite3_tokenizer base;
char delim[128]; /* flag ASCII delimiters */
} simple_tokenizer;
typedef struct simple_tokenizer_cursor {
sqlite3_tokenizer_cursor base;
const char *pInput; /* input we are tokenizing */
int nBytes; /* size of the input */
int iOffset; /* current position in pInput */
int iToken; /* index of next token to be returned */
char *pToken; /* storage for current token */
int nTokenAllocated; /* space allocated to zToken buffer */
} simple_tokenizer_cursor;
/* Forward declaration */
static const sqlite3_tokenizer_module simpleTokenizerModule;
static int simpleDelim(simple_tokenizer *t, unsigned char c){
return c<0x80 && t->delim[c];
}
/*
** Create a new tokenizer instance.
*/
static int simpleCreate(
int argc, const char * const *argv,
sqlite3_tokenizer **ppTokenizer
){
simple_tokenizer *t;
t = (simple_tokenizer *) sqlite3_malloc(sizeof(*t));
if( t==NULL ) return SQLITE_NOMEM;
memset(t, 0, sizeof(*t));
/* TODO(shess) Delimiters need to remain the same from run to run,
** else we need to reindex. One solution would be a meta-table to
** track such information in the database, then we'd only want this
** information on the initial create.
*/
if( argc>1 ){
int i, n = strlen(argv[1]);
for(i=0; i<n; i++){
unsigned char ch = argv[1][i];
/* We explicitly don't support UTF-8 delimiters for now. */
if( ch>=0x80 ){
sqlite3_free(t);
return SQLITE_ERROR;
}
t->delim[ch] = 1;
}
} else {
/* Mark non-alphanumeric ASCII characters as delimiters */
int i;
for(i=1; i<0x80; i++){
t->delim[i] = !((i>='0' && i<='9') || (i>='A' && i<='Z') ||
(i>='a' && i<='z'));
}
}
*ppTokenizer = &t->base;
return SQLITE_OK;
}
/*
** Destroy a tokenizer
*/
static int simpleDestroy(sqlite3_tokenizer *pTokenizer){
sqlite3_free(pTokenizer);
return SQLITE_OK;
}
/*
** Prepare to begin tokenizing a particular string. The input
** string to be tokenized is pInput[0..nBytes-1]. A cursor
** used to incrementally tokenize this string is returned in
** *ppCursor.
*/
static int simpleOpen(
sqlite3_tokenizer *pTokenizer, /* The tokenizer */
const char *pInput, int nBytes, /* String to be tokenized */
sqlite3_tokenizer_cursor **ppCursor /* OUT: Tokenization cursor */
){
simple_tokenizer_cursor *c;
c = (simple_tokenizer_cursor *) sqlite3_malloc(sizeof(*c));
if( c==NULL ) return SQLITE_NOMEM;
c->pInput = pInput;
if( pInput==0 ){
c->nBytes = 0;
}else if( nBytes<0 ){
c->nBytes = (int)strlen(pInput);
}else{
c->nBytes = nBytes;
}
c->iOffset = 0; /* start tokenizing at the beginning */
c->iToken = 0;
c->pToken = NULL; /* no space allocated, yet. */
c->nTokenAllocated = 0;
*ppCursor = &c->base;
return SQLITE_OK;
}
/*
** Close a tokenization cursor previously opened by a call to
** simpleOpen() above.
*/
static int simpleClose(sqlite3_tokenizer_cursor *pCursor){
simple_tokenizer_cursor *c = (simple_tokenizer_cursor *) pCursor;
sqlite3_free(c->pToken);
sqlite3_free(c);
return SQLITE_OK;
}
/*
** Extract the next token from a tokenization cursor. The cursor must
** have been opened by a prior call to simpleOpen().
*/
static int simpleNext(
sqlite3_tokenizer_cursor *pCursor, /* Cursor returned by simpleOpen */
const char **ppToken, /* OUT: *ppToken is the token text */
int *pnBytes, /* OUT: Number of bytes in token */
int *piStartOffset, /* OUT: Starting offset of token */
int *piEndOffset, /* OUT: Ending offset of token */
int *piPosition /* OUT: Position integer of token */
){
simple_tokenizer_cursor *c = (simple_tokenizer_cursor *) pCursor;
simple_tokenizer *t = (simple_tokenizer *) pCursor->pTokenizer;
unsigned char *p = (unsigned char *)c->pInput;
while( c->iOffset<c->nBytes ){
int iStartOffset;
/* Scan past delimiter characters */
while( c->iOffset<c->nBytes && simpleDelim(t, p[c->iOffset]) ){
c->iOffset++;
}
/* Count non-delimiter characters. */
iStartOffset = c->iOffset;
while( c->iOffset<c->nBytes && !simpleDelim(t, p[c->iOffset]) ){
c->iOffset++;
}
if( c->iOffset>iStartOffset ){
int i, n = c->iOffset-iStartOffset;
if( n>c->nTokenAllocated ){
c->nTokenAllocated = n+20;
c->pToken = sqlite3_realloc(c->pToken, c->nTokenAllocated);
if( c->pToken==NULL ) return SQLITE_NOMEM;
}
for(i=0; i<n; i++){
/* TODO(shess) This needs expansion to handle UTF-8
** case-insensitivity.
*/
unsigned char ch = p[iStartOffset+i];
c->pToken[i] = (ch>='A' && ch<='Z') ? (ch - 'A' + 'a') : ch;
}
*ppToken = c->pToken;
*pnBytes = n;
*piStartOffset = iStartOffset;
*piEndOffset = c->iOffset;
*piPosition = c->iToken++;
return SQLITE_OK;
}
}
return SQLITE_DONE;
}
/*
** The set of routines that implement the simple tokenizer
*/
static const sqlite3_tokenizer_module simpleTokenizerModule = {
0,
simpleCreate,
simpleDestroy,
simpleOpen,
simpleClose,
simpleNext,
};
/*
** Allocate a new simple tokenizer. Return a pointer to the new
** tokenizer in *ppModule
*/
void sqlite3Fts2SimpleTokenizerModule(
sqlite3_tokenizer_module const**ppModule
){
*ppModule = &simpleTokenizerModule;
}
#endif /* !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS2) */
-116
View File
@@ -1,116 +0,0 @@
#!/usr/bin/tclsh
#
# This script builds a single C code file holding all of FTS2 code.
# The name of the output file is fts2amal.c. To build this file,
# first do:
#
# make target_source
#
# The make target above moves all of the source code files into
# a subdirectory named "tsrc". (This script expects to find the files
# there and will not work if they are not found.)
#
# After the "tsrc" directory has been created and populated, run
# this script:
#
# tclsh mkfts2amal.tcl
#
# The amalgamated FTS2 code will be written into fts2amal.c
#
# Open the output file and write a header comment at the beginning
# of the file.
#
set out [open fts2amal.c w]
set today [clock format [clock seconds] -format "%Y-%m-%d %H:%M:%S UTC" -gmt 1]
puts $out [subst \
{/******************************************************************************
** This file is an amalgamation of separate C source files from the SQLite
** Full Text Search extension 2 (fts2). By combining all the individual C
** code files into this single large file, the entire code can be compiled
** as a one translation unit. This allows many compilers to do optimizations
** that would not be possible if the files were compiled separately. It also
** makes the code easier to import into other projects.
**
** This amalgamation was generated on $today.
*/}]
# These are the header files used by FTS2. The first time any of these
# files are seen in a #include statement in the C code, include the complete
# text of the file in-line. The file only needs to be included once.
#
foreach hdr {
fts2.h
fts2_hash.h
fts2_tokenizer.h
sqlite3.h
sqlite3ext.h
} {
set available_hdr($hdr) 1
}
# 78 stars used for comment formatting.
set s78 \
{*****************************************************************************}
# Insert a comment into the code
#
proc section_comment {text} {
global out s78
set n [string length $text]
set nstar [expr {60 - $n}]
set stars [string range $s78 0 $nstar]
puts $out "/************** $text $stars/"
}
# Read the source file named $filename and write it into the
# sqlite3.c output file. If any #include statements are seen,
# process them approprately.
#
proc copy_file {filename} {
global seen_hdr available_hdr out
set tail [file tail $filename]
section_comment "Begin file $tail"
set in [open $filename r]
while {![eof $in]} {
set line [gets $in]
if {[regexp {^#\s*include\s+["<]([^">]+)[">]} $line all hdr]} {
if {[info exists available_hdr($hdr)]} {
if {$available_hdr($hdr)} {
section_comment "Include $hdr in the middle of $tail"
copy_file tsrc/$hdr
section_comment "Continuing where we left off in $tail"
}
} elseif {![info exists seen_hdr($hdr)]} {
set seen_hdr($hdr) 1
puts $out $line
}
} elseif {[regexp {^#ifdef __cplusplus} $line]} {
puts $out "#if 0"
} elseif {[regexp {^#line} $line]} {
# Skip #line directives.
} else {
puts $out $line
}
}
close $in
section_comment "End of $tail"
}
# Process the source files. Process files containing commonly
# used subroutines first in order to help the compiler find
# inlining opportunities.
#
foreach file {
fts2.c
fts2_hash.c
fts2_porter.c
fts2_tokenizer.c
fts2_tokenizer1.c
fts2_icu.c
} {
copy_file tsrc/$file
}
close $out
+121 -46
View File
@@ -640,6 +640,7 @@ static void fts3DeclareVtab(int *pRc, Fts3Table *p){
zLanguageid = (p->zLanguageid ? p->zLanguageid : "__langid");
sqlite3_vtab_config(p->db, SQLITE_VTAB_CONSTRAINT_SUPPORT, 1);
sqlite3_vtab_config(p->db, SQLITE_VTAB_INNOCUOUS);
/* Create a list of user columns for the virtual table */
zCols = sqlite3_mprintf("%Q, ", p->azColumn[0]);
@@ -2888,7 +2889,7 @@ static int fts3TermSelectMerge(
**
** Similar padding is added in the fts3DoclistOrMerge() function.
*/
pTS->aaOutput[0] = sqlite3_malloc(nDoclist + FTS3_VARINT_MAX + 1);
pTS->aaOutput[0] = sqlite3_malloc64((i64)nDoclist + FTS3_VARINT_MAX + 1);
pTS->anOutput[0] = nDoclist;
if( pTS->aaOutput[0] ){
memcpy(pTS->aaOutput[0], aDoclist, nDoclist);
@@ -3889,6 +3890,8 @@ static int fts3RenameMethod(
rc = sqlite3Fts3PendingTermsFlush(p);
}
p->bIgnoreSavepoint = 1;
if( p->zContentTbl==0 ){
fts3DbExec(&rc, db,
"ALTER TABLE %Q.'%q_content' RENAME TO '%q_content';",
@@ -3916,6 +3919,8 @@ static int fts3RenameMethod(
"ALTER TABLE %Q.'%q_segdir' RENAME TO '%q_segdir';",
p->zDb, p->zName, zName
);
p->bIgnoreSavepoint = 0;
return rc;
}
@@ -3926,12 +3931,28 @@ static int fts3RenameMethod(
*/
static int fts3SavepointMethod(sqlite3_vtab *pVtab, int iSavepoint){
int rc = SQLITE_OK;
UNUSED_PARAMETER(iSavepoint);
assert( ((Fts3Table *)pVtab)->inTransaction );
assert( ((Fts3Table *)pVtab)->mxSavepoint <= iSavepoint );
TESTONLY( ((Fts3Table *)pVtab)->mxSavepoint = iSavepoint );
if( ((Fts3Table *)pVtab)->bIgnoreSavepoint==0 ){
rc = fts3SyncMethod(pVtab);
Fts3Table *pTab = (Fts3Table*)pVtab;
assert( pTab->inTransaction );
assert( pTab->mxSavepoint<=iSavepoint );
TESTONLY( pTab->mxSavepoint = iSavepoint );
if( pTab->bIgnoreSavepoint==0 ){
if( fts3HashCount(&pTab->aIndex[0].hPending)>0 ){
char *zSql = sqlite3_mprintf("INSERT INTO %Q.%Q(%Q) VALUES('flush')",
pTab->zDb, pTab->zName, pTab->zName
);
if( zSql ){
pTab->bIgnoreSavepoint = 1;
rc = sqlite3_exec(pTab->db, zSql, 0, 0, 0);
pTab->bIgnoreSavepoint = 0;
sqlite3_free(zSql);
}else{
rc = SQLITE_NOMEM;
}
}
if( rc==SQLITE_OK ){
pTab->iSavepoint = iSavepoint+1;
}
}
return rc;
}
@@ -3942,12 +3963,11 @@ static int fts3SavepointMethod(sqlite3_vtab *pVtab, int iSavepoint){
** This is a no-op.
*/
static int fts3ReleaseMethod(sqlite3_vtab *pVtab, int iSavepoint){
TESTONLY( Fts3Table *p = (Fts3Table*)pVtab );
UNUSED_PARAMETER(iSavepoint);
UNUSED_PARAMETER(pVtab);
assert( p->inTransaction );
assert( p->mxSavepoint >= iSavepoint );
TESTONLY( p->mxSavepoint = iSavepoint-1 );
Fts3Table *pTab = (Fts3Table*)pVtab;
assert( pTab->inTransaction );
assert( pTab->mxSavepoint >= iSavepoint );
TESTONLY( pTab->mxSavepoint = iSavepoint-1 );
pTab->iSavepoint = iSavepoint;
return SQLITE_OK;
}
@@ -3957,11 +3977,13 @@ static int fts3ReleaseMethod(sqlite3_vtab *pVtab, int iSavepoint){
** Discard the contents of the pending terms table.
*/
static int fts3RollbackToMethod(sqlite3_vtab *pVtab, int iSavepoint){
Fts3Table *p = (Fts3Table*)pVtab;
Fts3Table *pTab = (Fts3Table*)pVtab;
UNUSED_PARAMETER(iSavepoint);
assert( p->inTransaction );
TESTONLY( p->mxSavepoint = iSavepoint );
sqlite3Fts3PendingTermsClear(p);
assert( pTab->inTransaction );
TESTONLY( pTab->mxSavepoint = iSavepoint );
if( (iSavepoint+1)<=pTab->iSavepoint ){
sqlite3Fts3PendingTermsClear(pTab);
}
return SQLITE_OK;
}
@@ -3980,8 +4002,42 @@ static int fts3ShadowName(const char *zName){
return 0;
}
/*
** Implementation of the xIntegrity() method on the FTS3/FTS4 virtual
** table.
*/
static int fts3IntegrityMethod(
sqlite3_vtab *pVtab, /* The virtual table to be checked */
const char *zSchema, /* Name of schema in which pVtab lives */
const char *zTabname, /* Name of the pVTab table */
int isQuick, /* True if this is a quick_check */
char **pzErr /* Write error message here */
){
Fts3Table *p = (Fts3Table*)pVtab;
int rc = SQLITE_OK;
int bOk = 0;
UNUSED_PARAMETER(isQuick);
rc = sqlite3Fts3IntegrityCheck(p, &bOk);
assert( rc!=SQLITE_CORRUPT_VTAB );
if( rc==SQLITE_ERROR || (rc&0xFF)==SQLITE_CORRUPT ){
*pzErr = sqlite3_mprintf("unable to validate the inverted index for"
" FTS%d table %s.%s: %s",
p->bFts4 ? 4 : 3, zSchema, zTabname, sqlite3_errstr(rc));
if( *pzErr ) rc = SQLITE_OK;
}else if( rc==SQLITE_OK && bOk==0 ){
*pzErr = sqlite3_mprintf("malformed inverted index for FTS%d table %s.%s",
p->bFts4 ? 4 : 3, zSchema, zTabname);
if( *pzErr==0 ) rc = SQLITE_NOMEM;
}
sqlite3Fts3SegmentsClose(p);
return rc;
}
static const sqlite3_module fts3Module = {
/* iVersion */ 3,
/* iVersion */ 4,
/* xCreate */ fts3CreateMethod,
/* xConnect */ fts3ConnectMethod,
/* xBestIndex */ fts3BestIndexMethod,
@@ -4005,6 +4061,7 @@ static const sqlite3_module fts3Module = {
/* xRelease */ fts3ReleaseMethod,
/* xRollbackTo */ fts3RollbackToMethod,
/* xShadowName */ fts3ShadowName,
/* xIntegrity */ fts3IntegrityMethod,
};
/*
@@ -4308,8 +4365,7 @@ static int fts3EvalDeferredPhrase(Fts3Cursor *pCsr, Fts3Phrase *pPhrase){
char *aPoslist = 0; /* Position list for deferred tokens */
int nPoslist = 0; /* Number of bytes in aPoslist */
int iPrev = -1; /* Token number of previous deferred token */
assert( pPhrase->doclist.bFreeList==0 );
char *aFree = (pPhrase->doclist.bFreeList ? pPhrase->doclist.pList : 0);
for(iToken=0; iToken<pPhrase->nToken; iToken++){
Fts3PhraseToken *pToken = &pPhrase->aToken[iToken];
@@ -4323,6 +4379,7 @@ static int fts3EvalDeferredPhrase(Fts3Cursor *pCsr, Fts3Phrase *pPhrase){
if( pList==0 ){
sqlite3_free(aPoslist);
sqlite3_free(aFree);
pPhrase->doclist.pList = 0;
pPhrase->doclist.nList = 0;
return SQLITE_OK;
@@ -4343,6 +4400,7 @@ static int fts3EvalDeferredPhrase(Fts3Cursor *pCsr, Fts3Phrase *pPhrase){
nPoslist = (int)(aOut - aPoslist);
if( nPoslist==0 ){
sqlite3_free(aPoslist);
sqlite3_free(aFree);
pPhrase->doclist.pList = 0;
pPhrase->doclist.nList = 0;
return SQLITE_OK;
@@ -4375,13 +4433,14 @@ static int fts3EvalDeferredPhrase(Fts3Cursor *pCsr, Fts3Phrase *pPhrase){
nDistance = iPrev - nMaxUndeferred;
}
aOut = (char *)sqlite3_malloc(nPoslist+8);
aOut = (char *)sqlite3Fts3MallocZero(nPoslist+FTS3_BUFFER_PADDING);
if( !aOut ){
sqlite3_free(aPoslist);
return SQLITE_NOMEM;
}
pPhrase->doclist.pList = aOut;
assert( p1 && p2 );
if( fts3PoslistPhraseMerge(&aOut, nDistance, 0, 1, &p1, &p2) ){
pPhrase->doclist.bFreeList = 1;
pPhrase->doclist.nList = (int)(aOut - pPhrase->doclist.pList);
@@ -4394,6 +4453,7 @@ static int fts3EvalDeferredPhrase(Fts3Cursor *pCsr, Fts3Phrase *pPhrase){
}
}
if( pPhrase->doclist.pList!=aFree ) sqlite3_free(aFree);
return SQLITE_OK;
}
#endif /* SQLITE_DISABLE_FTS4_DEFERRED */
@@ -4742,7 +4802,7 @@ static int fts3EvalIncrPhraseNext(
if( bEof==0 ){
int nList = 0;
int nByte = a[p->nToken-1].nList;
char *aDoclist = sqlite3_malloc(nByte+FTS3_BUFFER_PADDING);
char *aDoclist = sqlite3_malloc64((i64)nByte+FTS3_BUFFER_PADDING);
if( !aDoclist ) return SQLITE_NOMEM;
memcpy(aDoclist, a[p->nToken-1].pList, nByte+1);
memset(&aDoclist[nByte], 0, FTS3_BUFFER_PADDING);
@@ -5284,9 +5344,8 @@ static void fts3EvalNextRow(
Fts3Expr *pExpr, /* Expr. to advance to next matching row */
int *pRc /* IN/OUT: Error code */
){
if( *pRc==SQLITE_OK ){
if( *pRc==SQLITE_OK && pExpr->bEof==0 ){
int bDescDoclist = pCsr->bDesc; /* Used by DOCID_CMP() macro */
assert( pExpr->bEof==0 );
pExpr->bStart = 1;
switch( pExpr->eType ){
@@ -5568,11 +5627,10 @@ static int fts3EvalTestExpr(
default: {
#ifndef SQLITE_DISABLE_FTS4_DEFERRED
if( pCsr->pDeferred
&& (pExpr->iDocid==pCsr->iPrevId || pExpr->bDeferred)
){
if( pCsr->pDeferred && (pExpr->bDeferred || (
pExpr->iDocid==pCsr->iPrevId && pExpr->pPhrase->doclist.pList
))){
Fts3Phrase *pPhrase = pExpr->pPhrase;
assert( pExpr->bDeferred || pPhrase->doclist.bFreeList==0 );
if( pExpr->bDeferred ){
fts3EvalInvalidatePoslist(pPhrase);
}
@@ -5763,6 +5821,22 @@ static void fts3EvalUpdateCounts(Fts3Expr *pExpr, int nCol){
}
}
/*
** This is an sqlite3Fts3ExprIterate() callback. If the Fts3Expr.aMI[] array
** has not yet been allocated, allocate and zero it. Otherwise, just zero
** it.
*/
static int fts3AllocateMSI(Fts3Expr *pExpr, int iPhrase, void *pCtx){
Fts3Table *pTab = (Fts3Table*)pCtx;
UNUSED_PARAMETER(iPhrase);
if( pExpr->aMI==0 ){
pExpr->aMI = (u32 *)sqlite3_malloc64(pTab->nColumn * 3 * sizeof(u32));
if( pExpr->aMI==0 ) return SQLITE_NOMEM;
}
memset(pExpr->aMI, 0, pTab->nColumn * 3 * sizeof(u32));
return SQLITE_OK;
}
/*
** Expression pExpr must be of type FTSQUERY_PHRASE.
**
@@ -5784,7 +5858,6 @@ static int fts3EvalGatherStats(
if( pExpr->aMI==0 ){
Fts3Table *pTab = (Fts3Table *)pCsr->base.pVtab;
Fts3Expr *pRoot; /* Root of NEAR expression */
Fts3Expr *p; /* Iterator used for several purposes */
sqlite3_int64 iPrevId = pCsr->iPrevId;
sqlite3_int64 iDocid;
@@ -5792,7 +5865,9 @@ static int fts3EvalGatherStats(
/* Find the root of the NEAR expression */
pRoot = pExpr;
while( pRoot->pParent && pRoot->pParent->eType==FTSQUERY_NEAR ){
while( pRoot->pParent
&& (pRoot->pParent->eType==FTSQUERY_NEAR || pRoot->bDeferred)
){
pRoot = pRoot->pParent;
}
iDocid = pRoot->iDocid;
@@ -5800,14 +5875,8 @@ static int fts3EvalGatherStats(
assert( pRoot->bStart );
/* Allocate space for the aMSI[] array of each FTSQUERY_PHRASE node */
for(p=pRoot; p; p=p->pLeft){
Fts3Expr *pE = (p->eType==FTSQUERY_PHRASE?p:p->pRight);
assert( pE->aMI==0 );
pE->aMI = (u32 *)sqlite3_malloc64(pTab->nColumn * 3 * sizeof(u32));
if( !pE->aMI ) return SQLITE_NOMEM;
memset(pE->aMI, 0, pTab->nColumn * 3 * sizeof(u32));
}
rc = sqlite3Fts3ExprIterate(pRoot, fts3AllocateMSI, (void*)pTab);
if( rc!=SQLITE_OK ) return rc;
fts3EvalRestart(pCsr, pRoot, &rc);
while( pCsr->isEof==0 && rc==SQLITE_OK ){
@@ -5963,6 +6032,7 @@ int sqlite3Fts3EvalPhrasePoslist(
u8 bTreeEof = 0;
Fts3Expr *p; /* Used to iterate from pExpr to root */
Fts3Expr *pNear; /* Most senior NEAR ancestor (or pExpr) */
Fts3Expr *pRun; /* Closest non-deferred ancestor of pNear */
int bMatch;
/* Check if this phrase descends from an OR expression node. If not,
@@ -5977,25 +6047,30 @@ int sqlite3Fts3EvalPhrasePoslist(
if( p->bEof ) bTreeEof = 1;
}
if( bOr==0 ) return SQLITE_OK;
pRun = pNear;
while( pRun->bDeferred ){
assert( pRun->pParent );
pRun = pRun->pParent;
}
/* This is the descendent of an OR node. In this case we cannot use
** an incremental phrase. Load the entire doclist for the phrase
** into memory in this case. */
if( pPhrase->bIncr ){
int bEofSave = pNear->bEof;
fts3EvalRestart(pCsr, pNear, &rc);
while( rc==SQLITE_OK && !pNear->bEof ){
fts3EvalNextRow(pCsr, pNear, &rc);
if( bEofSave==0 && pNear->iDocid==iDocid ) break;
int bEofSave = pRun->bEof;
fts3EvalRestart(pCsr, pRun, &rc);
while( rc==SQLITE_OK && !pRun->bEof ){
fts3EvalNextRow(pCsr, pRun, &rc);
if( bEofSave==0 && pRun->iDocid==iDocid ) break;
}
assert( rc!=SQLITE_OK || pPhrase->bIncr==0 );
if( rc==SQLITE_OK && pNear->bEof!=bEofSave ){
if( rc==SQLITE_OK && pRun->bEof!=bEofSave ){
rc = FTS_CORRUPT_VTAB;
}
}
if( bTreeEof ){
while( rc==SQLITE_OK && !pNear->bEof ){
fts3EvalNextRow(pCsr, pNear, &rc);
while( rc==SQLITE_OK && !pRun->bEof ){
fts3EvalNextRow(pCsr, pRun, &rc);
}
}
if( rc!=SQLITE_OK ) return rc;
+6 -1
View File
@@ -265,6 +265,7 @@ struct Fts3Table {
int nPgsz; /* Page size for host database */
char *zSegmentsTbl; /* Name of %_segments table */
sqlite3_blob *pSegments; /* Blob handle open on %_segments table */
int iSavepoint;
/*
** The following array of hash tables is used to buffer pending index
@@ -558,7 +559,7 @@ struct Fts3MultiSegReader {
int nAdvance; /* How many seg-readers to advance */
Fts3SegFilter *pFilter; /* Pointer to filter object */
char *aBuffer; /* Buffer to merge doclists in */
int nBuffer; /* Allocated size of aBuffer[] in bytes */
i64 nBuffer; /* Allocated size of aBuffer[] in bytes */
int iColFilter; /* If >=0, filter for this column */
int bRestart;
@@ -650,5 +651,9 @@ int sqlite3FtsUnicodeIsalnum(int);
int sqlite3FtsUnicodeIsdiacritic(int);
#endif
int sqlite3Fts3ExprIterate(Fts3Expr*, int (*x)(Fts3Expr*,int,void*), void*);
int sqlite3Fts3IntegrityCheck(Fts3Table *p, int *pbOk);
#endif /* !SQLITE_CORE || SQLITE_ENABLE_FTS3 */
#endif /* _FTSINT_H */
+2 -1
View File
@@ -545,7 +545,8 @@ int sqlite3Fts3InitAux(sqlite3 *db){
0, /* xSavepoint */
0, /* xRelease */
0, /* xRollbackTo */
0 /* xShadowName */
0, /* xShadowName */
0 /* xIntegrity */
};
int rc; /* Return code */
+1 -1
View File
@@ -621,7 +621,7 @@ static int porterNext(
if( n>c->nAllocated ){
char *pNew;
c->nAllocated = n+20;
pNew = sqlite3_realloc(c->zToken, c->nAllocated);
pNew = sqlite3_realloc64(c->zToken, c->nAllocated);
if( !pNew ) return SQLITE_NOMEM;
c->zToken = pNew;
}
+28 -23
View File
@@ -41,7 +41,7 @@ typedef sqlite3_int64 i64;
/*
** Used as an fts3ExprIterate() context when loading phrase doclists to
** Used as an sqlite3Fts3ExprIterate() context when loading phrase doclists to
** Fts3Expr.aDoclist[]/nDoclist.
*/
typedef struct LoadDoclistCtx LoadDoclistCtx;
@@ -85,7 +85,7 @@ struct SnippetFragment {
};
/*
** This type is used as an fts3ExprIterate() context object while
** This type is used as an sqlite3Fts3ExprIterate() context object while
** accumulating the data returned by the matchinfo() function.
*/
typedef struct MatchInfo MatchInfo;
@@ -244,7 +244,7 @@ static void fts3GetDeltaPosition(char **pp, i64 *piPos){
}
/*
** Helper function for fts3ExprIterate() (see below).
** Helper function for sqlite3Fts3ExprIterate() (see below).
*/
static int fts3ExprIterate2(
Fts3Expr *pExpr, /* Expression to iterate phrases of */
@@ -278,7 +278,7 @@ static int fts3ExprIterate2(
** Otherwise, SQLITE_OK is returned after a callback has been made for
** all eligible phrase nodes.
*/
static int fts3ExprIterate(
int sqlite3Fts3ExprIterate(
Fts3Expr *pExpr, /* Expression to iterate phrases of */
int (*x)(Fts3Expr*,int,void*), /* Callback function to invoke for phrases */
void *pCtx /* Second argument to pass to callback */
@@ -287,10 +287,9 @@ static int fts3ExprIterate(
return fts3ExprIterate2(pExpr, &iPhrase, x, pCtx);
}
/*
** This is an fts3ExprIterate() callback used while loading the doclists
** for each phrase into Fts3Expr.aDoclist[]/nDoclist. See also
** This is an sqlite3Fts3ExprIterate() callback used while loading the
** doclists for each phrase into Fts3Expr.aDoclist[]/nDoclist. See also
** fts3ExprLoadDoclists().
*/
static int fts3ExprLoadDoclistsCb(Fts3Expr *pExpr, int iPhrase, void *ctx){
@@ -322,9 +321,9 @@ static int fts3ExprLoadDoclists(
int *pnToken /* OUT: Number of tokens in query */
){
int rc; /* Return Code */
LoadDoclistCtx sCtx = {0,0,0}; /* Context for fts3ExprIterate() */
LoadDoclistCtx sCtx = {0,0,0}; /* Context for sqlite3Fts3ExprIterate() */
sCtx.pCsr = pCsr;
rc = fts3ExprIterate(pCsr->pExpr, fts3ExprLoadDoclistsCb, (void *)&sCtx);
rc = sqlite3Fts3ExprIterate(pCsr->pExpr,fts3ExprLoadDoclistsCb,(void*)&sCtx);
if( pnPhrase ) *pnPhrase = sCtx.nPhrase;
if( pnToken ) *pnToken = sCtx.nToken;
return rc;
@@ -337,7 +336,7 @@ static int fts3ExprPhraseCountCb(Fts3Expr *pExpr, int iPhrase, void *ctx){
}
static int fts3ExprPhraseCount(Fts3Expr *pExpr){
int nPhrase = 0;
(void)fts3ExprIterate(pExpr, fts3ExprPhraseCountCb, (void *)&nPhrase);
(void)sqlite3Fts3ExprIterate(pExpr, fts3ExprPhraseCountCb, (void *)&nPhrase);
return nPhrase;
}
@@ -399,6 +398,7 @@ static int fts3SnippetNextCandidate(SnippetIter *pIter){
return 1;
}
assert( pIter->nSnippet>=0 );
pIter->iCurrent = iStart = iEnd - pIter->nSnippet + 1;
for(i=0; i<pIter->nPhrase; i++){
SnippetPhrase *pPhrase = &pIter->aPhrase[i];
@@ -447,7 +447,7 @@ static void fts3SnippetDetails(
}
mCover |= mPhrase;
for(j=0; j<pPhrase->nToken; j++){
for(j=0; j<pPhrase->nToken && j<pIter->nSnippet; j++){
mHighlight |= (mPos>>j);
}
@@ -465,8 +465,9 @@ static void fts3SnippetDetails(
}
/*
** This function is an fts3ExprIterate() callback used by fts3BestSnippet().
** Each invocation populates an element of the SnippetIter.aPhrase[] array.
** This function is an sqlite3Fts3ExprIterate() callback used by
** fts3BestSnippet(). Each invocation populates an element of the
** SnippetIter.aPhrase[] array.
*/
static int fts3SnippetFindPositions(Fts3Expr *pExpr, int iPhrase, void *ctx){
SnippetIter *p = (SnippetIter *)ctx;
@@ -556,7 +557,9 @@ static int fts3BestSnippet(
sIter.nSnippet = nSnippet;
sIter.nPhrase = nList;
sIter.iCurrent = -1;
rc = fts3ExprIterate(pCsr->pExpr, fts3SnippetFindPositions, (void*)&sIter);
rc = sqlite3Fts3ExprIterate(
pCsr->pExpr, fts3SnippetFindPositions, (void*)&sIter
);
if( rc==SQLITE_OK ){
/* Set the *pmSeen output variable. */
@@ -917,10 +920,10 @@ static int fts3ExprLHitGather(
}
/*
** fts3ExprIterate() callback used to collect the "global" matchinfo stats
** for a single query.
** sqlite3Fts3ExprIterate() callback used to collect the "global" matchinfo
** stats for a single query.
**
** fts3ExprIterate() callback to load the 'global' elements of a
** sqlite3Fts3ExprIterate() callback to load the 'global' elements of a
** FTS3_MATCHINFO_HITS matchinfo array. The global stats are those elements
** of the matchinfo array that are constant for all rows returned by the
** current query.
@@ -955,7 +958,7 @@ static int fts3ExprGlobalHitsCb(
}
/*
** fts3ExprIterate() callback used to collect the "local" part of the
** sqlite3Fts3ExprIterate() callback used to collect the "local" part of the
** FTS3_MATCHINFO_HITS array. The local stats are those elements of the
** array that are different for each row returned by the query.
*/
@@ -1151,7 +1154,7 @@ static int fts3MatchinfoLcs(Fts3Cursor *pCsr, MatchInfo *pInfo){
**/
aIter = sqlite3Fts3MallocZero(sizeof(LcsIterator) * pCsr->nPhrase);
if( !aIter ) return SQLITE_NOMEM;
(void)fts3ExprIterate(pCsr->pExpr, fts3MatchinfoLcsCb, (void*)aIter);
(void)sqlite3Fts3ExprIterate(pCsr->pExpr, fts3MatchinfoLcsCb, (void*)aIter);
for(i=0; i<pInfo->nPhrase; i++){
LcsIterator *pIter = &aIter[i];
@@ -1328,11 +1331,11 @@ static int fts3MatchinfoValues(
rc = fts3MatchinfoSelectDoctotal(pTab, &pSelect, &pInfo->nDoc,0,0);
if( rc!=SQLITE_OK ) break;
}
rc = fts3ExprIterate(pExpr, fts3ExprGlobalHitsCb,(void*)pInfo);
rc = sqlite3Fts3ExprIterate(pExpr, fts3ExprGlobalHitsCb,(void*)pInfo);
sqlite3Fts3EvalTestDeferred(pCsr, &rc);
if( rc!=SQLITE_OK ) break;
}
(void)fts3ExprIterate(pExpr, fts3ExprLocalHitsCb,(void*)pInfo);
(void)sqlite3Fts3ExprIterate(pExpr, fts3ExprLocalHitsCb,(void*)pInfo);
break;
}
}
@@ -1555,7 +1558,7 @@ struct TermOffsetCtx {
};
/*
** This function is an fts3ExprIterate() callback used by sqlite3Fts3Offsets().
** This function is an sqlite3Fts3ExprIterate() callback used by sqlite3Fts3Offsets().
*/
static int fts3ExprTermOffsetInit(Fts3Expr *pExpr, int iPhrase, void *ctx){
TermOffsetCtx *p = (TermOffsetCtx *)ctx;
@@ -1637,7 +1640,9 @@ void sqlite3Fts3Offsets(
*/
sCtx.iCol = iCol;
sCtx.iTerm = 0;
rc = fts3ExprIterate(pCsr->pExpr, fts3ExprTermOffsetInit, (void*)&sCtx);
rc = sqlite3Fts3ExprIterate(
pCsr->pExpr, fts3ExprTermOffsetInit, (void*)&sCtx
);
if( rc!=SQLITE_OK ) goto offsets_out;
/* Retreive the text stored in column iCol. If an SQL NULL is stored
+14 -5
View File
@@ -78,6 +78,8 @@ static int fts3termConnectMethod(
iIndex = atoi(argv[4]);
argc--;
}
*ppVtab = 0;
/* The user should specify a single argument - the name of an fts3 table. */
if( argc!=4 ){
@@ -95,12 +97,17 @@ static int fts3termConnectMethod(
rc = sqlite3_declare_vtab(db, FTS3_TERMS_SCHEMA);
if( rc!=SQLITE_OK ) return rc;
nByte = sizeof(Fts3termTable) + sizeof(Fts3Table) + nDb + nFts3 + 2;
p = (Fts3termTable *)sqlite3_malloc64(nByte);
nByte = sizeof(Fts3termTable);
p = (Fts3termTable *)sqlite3Fts3MallocZero(nByte);
if( !p ) return SQLITE_NOMEM;
memset(p, 0, (size_t)nByte);
p->pFts3Tab = (Fts3Table *)&p[1];
p->pFts3Tab = (Fts3Table*)sqlite3Fts3MallocZero(
sizeof(Fts3Table) + nDb + nFts3 + 2
);
if( p->pFts3Tab==0 ){
sqlite3_free(p);
return SQLITE_NOMEM;
}
p->pFts3Tab->zDb = (char *)&p->pFts3Tab[1];
p->pFts3Tab->zName = &p->pFts3Tab->zDb[nDb+1];
p->pFts3Tab->db = db;
@@ -130,6 +137,7 @@ static int fts3termDisconnectMethod(sqlite3_vtab *pVtab){
sqlite3_finalize(pFts3->aStmt[i]);
}
sqlite3_free(pFts3->zSegmentsTbl);
sqlite3_free(pFts3);
sqlite3_free(p);
return SQLITE_OK;
}
@@ -362,7 +370,8 @@ int sqlite3Fts3InitTerm(sqlite3 *db){
0, /* xSavepoint */
0, /* xRelease */
0, /* xRollbackTo */
0 /* xShadowName */
0, /* xShadowName */
0 /* xIntegrity */
};
int rc; /* Return code */
+13 -16
View File
@@ -18,14 +18,7 @@
** that the sqlite3_tokenizer_module.xLanguage() method is invoked correctly.
*/
#if defined(INCLUDE_SQLITE_TCL_H)
# include "sqlite_tcl.h"
#else
# include "tcl.h"
# ifndef SQLITE_TCLAPI
# define SQLITE_TCLAPI
# endif
#endif
#include "tclsqlite.h"
#include <string.h>
#include <assert.h>
@@ -167,7 +160,8 @@ static int SQLITE_TCLAPI fts3_near_match_cmd(
Tcl_Obj *pPhrasecount = 0;
Tcl_Obj **apExprToken;
int nExprToken;
Tcl_Size nExprToken;
Tcl_Size nn;
UNUSED_PARAMETER(clientData);
@@ -201,23 +195,25 @@ static int SQLITE_TCLAPI fts3_near_match_cmd(
}
}
rc = Tcl_ListObjGetElements(interp, objv[1], &doc.nToken, &apDocToken);
rc = Tcl_ListObjGetElements(interp, objv[1], &nn, &apDocToken);
doc.nToken = (int)nn;
if( rc!=TCL_OK ) goto near_match_out;
doc.aToken = (NearToken *)ckalloc(doc.nToken*sizeof(NearToken));
for(ii=0; ii<doc.nToken; ii++){
doc.aToken[ii].z = Tcl_GetStringFromObj(apDocToken[ii], &doc.aToken[ii].n);
doc.aToken[ii].z = Tcl_GetStringFromObj(apDocToken[ii], &nn);
doc.aToken[ii].n = (int)nn;
}
rc = Tcl_ListObjGetElements(interp, objv[2], &nExprToken, &apExprToken);
if( rc!=TCL_OK ) goto near_match_out;
nPhrase = (nExprToken + 1) / 2;
nPhrase = (int)(nExprToken + 1) / 2;
aPhrase = (NearPhrase *)ckalloc(nPhrase * sizeof(NearPhrase));
memset(aPhrase, 0, nPhrase * sizeof(NearPhrase));
for(ii=0; ii<nPhrase; ii++){
Tcl_Obj *pPhrase = apExprToken[ii*2];
Tcl_Obj **apToken;
int nToken;
Tcl_Size nToken;
int jj;
rc = Tcl_ListObjGetElements(interp, pPhrase, &nToken, &apToken);
@@ -227,11 +223,12 @@ static int SQLITE_TCLAPI fts3_near_match_cmd(
rc = TCL_ERROR;
goto near_match_out;
}
for(jj=0; jj<nToken; jj++){
for(jj=0; jj<(int)nToken; jj++){
NearToken *pT = &aPhrase[ii].aToken[jj];
pT->z = Tcl_GetStringFromObj(apToken[jj], &pT->n);
pT->z = Tcl_GetStringFromObj(apToken[jj], &nn);
pT->n = (int)nn;
}
aPhrase[ii].nToken = nToken;
aPhrase[ii].nToken = (int)nToken;
}
for(ii=1; ii<nPhrase; ii++){
Tcl_Obj *pNear = apExprToken[2*ii-1];
+2 -1
View File
@@ -445,7 +445,8 @@ int sqlite3Fts3InitTok(sqlite3 *db, Fts3Hash *pHash, void(*xDestroy)(void*)){
0, /* xSavepoint */
0, /* xRelease */
0, /* xRollbackTo */
0 /* xShadowName */
0, /* xShadowName */
0 /* xIntegrity */
};
int rc; /* Return code */
+1 -5
View File
@@ -226,11 +226,7 @@ int sqlite3Fts3InitTokenizer(
#ifdef SQLITE_TEST
#if defined(INCLUDE_SQLITE_TCL_H)
# include "sqlite_tcl.h"
#else
# include "tcl.h"
#endif
#include "tclsqlite.h"
#include <string.h>
/*
+1 -1
View File
@@ -185,7 +185,7 @@ static int simpleNext(
if( n>c->nTokenAllocated ){
char *pNew;
c->nTokenAllocated = n+20;
pNew = sqlite3_realloc(c->pToken, c->nTokenAllocated);
pNew = sqlite3_realloc64(c->pToken, c->nTokenAllocated);
if( !pNew ) return SQLITE_NOMEM;
c->pToken = pNew;
}
+65 -43
View File
@@ -649,7 +649,7 @@ static int fts3PendingListAppendVarint(
/* Allocate or grow the PendingList as required. */
if( !p ){
p = sqlite3_malloc(sizeof(*p) + 100);
p = sqlite3_malloc64(sizeof(*p) + 100);
if( !p ){
return SQLITE_NOMEM;
}
@@ -658,14 +658,14 @@ static int fts3PendingListAppendVarint(
p->nData = 0;
}
else if( p->nData+FTS3_VARINT_MAX+1>p->nSpace ){
int nNew = p->nSpace * 2;
p = sqlite3_realloc(p, sizeof(*p) + nNew);
i64 nNew = p->nSpace * 2;
p = sqlite3_realloc64(p, sizeof(*p) + nNew);
if( !p ){
sqlite3_free(*pp);
*pp = 0;
return SQLITE_NOMEM;
}
p->nSpace = nNew;
p->nSpace = (int)nNew;
p->aData = (char *)&p[1];
}
@@ -1222,7 +1222,7 @@ int sqlite3Fts3ReadBlock(
int nByte = sqlite3_blob_bytes(p->pSegments);
*pnBlob = nByte;
if( paBlob ){
char *aByte = sqlite3_malloc(nByte + FTS3_NODE_PADDING);
char *aByte = sqlite3_malloc64((i64)nByte + FTS3_NODE_PADDING);
if( !aByte ){
rc = SQLITE_NOMEM;
}else{
@@ -1339,7 +1339,7 @@ static int fts3SegReaderNext(
int nTerm = fts3HashKeysize(pElem);
if( (nTerm+1)>pReader->nTermAlloc ){
sqlite3_free(pReader->zTerm);
pReader->zTerm = (char*)sqlite3_malloc((nTerm+1)*2);
pReader->zTerm = (char*)sqlite3_malloc64(((i64)nTerm+1)*2);
if( !pReader->zTerm ) return SQLITE_NOMEM;
pReader->nTermAlloc = (nTerm+1)*2;
}
@@ -1347,7 +1347,7 @@ static int fts3SegReaderNext(
pReader->zTerm[nTerm] = '\0';
pReader->nTerm = nTerm;
aCopy = (char*)sqlite3_malloc(nCopy);
aCopy = (char*)sqlite3_malloc64(nCopy);
if( !aCopy ) return SQLITE_NOMEM;
memcpy(aCopy, pList->aData, nCopy);
pReader->nNode = pReader->nDoclist = nCopy;
@@ -1634,7 +1634,7 @@ int sqlite3Fts3SegReaderNew(
nExtra = nRoot + FTS3_NODE_PADDING;
}
pReader = (Fts3SegReader *)sqlite3_malloc(sizeof(Fts3SegReader) + nExtra);
pReader = (Fts3SegReader *)sqlite3_malloc64(sizeof(Fts3SegReader) + nExtra);
if( !pReader ){
return SQLITE_NOMEM;
}
@@ -1726,7 +1726,7 @@ int sqlite3Fts3SegReaderPending(
if( nElem==nAlloc ){
Fts3HashElem **aElem2;
nAlloc += 16;
aElem2 = (Fts3HashElem **)sqlite3_realloc(
aElem2 = (Fts3HashElem **)sqlite3_realloc64(
aElem, nAlloc*sizeof(Fts3HashElem *)
);
if( !aElem2 ){
@@ -2060,7 +2060,7 @@ static int fts3NodeAddTerm(
** this is not expected to be a serious problem.
*/
assert( pTree->aData==(char *)&pTree[1] );
pTree->aData = (char *)sqlite3_malloc(nReq);
pTree->aData = (char *)sqlite3_malloc64(nReq);
if( !pTree->aData ){
return SQLITE_NOMEM;
}
@@ -2078,7 +2078,7 @@ static int fts3NodeAddTerm(
if( isCopyTerm ){
if( pTree->nMalloc<nTerm ){
char *zNew = sqlite3_realloc(pTree->zMalloc, nTerm*2);
char *zNew = sqlite3_realloc64(pTree->zMalloc, (i64)nTerm*2);
if( !zNew ){
return SQLITE_NOMEM;
}
@@ -2104,7 +2104,7 @@ static int fts3NodeAddTerm(
** now. Instead, the term is inserted into the parent of pTree. If pTree
** has no parent, one is created here.
*/
pNew = (SegmentNode *)sqlite3_malloc(sizeof(SegmentNode) + p->nNodeSize);
pNew = (SegmentNode *)sqlite3_malloc64(sizeof(SegmentNode) + p->nNodeSize);
if( !pNew ){
return SQLITE_NOMEM;
}
@@ -2242,7 +2242,7 @@ static int fts3SegWriterAdd(
){
int nPrefix; /* Size of term prefix in bytes */
int nSuffix; /* Size of term suffix in bytes */
int nReq; /* Number of bytes required on leaf page */
i64 nReq; /* Number of bytes required on leaf page */
int nData;
SegmentWriter *pWriter = *ppWriter;
@@ -2251,13 +2251,13 @@ static int fts3SegWriterAdd(
sqlite3_stmt *pStmt;
/* Allocate the SegmentWriter structure */
pWriter = (SegmentWriter *)sqlite3_malloc(sizeof(SegmentWriter));
pWriter = (SegmentWriter *)sqlite3_malloc64(sizeof(SegmentWriter));
if( !pWriter ) return SQLITE_NOMEM;
memset(pWriter, 0, sizeof(SegmentWriter));
*ppWriter = pWriter;
/* Allocate a buffer in which to accumulate data */
pWriter->aData = (char *)sqlite3_malloc(p->nNodeSize);
pWriter->aData = (char *)sqlite3_malloc64(p->nNodeSize);
if( !pWriter->aData ) return SQLITE_NOMEM;
pWriter->nSize = p->nNodeSize;
@@ -2332,7 +2332,7 @@ static int fts3SegWriterAdd(
** the buffer to make it large enough.
*/
if( nReq>pWriter->nSize ){
char *aNew = sqlite3_realloc(pWriter->aData, nReq);
char *aNew = sqlite3_realloc64(pWriter->aData, nReq);
if( !aNew ) return SQLITE_NOMEM;
pWriter->aData = aNew;
pWriter->nSize = nReq;
@@ -2357,7 +2357,7 @@ static int fts3SegWriterAdd(
*/
if( isCopyTerm ){
if( nTerm>pWriter->nMalloc ){
char *zNew = sqlite3_realloc(pWriter->zMalloc, nTerm*2);
char *zNew = sqlite3_realloc64(pWriter->zMalloc, (i64)nTerm*2);
if( !zNew ){
return SQLITE_NOMEM;
}
@@ -2665,18 +2665,20 @@ static void fts3ColumnFilter(
static int fts3MsrBufferData(
Fts3MultiSegReader *pMsr, /* Multi-segment-reader handle */
char *pList,
int nList
i64 nList
){
if( nList>pMsr->nBuffer ){
if( (nList+FTS3_NODE_PADDING)>pMsr->nBuffer ){
char *pNew;
pMsr->nBuffer = nList*2;
pNew = (char *)sqlite3_realloc(pMsr->aBuffer, pMsr->nBuffer);
int nNew = nList*2 + FTS3_NODE_PADDING;
pNew = (char *)sqlite3_realloc64(pMsr->aBuffer, nNew);
if( !pNew ) return SQLITE_NOMEM;
pMsr->aBuffer = pNew;
pMsr->nBuffer = nNew;
}
assert( nList>0 );
memcpy(pMsr->aBuffer, pList, nList);
memset(&pMsr->aBuffer[nList], 0, FTS3_NODE_PADDING);
return SQLITE_OK;
}
@@ -2726,7 +2728,7 @@ int sqlite3Fts3MsrIncrNext(
fts3SegReaderSort(pMsr->apSegment, nMerge, j, xCmp);
if( nList>0 && fts3SegReaderIsPending(apSegment[0]) ){
rc = fts3MsrBufferData(pMsr, pList, nList+1);
rc = fts3MsrBufferData(pMsr, pList, (i64)nList+1);
if( rc!=SQLITE_OK ) return rc;
assert( (pMsr->aBuffer[nList] & 0xFE)==0x00 );
pList = pMsr->aBuffer;
@@ -2863,11 +2865,11 @@ int sqlite3Fts3MsrIncrRestart(Fts3MultiSegReader *pCsr){
return SQLITE_OK;
}
static int fts3GrowSegReaderBuffer(Fts3MultiSegReader *pCsr, int nReq){
static int fts3GrowSegReaderBuffer(Fts3MultiSegReader *pCsr, i64 nReq){
if( nReq>pCsr->nBuffer ){
char *aNew;
pCsr->nBuffer = nReq*2;
aNew = sqlite3_realloc(pCsr->aBuffer, pCsr->nBuffer);
aNew = sqlite3_realloc64(pCsr->aBuffer, pCsr->nBuffer);
if( !aNew ){
return SQLITE_NOMEM;
}
@@ -2958,7 +2960,8 @@ int sqlite3Fts3SegReaderStep(
){
pCsr->nDoclist = apSegment[0]->nDoclist;
if( fts3SegReaderIsPending(apSegment[0]) ){
rc = fts3MsrBufferData(pCsr, apSegment[0]->aDoclist, pCsr->nDoclist);
rc = fts3MsrBufferData(pCsr, apSegment[0]->aDoclist,
(i64)pCsr->nDoclist);
pCsr->aDoclist = pCsr->aBuffer;
}else{
pCsr->aDoclist = apSegment[0]->aDoclist;
@@ -3011,7 +3014,8 @@ int sqlite3Fts3SegReaderStep(
nByte = sqlite3Fts3VarintLen(iDelta) + (isRequirePos?nList+1:0);
rc = fts3GrowSegReaderBuffer(pCsr, nByte+nDoclist+FTS3_NODE_PADDING);
rc = fts3GrowSegReaderBuffer(pCsr,
(i64)nByte+nDoclist+FTS3_NODE_PADDING);
if( rc ) return rc;
if( isFirst ){
@@ -3037,7 +3041,7 @@ int sqlite3Fts3SegReaderStep(
fts3SegReaderSort(apSegment, nMerge, j, xCmp);
}
if( nDoclist>0 ){
rc = fts3GrowSegReaderBuffer(pCsr, nDoclist+FTS3_NODE_PADDING);
rc = fts3GrowSegReaderBuffer(pCsr, (i64)nDoclist+FTS3_NODE_PADDING);
if( rc ) return rc;
memset(&pCsr->aBuffer[nDoclist], 0, FTS3_NODE_PADDING);
pCsr->aDoclist = pCsr->aBuffer;
@@ -3321,7 +3325,6 @@ int sqlite3Fts3PendingTermsFlush(Fts3Table *p){
rc = fts3SegmentMerge(p, p->iPrevLangid, i, FTS3_SEGCURSOR_PENDING);
if( rc==SQLITE_DONE ) rc = SQLITE_OK;
}
sqlite3Fts3PendingTermsClear(p);
/* Determine the auto-incr-merge setting if unknown. If enabled,
** estimate the number of leaf blocks of content to be written
@@ -3343,6 +3346,10 @@ int sqlite3Fts3PendingTermsFlush(Fts3Table *p){
rc = sqlite3_reset(pStmt);
}
}
if( rc==SQLITE_OK ){
sqlite3Fts3PendingTermsClear(p);
}
return rc;
}
@@ -3750,7 +3757,7 @@ struct NodeReader {
static void blobGrowBuffer(Blob *pBlob, int nMin, int *pRc){
if( *pRc==SQLITE_OK && nMin>pBlob->nAlloc ){
int nAlloc = nMin;
char *a = (char *)sqlite3_realloc(pBlob->a, nAlloc);
char *a = (char *)sqlite3_realloc64(pBlob->a, nAlloc);
if( a ){
pBlob->nAlloc = nAlloc;
pBlob->a = a;
@@ -3974,6 +3981,8 @@ static int fts3AppendToNode(
blobGrowBuffer(pPrev, nTerm, &rc);
if( rc!=SQLITE_OK ) return rc;
assert( pPrev!=0 );
assert( pPrev->a!=0 );
nPrefix = fts3PrefixCompress(pPrev->a, pPrev->n, zTerm, nTerm);
nSuffix = nTerm - nPrefix;
@@ -4030,9 +4039,13 @@ static int fts3IncrmergeAppend(
nSpace += sqlite3Fts3VarintLen(nDoclist) + nDoclist;
/* If the current block is not empty, and if adding this term/doclist
** to the current block would make it larger than Fts3Table.nNodeSize
** bytes, write this block out to the database. */
if( pLeaf->block.n>0 && (pLeaf->block.n + nSpace)>p->nNodeSize ){
** to the current block would make it larger than Fts3Table.nNodeSize bytes,
** and if there is still room for another leaf page, write this block out to
** the database. */
if( pLeaf->block.n>0
&& (pLeaf->block.n + nSpace)>p->nNodeSize
&& pLeaf->iBlock < (pWriter->iStart + pWriter->nLeafEst)
){
rc = fts3WriteSegment(p, pLeaf->iBlock, pLeaf->block.a, pLeaf->block.n);
pWriter->nWork++;
@@ -4343,6 +4356,7 @@ static int fts3IncrmergeLoad(
for(i=nHeight; i>=0 && rc==SQLITE_OK; i--){
NodeReader reader;
memset(&reader, 0, sizeof(reader));
pNode = &pWriter->aNodeWriter[i];
if( pNode->block.a){
@@ -4363,7 +4377,7 @@ static int fts3IncrmergeLoad(
rc = sqlite3Fts3ReadBlock(p, reader.iChild, &aBlock, &nBlock,0);
blobGrowBuffer(&pNode->block,
MAX(nBlock, p->nNodeSize)+FTS3_NODE_PADDING, &rc
);
);
if( rc==SQLITE_OK ){
memcpy(pNode->block.a, aBlock, nBlock);
pNode->block.n = nBlock;
@@ -4547,7 +4561,7 @@ static int fts3RepackSegdirLevel(
if( nIdx>=nAlloc ){
int *aNew;
nAlloc += 16;
aNew = sqlite3_realloc(aIdx, nAlloc*sizeof(int));
aNew = sqlite3_realloc64(aIdx, nAlloc*sizeof(int));
if( !aNew ){
rc = SQLITE_NOMEM;
break;
@@ -4921,7 +4935,7 @@ int sqlite3Fts3Incrmerge(Fts3Table *p, int nMerge, int nMin){
/* Allocate space for the cursor, filter and writer objects */
const int nAlloc = sizeof(*pCsr) + sizeof(*pFilter) + sizeof(*pWriter);
pWriter = (IncrmergeWriter *)sqlite3_malloc(nAlloc);
pWriter = (IncrmergeWriter *)sqlite3_malloc64(nAlloc);
if( !pWriter ) return SQLITE_NOMEM;
pFilter = (Fts3SegFilter *)&pWriter[1];
pCsr = (Fts3MultiSegReader *)&pFilter[1];
@@ -5213,7 +5227,7 @@ static u64 fts3ChecksumIndex(
int rc;
u64 cksum = 0;
assert( *pRc==SQLITE_OK );
if( *pRc ) return 0;
memset(&filter, 0, sizeof(filter));
memset(&csr, 0, sizeof(csr));
@@ -5280,7 +5294,7 @@ static u64 fts3ChecksumIndex(
** If an error occurs (e.g. an OOM or IO error), return an SQLite error
** code. The final value of *pbOk is undefined in this case.
*/
static int fts3IntegrityCheck(Fts3Table *p, int *pbOk){
int sqlite3Fts3IntegrityCheck(Fts3Table *p, int *pbOk){
int rc = SQLITE_OK; /* Return code */
u64 cksum1 = 0; /* Checksum based on FTS index contents */
u64 cksum2 = 0; /* Checksum based on %_content contents */
@@ -5358,7 +5372,12 @@ static int fts3IntegrityCheck(Fts3Table *p, int *pbOk){
sqlite3_finalize(pStmt);
}
*pbOk = (cksum1==cksum2);
if( rc==SQLITE_CORRUPT_VTAB ){
rc = SQLITE_OK;
*pbOk = 0;
}else{
*pbOk = (rc==SQLITE_OK && cksum1==cksum2);
}
return rc;
}
@@ -5398,7 +5417,7 @@ static int fts3DoIntegrityCheck(
){
int rc;
int bOk = 0;
rc = fts3IntegrityCheck(p, &bOk);
rc = sqlite3Fts3IntegrityCheck(p, &bOk);
if( rc==SQLITE_OK && bOk==0 ) rc = FTS_CORRUPT_VTAB;
return rc;
}
@@ -5428,8 +5447,11 @@ static int fts3SpecialInsert(Fts3Table *p, sqlite3_value *pVal){
rc = fts3DoIncrmerge(p, &zVal[6]);
}else if( nVal>10 && 0==sqlite3_strnicmp(zVal, "automerge=", 10) ){
rc = fts3DoAutoincrmerge(p, &zVal[10]);
}else if( nVal==5 && 0==sqlite3_strnicmp(zVal, "flush", 5) ){
rc = sqlite3Fts3PendingTermsFlush(p);
}
#if defined(SQLITE_DEBUG) || defined(SQLITE_TEST)
}else{
else{
int v;
if( nVal>9 && 0==sqlite3_strnicmp(zVal, "nodesize=", 9) ){
v = atoi(&zVal[9]);
@@ -5447,8 +5469,8 @@ static int fts3SpecialInsert(Fts3Table *p, sqlite3_value *pVal){
if( v>=4 && v<=FTS3_MERGE_COUNT && (v&1)==0 ) p->nMergeCount = v;
rc = SQLITE_OK;
}
#endif
}
#endif
return rc;
}
@@ -5557,7 +5579,7 @@ int sqlite3Fts3DeferredTokenList(
return SQLITE_OK;
}
pRet = (char *)sqlite3_malloc(p->pList->nData);
pRet = (char *)sqlite3_malloc64(p->pList->nData);
if( !pRet ) return SQLITE_NOMEM;
nSkip = sqlite3Fts3GetVarint(p->pList->aData, &dummy);
@@ -5577,7 +5599,7 @@ int sqlite3Fts3DeferToken(
int iCol /* Column that token must appear in (or -1) */
){
Fts3DeferredToken *pDeferred;
pDeferred = sqlite3_malloc(sizeof(*pDeferred));
pDeferred = sqlite3_malloc64(sizeof(*pDeferred));
if( !pDeferred ){
return SQLITE_NOMEM;
}
+3
View File
@@ -628,6 +628,9 @@ proc print_categories {lMap} {
$caseP
$caseS
$caseZ
default:
return 1;
}
return 0;
}
+8 -3
View File
@@ -82,7 +82,7 @@ proc get_struct_docs {data names} {
set current_doc ""
}
set subject n/a
regexp {^ *([[:alpha:]]*)} $line -> subject
regexp {^ *([[:alnum:]_]*)} $line -> subject
if {[lsearch $names $subject]>=0} {
set current_header $subject
} else {
@@ -108,8 +108,11 @@ proc get_tokenizer_docs {data} {
append res "<dt><b>$line</b></dt><dd><p style=margin-top:0>\n"
continue
}
if {[regexp {FTS5_TOKENIZER} $line]} {
set line </dl><p>
}
if {[regexp {SYNONYM SUPPORT} $line]} {
set line "</dl><h3>Synonym Support</h3>"
set line "<h3>Synonym Support</h3>"
}
if {[string trim $line] == ""} {
append res "<p>\n"
@@ -223,10 +226,12 @@ proc main {data} {
Fts5ExtensionApi {
set struct [get_fts5_struct $data "^struct Fts5ExtensionApi" "^.;"]
set map [list]
set lKey [list]
foreach {k v} [get_struct_members $data] {
if {[string match x* $k]==0} continue
lappend map $k "<a href=#$k>$k</a>"
lappend lKey $k
}
foreach k [lsort -decr $lKey] { lappend map $k "<a href=#$k>$k</a>" }
output [string map $map $struct]
}
+184 -19
View File
@@ -55,8 +55,8 @@ struct Fts5PhraseIter {
** EXTENSION API FUNCTIONS
**
** xUserData(pFts):
** Return a copy of the context pointer the extension function was
** registered with.
** Return a copy of the pUserData pointer passed to the xCreateFunction()
** API when the extension function was registered.
**
** xColumnTotalSize(pFts, iCol, pnToken):
** If parameter iCol is less than zero, set output variable *pnToken
@@ -88,8 +88,11 @@ struct Fts5PhraseIter {
** created with the "columnsize=0" option.
**
** xColumnText:
** This function attempts to retrieve the text of column iCol of the
** current document. If successful, (*pz) is set to point to a buffer
** If parameter iCol is less than zero, or greater than or equal to the
** number of columns in the table, SQLITE_RANGE is returned.
**
** Otherwise, this function attempts to retrieve the text of column iCol of
** the current document. If successful, (*pz) is set to point to a buffer
** containing the text in utf-8 encoding, (*pn) is set to the size in bytes
** (not characters) of the buffer and SQLITE_OK is returned. Otherwise,
** if an error occurs, an SQLite error code is returned and the final values
@@ -99,8 +102,10 @@ struct Fts5PhraseIter {
** Returns the number of phrases in the current query expression.
**
** xPhraseSize:
** Returns the number of tokens in phrase iPhrase of the query. Phrases
** are numbered starting from zero.
** If parameter iCol is less than zero, or greater than or equal to the
** number of phrases in the current query, as returned by xPhraseCount,
** 0 is returned. Otherwise, this function returns the number of tokens in
** phrase iPhrase of the query. Phrases are numbered starting from zero.
**
** xInstCount:
** Set *pnInst to the total number of occurrences of all phrases within
@@ -116,12 +121,13 @@ struct Fts5PhraseIter {
** Query for the details of phrase match iIdx within the current row.
** Phrase matches are numbered starting from zero, so the iIdx argument
** should be greater than or equal to zero and smaller than the value
** output by xInstCount().
** output by xInstCount(). If iIdx is less than zero or greater than
** or equal to the value returned by xInstCount(), SQLITE_RANGE is returned.
**
** Usually, output parameter *piPhrase is set to the phrase number, *piCol
** Otherwise, output parameter *piPhrase is set to the phrase number, *piCol
** to the column in which it occurs and *piOff the token offset of the
** first token of the phrase. Returns SQLITE_OK if successful, or an error
** code (i.e. SQLITE_NOMEM) if an error occurs.
** first token of the phrase. SQLITE_OK is returned if successful, or an
** error code (i.e. SQLITE_NOMEM) if an error occurs.
**
** This API can be quite slow if used with an FTS5 table created with the
** "detail=none" or "detail=column" option.
@@ -147,6 +153,10 @@ struct Fts5PhraseIter {
** Invoking Api.xUserData() returns a copy of the pointer passed as
** the third argument to pUserData.
**
** If parameter iPhrase is less than zero, or greater than or equal to
** the number of phrases in the query, as returned by xPhraseCount(),
** this function returns SQLITE_RANGE.
**
** If the callback function returns any value other than SQLITE_OK, the
** query is abandoned and the xQueryPhrase function returns immediately.
** If the returned value is SQLITE_DONE, xQueryPhrase returns SQLITE_OK.
@@ -228,6 +238,10 @@ struct Fts5PhraseIter {
** (i.e. if it is a contentless table), then this API always iterates
** through an empty set (all calls to xPhraseFirst() set iCol to -1).
**
** In all cases, matches are visited in (column ASC, offset ASC) order.
** i.e. all those in column 0, sorted by offset, followed by those in
** column 1, etc.
**
** xPhraseNext()
** See xPhraseFirst above.
**
@@ -261,9 +275,65 @@ struct Fts5PhraseIter {
**
** xPhraseNextColumn()
** See xPhraseFirstColumn above.
**
** xQueryToken(pFts5, iPhrase, iToken, ppToken, pnToken)
** This is used to access token iToken of phrase iPhrase of the current
** query. Before returning, output parameter *ppToken is set to point
** to a buffer containing the requested token, and *pnToken to the
** size of this buffer in bytes.
**
** If iPhrase or iToken are less than zero, or if iPhrase is greater than
** or equal to the number of phrases in the query as reported by
** xPhraseCount(), or if iToken is equal to or greater than the number of
** tokens in the phrase, SQLITE_RANGE is returned and *ppToken and *pnToken
are both zeroed.
**
** The output text is not a copy of the query text that specified the
** token. It is the output of the tokenizer module. For tokendata=1
** tables, this includes any embedded 0x00 and trailing data.
**
** xInstToken(pFts5, iIdx, iToken, ppToken, pnToken)
** This is used to access token iToken of phrase hit iIdx within the
** current row. If iIdx is less than zero or greater than or equal to the
** value returned by xInstCount(), SQLITE_RANGE is returned. Otherwise,
** output variable (*ppToken) is set to point to a buffer containing the
** matching document token, and (*pnToken) to the size of that buffer in
** bytes. This API is not available if the specified token matches a
** prefix query term. In that case both output variables are always set
** to 0.
**
** The output text is not a copy of the document text that was tokenized.
** It is the output of the tokenizer module. For tokendata=1 tables, this
** includes any embedded 0x00 and trailing data.
**
** This API can be quite slow if used with an FTS5 table created with the
** "detail=none" or "detail=column" option.
**
** xColumnLocale(pFts5, iIdx, pzLocale, pnLocale)
** If parameter iCol is less than zero, or greater than or equal to the
** number of columns in the table, SQLITE_RANGE is returned.
**
** Otherwise, this function attempts to retrieve the locale associated
** with column iCol of the current row. Usually, there is no associated
** locale, and output parameters (*pzLocale) and (*pnLocale) are set
** to NULL and 0, respectively. However, if the fts5_locale() function
** was used to associate a locale with the value when it was inserted
** into the fts5 table, then (*pzLocale) is set to point to a nul-terminated
** buffer containing the name of the locale in utf-8 encoding. (*pnLocale)
** is set to the size in bytes of the buffer, not including the
** nul-terminator.
**
** If successful, SQLITE_OK is returned. Or, if an error occurs, an
** SQLite error code is returned. The final value of the output parameters
** is undefined in this case.
**
** xTokenize_v2:
** Tokenize text using the tokenizer belonging to the FTS5 table. This
** API is the same as the xTokenize() API, except that it allows a tokenizer
** locale to be specified.
*/
struct Fts5ExtensionApi {
int iVersion; /* Currently always set to 3 */
int iVersion; /* Currently always set to 4 */
void *(*xUserData)(Fts5Context*);
@@ -298,6 +368,22 @@ struct Fts5ExtensionApi {
int (*xPhraseFirstColumn)(Fts5Context*, int iPhrase, Fts5PhraseIter*, int*);
void (*xPhraseNextColumn)(Fts5Context*, Fts5PhraseIter*, int *piCol);
/* Below this point are iVersion>=3 only */
int (*xQueryToken)(Fts5Context*,
int iPhrase, int iToken,
const char **ppToken, int *pnToken
);
int (*xInstToken)(Fts5Context*, int iIdx, int iToken, const char**, int*);
/* Below this point are iVersion>=4 only */
int (*xColumnLocale)(Fts5Context*, int iCol, const char **pz, int *pn);
int (*xTokenize_v2)(Fts5Context*,
const char *pText, int nText, /* Text to tokenize */
const char *pLocale, int nLocale, /* Locale to pass to tokenizer */
void *pCtx, /* Context passed to xToken() */
int (*xToken)(void*, int, const char*, int, int, int) /* Callback */
);
};
/*
@@ -310,6 +396,7 @@ struct Fts5ExtensionApi {
** Applications may also register custom tokenizer types. A tokenizer
** is registered by providing fts5 with a populated instance of the
** following structure. All structure methods must be defined, setting
**
** any member of the fts5_tokenizer struct to NULL leads to undefined
** behaviour. The structure methods are expected to function as follows:
**
@@ -318,7 +405,7 @@ struct Fts5ExtensionApi {
** A tokenizer instance is required to actually tokenize text.
**
** The first argument passed to this function is a copy of the (void*)
** pointer provided by the application when the fts5_tokenizer object
** pointer provided by the application when the fts5_tokenizer_v2 object
** was registered with FTS5 (the third argument to xCreateTokenizer()).
** The second and third arguments are an array of nul-terminated strings
** containing the tokenizer arguments, if any, specified following the
@@ -342,7 +429,7 @@ struct Fts5ExtensionApi {
** argument passed to this function is a pointer to an Fts5Tokenizer object
** returned by an earlier call to xCreate().
**
** The second argument indicates the reason that FTS5 is requesting
** The third argument indicates the reason that FTS5 is requesting
** tokenization of the supplied text. This is always one of the following
** four values:
**
@@ -366,6 +453,13 @@ struct Fts5ExtensionApi {
** on a columnsize=0 database.
** </ul>
**
** The sixth and seventh arguments passed to xTokenize() - pLocale and
** nLocale - are a pointer to a buffer containing the locale to use for
** tokenization (e.g. "en_US") and its size in bytes, respectively. The
** pLocale buffer is not nul-terminated. pLocale may be passed NULL (in
** which case nLocale is always 0) to indicate that the tokenizer should
** use its default locale.
**
** For each token in the input string, the supplied callback xToken() must
** be invoked. The first argument to it should be a copy of the pointer
** passed as the second argument to xTokenize(). The third and fourth
@@ -389,6 +483,30 @@ struct Fts5ExtensionApi {
** may abandon the tokenization and return any error code other than
** SQLITE_OK or SQLITE_DONE.
**
** If the tokenizer is registered using an fts5_tokenizer_v2 object,
** then the xTokenize() method has two additional arguments - pLocale
** and nLocale. These specify the locale that the tokenizer should use
** for the current request. If pLocale and nLocale are both 0, then the
** tokenizer should use its default locale. Otherwise, pLocale points to
** an nLocale byte buffer containing the name of the locale to use as utf-8
** text. pLocale is not nul-terminated.
**
** FTS5_TOKENIZER
**
** There is also an fts5_tokenizer object. This is an older, deprecated,
** version of fts5_tokenizer_v2. It is similar except that:
**
** <ul>
** <li> There is no "iVersion" field, and
** <li> The xTokenize() method does not take a locale argument.
** </ul>
**
** Legacy fts5_tokenizer tokenizers must be registered using the
** legacy xCreateTokenizer() function, instead of xCreateTokenizer_v2().
**
** Tokenizer implementations registered using either API may be retrieved
** using both xFindTokenizer() and xFindTokenizer_v2().
**
** SYNONYM SUPPORT
**
** Custom tokenizers may also support synonyms. Consider a case in which a
@@ -492,11 +610,38 @@ struct Fts5ExtensionApi {
** as separate queries of the FTS index are required for each synonym.
**
** When using methods (2) or (3), it is important that the tokenizer only
** provide synonyms when tokenizing document text (method (2)) or query
** text (method (3)), not both. Doing so will not cause any errors, but is
** provide synonyms when tokenizing document text (method (3)) or query
** text (method (2)), not both. Doing so will not cause any errors, but is
** inefficient.
*/
typedef struct Fts5Tokenizer Fts5Tokenizer;
typedef struct fts5_tokenizer_v2 fts5_tokenizer_v2;
struct fts5_tokenizer_v2 {
int iVersion; /* Currently always 2 */
int (*xCreate)(void*, const char **azArg, int nArg, Fts5Tokenizer **ppOut);
void (*xDelete)(Fts5Tokenizer*);
int (*xTokenize)(Fts5Tokenizer*,
void *pCtx,
int flags, /* Mask of FTS5_TOKENIZE_* flags */
const char *pText, int nText,
const char *pLocale, int nLocale,
int (*xToken)(
void *pCtx, /* Copy of 2nd argument to xTokenize() */
int tflags, /* Mask of FTS5_TOKEN_* flags */
const char *pToken, /* Pointer to buffer containing token */
int nToken, /* Size of token in bytes */
int iStart, /* Byte offset of token within input text */
int iEnd /* Byte offset of end of token within input text */
)
);
};
/*
** New code should use the fts5_tokenizer_v2 type to define tokenizer
** implementations. The following type is included for legacy applications
** that still use it.
*/
typedef struct fts5_tokenizer fts5_tokenizer;
struct fts5_tokenizer {
int (*xCreate)(void*, const char **azArg, int nArg, Fts5Tokenizer **ppOut);
@@ -516,6 +661,7 @@ struct fts5_tokenizer {
);
};
/* Flags that may be passed as the third argument to xTokenize() */
#define FTS5_TOKENIZE_QUERY 0x0001
#define FTS5_TOKENIZE_PREFIX 0x0002
@@ -535,13 +681,13 @@ struct fts5_tokenizer {
*/
typedef struct fts5_api fts5_api;
struct fts5_api {
int iVersion; /* Currently always set to 2 */
int iVersion; /* Currently always set to 3 */
/* Create a new tokenizer */
int (*xCreateTokenizer)(
fts5_api *pApi,
const char *zName,
void *pContext,
void *pUserData,
fts5_tokenizer *pTokenizer,
void (*xDestroy)(void*)
);
@@ -550,7 +696,7 @@ struct fts5_api {
int (*xFindTokenizer)(
fts5_api *pApi,
const char *zName,
void **ppContext,
void **ppUserData,
fts5_tokenizer *pTokenizer
);
@@ -558,10 +704,29 @@ struct fts5_api {
int (*xCreateFunction)(
fts5_api *pApi,
const char *zName,
void *pContext,
void *pUserData,
fts5_extension_function xFunction,
void (*xDestroy)(void*)
);
/* APIs below this point are only available if iVersion>=3 */
/* Create a new tokenizer */
int (*xCreateTokenizer_v2)(
fts5_api *pApi,
const char *zName,
void *pUserData,
fts5_tokenizer_v2 *pTokenizer,
void (*xDestroy)(void*)
);
/* Find an existing tokenizer */
int (*xFindTokenizer_v2)(
fts5_api *pApi,
const char *zName,
void **ppUserData,
fts5_tokenizer_v2 **ppTokenizer
);
};
/*
+101 -22
View File
@@ -59,6 +59,22 @@ typedef sqlite3_uint64 u64;
# define LARGEST_INT64 (0xffffffff|(((i64)0x7fffffff)<<32))
# define SMALLEST_INT64 (((i64)-1) - LARGEST_INT64)
/* The uptr type is an unsigned integer large enough to hold a pointer
*/
#if defined(HAVE_STDINT_H)
typedef uintptr_t uptr;
#elif SQLITE_PTRSIZE==4
typedef u32 uptr;
#else
typedef u64 uptr;
#endif
#ifdef SQLITE_4_BYTE_ALIGNED_MALLOC
# define EIGHT_BYTE_ALIGNMENT(X) ((((uptr)(X) - (uptr)0)&3)==0)
#else
# define EIGHT_BYTE_ALIGNMENT(X) ((((uptr)(X) - (uptr)0)&7)==0)
#endif
#endif
/* Truncate very long tokens to this many bytes. Hard limit is
@@ -142,6 +158,18 @@ struct Fts5Colset {
*/
typedef struct Fts5Config Fts5Config;
typedef struct Fts5TokenizerConfig Fts5TokenizerConfig;
struct Fts5TokenizerConfig {
Fts5Tokenizer *pTok;
fts5_tokenizer_v2 *pApi2;
fts5_tokenizer *pApi1;
const char **azArg;
int nArg;
int ePattern; /* FTS_PATTERN_XXX constant */
const char *pLocale; /* Current locale to use */
int nLocale; /* Size of pLocale in bytes */
};
/*
** An instance of the following structure encodes all information that can
@@ -154,6 +182,10 @@ typedef struct Fts5Config Fts5Config;
** attempt to merge together. A value of 1 sets the object to use the
** compile time default. Zero disables auto-merge altogether.
**
** bContentlessDelete:
** True if the contentless_delete option was present in the CREATE
** VIRTUAL TABLE statement.
**
** zContent:
**
** zContentRowid:
@@ -177,9 +209,12 @@ typedef struct Fts5Config Fts5Config;
**
** INSERT INTO tbl(tbl, rank) VALUES('prefix-index', $bPrefixIndex);
**
** bLocale:
** Set to true if locale=1 was specified when the table was created.
*/
struct Fts5Config {
sqlite3 *db; /* Database handle */
Fts5Global *pGlobal; /* Global fts5 object for handle db */
char *zDb; /* Database holding FTS index (e.g. "main") */
char *zName; /* Name of FTS index */
int nCol; /* Number of columns */
@@ -188,17 +223,20 @@ struct Fts5Config {
int nPrefix; /* Number of prefix indexes */
int *aPrefix; /* Sizes in bytes of nPrefix prefix indexes */
int eContent; /* An FTS5_CONTENT value */
int bContentlessDelete; /* "contentless_delete=" option (dflt==0) */
char *zContent; /* content table */
char *zContentRowid; /* "content_rowid=" option value */
int bColumnsize; /* "columnsize=" option value (dflt==1) */
int bTokendata; /* "tokendata=" option value (dflt==0) */
int bLocale; /* "locale=" option value (dflt==0) */
int eDetail; /* FTS5_DETAIL_XXX value */
char *zContentExprlist;
Fts5Tokenizer *pTok;
fts5_tokenizer *pTokApi;
Fts5TokenizerConfig t;
int bLock; /* True when table is preparing statement */
int ePattern; /* FTS_PATTERN_XXX constant */
/* Values loaded from the %_config table */
int iVersion; /* fts5 file format 'version' */
int iCookie; /* Incremented when %_config is modified */
int pgsz; /* Approximate page size used in %_data */
int nAutomerge; /* 'automerge' setting */
@@ -207,6 +245,8 @@ struct Fts5Config {
int nHashSize; /* Bytes of memory for in-memory hash */
char *zRank; /* Name of rank function */
char *zRankArgs; /* Arguments to rank function */
int bSecureDelete; /* 'secure-delete' */
int nDeleteMerge; /* 'deletemerge' */
/* If non-NULL, points to sqlite3_vtab.base.zErrmsg. Often NULL. */
char **pzErrmsg;
@@ -216,8 +256,11 @@ struct Fts5Config {
#endif
};
/* Current expected value of %_config table 'version' field */
#define FTS5_CURRENT_VERSION 4
/* Current expected value of %_config table 'version' field. And
** the expected version if the 'secure-delete' option has ever been
** set on the table. */
#define FTS5_CURRENT_VERSION 4
#define FTS5_CURRENT_VERSION_SECUREDELETE 5
#define FTS5_CONTENT_NORMAL 0
#define FTS5_CONTENT_NONE 1
@@ -256,6 +299,8 @@ int sqlite3Fts5ConfigSetValue(Fts5Config*, const char*, sqlite3_value*, int*);
int sqlite3Fts5ConfigParseRank(const char*, char**, char**);
void sqlite3Fts5ConfigErrmsg(Fts5Config *pConfig, const char *zFmt, ...);
/*
** End of interface to code in fts5_config.c.
**************************************************************************/
@@ -286,7 +331,7 @@ void sqlite3Fts5BufferAppendPrintf(int *, Fts5Buffer*, char *zFmt, ...);
char *sqlite3Fts5Mprintf(int *pRc, const char *zFmt, ...);
#define fts5BufferZero(x) sqlite3Fts5BufferZero(x)
#define fts5BufferAppendVarint(a,b,c) sqlite3Fts5BufferAppendVarint(a,b,c)
#define fts5BufferAppendVarint(a,b,c) sqlite3Fts5BufferAppendVarint(a,b,(i64)c)
#define fts5BufferFree(a) sqlite3Fts5BufferFree(a)
#define fts5BufferAppendBlob(a,b,c,d) sqlite3Fts5BufferAppendBlob(a,b,c,d)
#define fts5BufferSet(a,b,c,d) sqlite3Fts5BufferSet(a,b,c,d)
@@ -300,7 +345,7 @@ char *sqlite3Fts5Mprintf(int *pRc, const char *zFmt, ...);
void sqlite3Fts5Put32(u8*, int);
int sqlite3Fts5Get32(const u8*);
#define FTS5_POS2COLUMN(iPos) (int)(iPos >> 32)
#define FTS5_POS2COLUMN(iPos) (int)((iPos >> 32) & 0x7FFFFFFF)
#define FTS5_POS2OFFSET(iPos) (int)(iPos & 0x7FFFFFFF)
typedef struct Fts5PoslistReader Fts5PoslistReader;
@@ -373,16 +418,19 @@ struct Fts5IndexIter {
/*
** Values used as part of the flags argument passed to IndexQuery().
*/
#define FTS5INDEX_QUERY_PREFIX 0x0001 /* Prefix query */
#define FTS5INDEX_QUERY_DESC 0x0002 /* Docs in descending rowid order */
#define FTS5INDEX_QUERY_TEST_NOIDX 0x0004 /* Do not use prefix index */
#define FTS5INDEX_QUERY_SCAN 0x0008 /* Scan query (fts5vocab) */
#define FTS5INDEX_QUERY_PREFIX 0x0001 /* Prefix query */
#define FTS5INDEX_QUERY_DESC 0x0002 /* Docs in descending rowid order */
#define FTS5INDEX_QUERY_TEST_NOIDX 0x0004 /* Do not use prefix index */
#define FTS5INDEX_QUERY_SCAN 0x0008 /* Scan query (fts5vocab) */
/* The following are used internally by the fts5_index.c module. They are
** defined here only to make it easier to avoid clashes with the flags
** above. */
#define FTS5INDEX_QUERY_SKIPEMPTY 0x0010
#define FTS5INDEX_QUERY_NOOUTPUT 0x0020
#define FTS5INDEX_QUERY_SKIPEMPTY 0x0010
#define FTS5INDEX_QUERY_NOOUTPUT 0x0020
#define FTS5INDEX_QUERY_SKIPHASH 0x0040
#define FTS5INDEX_QUERY_NOTOKENDATA 0x0080
#define FTS5INDEX_QUERY_SCANONETERM 0x0100
/*
** Create/destroy an Fts5Index object.
@@ -451,6 +499,10 @@ void *sqlite3Fts5StructureRef(Fts5Index*);
void sqlite3Fts5StructureRelease(void*);
int sqlite3Fts5StructureTest(Fts5Index*, void*);
/*
** Used by xInstToken():
*/
int sqlite3Fts5IterToken(Fts5IndexIter*, i64, int, int, const char**, int*);
/*
** Insert or remove data to or from the index. Each time a document is
@@ -525,6 +577,16 @@ int sqlite3Fts5IndexReset(Fts5Index *p);
int sqlite3Fts5IndexLoadConfig(Fts5Index *p);
int sqlite3Fts5IndexGetOrigin(Fts5Index *p, i64 *piOrigin);
int sqlite3Fts5IndexContentlessDelete(Fts5Index *p, i64 iOrigin, i64 iRowid);
void sqlite3Fts5IndexIterClearTokendata(Fts5IndexIter*);
/* Used to populate hash tables for xInstToken in detail=none/column mode. */
int sqlite3Fts5IndexIterWriteTokendata(
Fts5IndexIter*, const char*, int, i64 iRowid, int iCol, int iOff
);
/*
** End of interface to code in fts5_index.c.
**************************************************************************/
@@ -537,7 +599,7 @@ int sqlite3Fts5GetVarintLen(u32 iVal);
u8 sqlite3Fts5GetVarint(const unsigned char*, u64*);
int sqlite3Fts5PutVarint(unsigned char *p, u64 v);
#define fts5GetVarint32(a,b) sqlite3Fts5GetVarint32(a,(u32*)&b)
#define fts5GetVarint32(a,b) sqlite3Fts5GetVarint32(a,(u32*)&(b))
#define fts5GetVarint sqlite3Fts5GetVarint
#define fts5FastGetVarint32(a, iOff, nVal) { \
@@ -568,18 +630,20 @@ struct Fts5Table {
Fts5Index *pIndex; /* Full-text index */
};
int sqlite3Fts5GetTokenizer(
Fts5Global*,
const char **azArg,
int nArg,
Fts5Config*,
char **pzErr
);
int sqlite3Fts5LoadTokenizer(Fts5Config *pConfig);
Fts5Table *sqlite3Fts5TableFromCsrid(Fts5Global*, i64);
int sqlite3Fts5FlushToDisk(Fts5Table*);
void sqlite3Fts5ClearLocale(Fts5Config *pConfig);
void sqlite3Fts5SetLocale(Fts5Config *pConfig, const char *pLoc, int nLoc);
int sqlite3Fts5IsLocaleValue(Fts5Config *pConfig, sqlite3_value *pVal);
int sqlite3Fts5DecodeLocaleValue(sqlite3_value *pVal,
const char **ppText, int *pnText, const char **ppLoc, int *pnLoc
);
/*
** End of interface to code in fts5.c.
**************************************************************************/
@@ -609,6 +673,11 @@ int sqlite3Fts5HashWrite(
*/
void sqlite3Fts5HashClear(Fts5Hash*);
/*
** Return true if the hash is empty, false otherwise.
*/
int sqlite3Fts5HashIsEmpty(Fts5Hash*);
int sqlite3Fts5HashQuery(
Fts5Hash*, /* Hash table to query */
int nPre,
@@ -625,11 +694,13 @@ void sqlite3Fts5HashScanNext(Fts5Hash*);
int sqlite3Fts5HashScanEof(Fts5Hash*);
void sqlite3Fts5HashScanEntry(Fts5Hash *,
const char **pzTerm, /* OUT: term (nul-terminated) */
int *pnTerm, /* OUT: Size of term in bytes */
const u8 **ppDoclist, /* OUT: pointer to doclist */
int *pnDoclist /* OUT: size of doclist in bytes */
);
/*
** End of interface to code in fts5_hash.c.
**************************************************************************/
@@ -652,7 +723,7 @@ int sqlite3Fts5StorageRename(Fts5Storage*, const char *zName);
int sqlite3Fts5DropAll(Fts5Config*);
int sqlite3Fts5CreateTable(Fts5Config*, const char*, const char*, int, char **);
int sqlite3Fts5StorageDelete(Fts5Storage *p, i64, sqlite3_value**);
int sqlite3Fts5StorageDelete(Fts5Storage *p, i64, sqlite3_value**, int);
int sqlite3Fts5StorageContentInsert(Fts5Storage *p, sqlite3_value**, i64*);
int sqlite3Fts5StorageIndexInsert(Fts5Storage *p, sqlite3_value**, i64);
@@ -678,6 +749,9 @@ int sqlite3Fts5StorageOptimize(Fts5Storage *p);
int sqlite3Fts5StorageMerge(Fts5Storage *p, int nMerge);
int sqlite3Fts5StorageReset(Fts5Storage *p);
void sqlite3Fts5StorageReleaseDeleteRow(Fts5Storage*);
int sqlite3Fts5StorageFindDeleteRow(Fts5Storage *p, i64 iDel);
/*
** End of interface to code in fts5_storage.c.
**************************************************************************/
@@ -750,6 +824,10 @@ int sqlite3Fts5ExprClonePhrase(Fts5Expr*, int, Fts5Expr**);
int sqlite3Fts5ExprPhraseCollist(Fts5Expr *, int, const u8 **, int *);
int sqlite3Fts5ExprQueryToken(Fts5Expr*, int, int, const char**, int*);
int sqlite3Fts5ExprInstToken(Fts5Expr*, i64, int, int, int, int, const char**, int*);
void sqlite3Fts5ExprClearTokens(Fts5Expr*);
/*******************************************
** The fts5_expr.c API above this point is used by the other hand-written
** C code in this module. The interfaces below this point are called by
@@ -826,6 +904,7 @@ int sqlite3Fts5TokenizerPattern(
int (*xCreate)(void*, const char**, int, Fts5Tokenizer**),
Fts5Tokenizer *pTok
);
int sqlite3Fts5TokenizerPreload(Fts5TokenizerConfig*);
/*
** End of interface to code in fts5_tokenizer.c.
**************************************************************************/
+129 -22
View File
@@ -110,15 +110,19 @@ static int fts5CInstIterInit(
*/
typedef struct HighlightContext HighlightContext;
struct HighlightContext {
CInstIter iter; /* Coalesced Instance Iterator */
int iPos; /* Current token offset in zIn[] */
/* Constant parameters to fts5HighlightCb() */
int iRangeStart; /* First token to include */
int iRangeEnd; /* If non-zero, last token to include */
const char *zOpen; /* Opening highlight */
const char *zClose; /* Closing highlight */
const char *zIn; /* Input text */
int nIn; /* Size of input text in bytes */
int iOff; /* Current offset within zIn[] */
/* Variables modified by fts5HighlightCb() */
CInstIter iter; /* Coalesced Instance Iterator */
int iPos; /* Current token offset in zIn[] */
int iOff; /* Have copied up to this offset in zIn[] */
int bOpen; /* True if highlight is open */
char *zOut; /* Output value */
};
@@ -151,8 +155,8 @@ static int fts5HighlightCb(
int tflags, /* Mask of FTS5_TOKEN_* flags */
const char *pToken, /* Buffer containing token */
int nToken, /* Size of token in bytes */
int iStartOff, /* Start offset of token */
int iEndOff /* End offset of token */
int iStartOff, /* Start byte offset of token */
int iEndOff /* End byte offset of token */
){
HighlightContext *p = (HighlightContext*)pContext;
int rc = SQLITE_OK;
@@ -163,40 +167,66 @@ static int fts5HighlightCb(
if( tflags & FTS5_TOKEN_COLOCATED ) return SQLITE_OK;
iPos = p->iPos++;
if( p->iRangeEnd>0 ){
if( p->iRangeEnd>=0 ){
if( iPos<p->iRangeStart || iPos>p->iRangeEnd ) return SQLITE_OK;
if( p->iRangeStart && iPos==p->iRangeStart ) p->iOff = iStartOff;
}
if( iPos==p->iter.iStart ){
/* If the parenthesis is open, and this token is not part of the current
** phrase, and the starting byte offset of this token is past the point
** that has currently been copied into the output buffer, close the
** parenthesis. */
if( p->bOpen
&& (iPos<=p->iter.iStart || p->iter.iStart<0)
&& iStartOff>p->iOff
){
fts5HighlightAppend(&rc, p, p->zClose, -1);
p->bOpen = 0;
}
/* If this is the start of a new phrase, and the highlight is not open:
**
** * copy text from the input up to the start of the phrase, and
** * open the highlight.
*/
if( iPos==p->iter.iStart && p->bOpen==0 ){
fts5HighlightAppend(&rc, p, &p->zIn[p->iOff], iStartOff - p->iOff);
fts5HighlightAppend(&rc, p, p->zOpen, -1);
p->iOff = iStartOff;
p->bOpen = 1;
}
if( iPos==p->iter.iEnd ){
if( p->iRangeEnd && p->iter.iStart<p->iRangeStart ){
if( p->bOpen==0 ){
assert( p->iRangeEnd>=0 );
fts5HighlightAppend(&rc, p, p->zOpen, -1);
p->bOpen = 1;
}
fts5HighlightAppend(&rc, p, &p->zIn[p->iOff], iEndOff - p->iOff);
fts5HighlightAppend(&rc, p, p->zClose, -1);
p->iOff = iEndOff;
if( rc==SQLITE_OK ){
rc = fts5CInstIterNext(&p->iter);
}
}
if( p->iRangeEnd>0 && iPos==p->iRangeEnd ){
if( iPos==p->iRangeEnd ){
if( p->bOpen ){
if( p->iter.iStart>=0 && iPos>=p->iter.iStart ){
fts5HighlightAppend(&rc, p, &p->zIn[p->iOff], iEndOff - p->iOff);
p->iOff = iEndOff;
}
fts5HighlightAppend(&rc, p, p->zClose, -1);
p->bOpen = 0;
}
fts5HighlightAppend(&rc, p, &p->zIn[p->iOff], iEndOff - p->iOff);
p->iOff = iEndOff;
if( iPos>=p->iter.iStart && iPos<p->iter.iEnd ){
fts5HighlightAppend(&rc, p, p->zClose, -1);
}
}
return rc;
}
/*
** Implementation of highlight() function.
*/
@@ -221,15 +251,28 @@ static void fts5HighlightFunction(
memset(&ctx, 0, sizeof(HighlightContext));
ctx.zOpen = (const char*)sqlite3_value_text(apVal[1]);
ctx.zClose = (const char*)sqlite3_value_text(apVal[2]);
ctx.iRangeEnd = -1;
rc = pApi->xColumnText(pFts, iCol, &ctx.zIn, &ctx.nIn);
if( ctx.zIn ){
if( rc==SQLITE_RANGE ){
sqlite3_result_text(pCtx, "", -1, SQLITE_STATIC);
rc = SQLITE_OK;
}else if( ctx.zIn ){
const char *pLoc = 0; /* Locale of column iCol */
int nLoc = 0; /* Size of pLoc in bytes */
if( rc==SQLITE_OK ){
rc = fts5CInstIterInit(pApi, pFts, iCol, &ctx.iter);
}
if( rc==SQLITE_OK ){
rc = pApi->xTokenize(pFts, ctx.zIn, ctx.nIn, (void*)&ctx,fts5HighlightCb);
rc = pApi->xColumnLocale(pFts, iCol, &pLoc, &nLoc);
}
if( rc==SQLITE_OK ){
rc = pApi->xTokenize_v2(
pFts, ctx.zIn, ctx.nIn, pLoc, nLoc, (void*)&ctx, fts5HighlightCb
);
}
if( ctx.bOpen ){
fts5HighlightAppend(&rc, &ctx, ctx.zClose, -1);
}
fts5HighlightAppend(&rc, &ctx, &ctx.zIn[ctx.iOff], ctx.nIn - ctx.iOff);
@@ -406,6 +449,7 @@ static void fts5SnippetFunction(
iCol = sqlite3_value_int(apVal[0]);
ctx.zOpen = fts5ValueToText(apVal[1]);
ctx.zClose = fts5ValueToText(apVal[2]);
ctx.iRangeEnd = -1;
zEllips = fts5ValueToText(apVal[3]);
nToken = sqlite3_value_int(apVal[4]);
@@ -422,6 +466,8 @@ static void fts5SnippetFunction(
memset(&sFinder, 0, sizeof(Fts5SFinder));
for(i=0; i<nCol; i++){
if( iCol<0 || iCol==i ){
const char *pLoc = 0; /* Locale of column iCol */
int nLoc = 0; /* Size of pLoc in bytes */
int nDoc;
int nDocsize;
int ii;
@@ -429,8 +475,10 @@ static void fts5SnippetFunction(
sFinder.nFirst = 0;
rc = pApi->xColumnText(pFts, i, &sFinder.zDoc, &nDoc);
if( rc!=SQLITE_OK ) break;
rc = pApi->xTokenize(pFts,
sFinder.zDoc, nDoc, (void*)&sFinder,fts5SentenceFinderCb
rc = pApi->xColumnLocale(pFts, i, &pLoc, &nLoc);
if( rc!=SQLITE_OK ) break;
rc = pApi->xTokenize_v2(pFts,
sFinder.zDoc, nDoc, pLoc, nLoc, (void*)&sFinder, fts5SentenceFinderCb
);
if( rc!=SQLITE_OK ) break;
rc = pApi->xColumnSize(pFts, i, &nDocsize);
@@ -488,6 +536,9 @@ static void fts5SnippetFunction(
rc = pApi->xColumnSize(pFts, iBestCol, &nColSize);
}
if( ctx.zIn ){
const char *pLoc = 0; /* Locale of column iBestCol */
int nLoc = 0; /* Bytes in pLoc */
if( rc==SQLITE_OK ){
rc = fts5CInstIterInit(pApi, pFts, iBestCol, &ctx.iter);
}
@@ -506,7 +557,15 @@ static void fts5SnippetFunction(
}
if( rc==SQLITE_OK ){
rc = pApi->xTokenize(pFts, ctx.zIn, ctx.nIn, (void*)&ctx,fts5HighlightCb);
rc = pApi->xColumnLocale(pFts, iBestCol, &pLoc, &nLoc);
}
if( rc==SQLITE_OK ){
rc = pApi->xTokenize_v2(
pFts, ctx.zIn, ctx.nIn, pLoc, nLoc, (void*)&ctx,fts5HighlightCb
);
}
if( ctx.bOpen ){
fts5HighlightAppend(&rc, &ctx, ctx.zClose, -1);
}
if( ctx.iRangeEnd>=(nColSize-1) ){
fts5HighlightAppend(&rc, &ctx, &ctx.zIn[ctx.iOff], ctx.nIn - ctx.iOff);
@@ -687,6 +746,53 @@ static void fts5Bm25Function(
}
}
/*
** Implementation of fts5_get_locale() function.
*/
static void fts5GetLocaleFunction(
const Fts5ExtensionApi *pApi, /* API offered by current FTS version */
Fts5Context *pFts, /* First arg to pass to pApi functions */
sqlite3_context *pCtx, /* Context for returning result/error */
int nVal, /* Number of values in apVal[] array */
sqlite3_value **apVal /* Array of trailing arguments */
){
int iCol = 0;
int eType = 0;
int rc = SQLITE_OK;
const char *zLocale = 0;
int nLocale = 0;
/* xColumnLocale() must be available */
assert( pApi->iVersion>=4 );
if( nVal!=1 ){
const char *z = "wrong number of arguments to function fts5_get_locale()";
sqlite3_result_error(pCtx, z, -1);
return;
}
eType = sqlite3_value_numeric_type(apVal[0]);
if( eType!=SQLITE_INTEGER ){
const char *z = "non-integer argument passed to function fts5_get_locale()";
sqlite3_result_error(pCtx, z, -1);
return;
}
iCol = sqlite3_value_int(apVal[0]);
if( iCol<0 || iCol>=pApi->xColumnCount(pFts) ){
sqlite3_result_error_code(pCtx, SQLITE_RANGE);
return;
}
rc = pApi->xColumnLocale(pFts, iCol, &zLocale, &nLocale);
if( rc!=SQLITE_OK ){
sqlite3_result_error_code(pCtx, rc);
return;
}
sqlite3_result_text(pCtx, zLocale, nLocale, SQLITE_TRANSIENT);
}
int sqlite3Fts5AuxInit(fts5_api *pApi){
struct Builtin {
const char *zFunc; /* Function name (nul-terminated) */
@@ -694,9 +800,10 @@ int sqlite3Fts5AuxInit(fts5_api *pApi){
fts5_extension_function xFunc;/* Callback function */
void (*xDestroy)(void*); /* Destructor function */
} aBuiltin [] = {
{ "snippet", 0, fts5SnippetFunction, 0 },
{ "highlight", 0, fts5HighlightFunction, 0 },
{ "bm25", 0, fts5Bm25Function, 0 },
{ "snippet", 0, fts5SnippetFunction, 0 },
{ "highlight", 0, fts5HighlightFunction, 0 },
{ "bm25", 0, fts5Bm25Function, 0 },
{ "fts5_get_locale", 0, fts5GetLocaleFunction, 0 },
};
int rc = SQLITE_OK; /* Return code */
int i; /* To iterate through builtin functions */
+3
View File
@@ -68,6 +68,7 @@ void sqlite3Fts5BufferAppendBlob(
){
if( nData ){
if( fts5BufferGrow(pRc, pBuf, nData) ) return;
assert( pBuf->p!=0 );
memcpy(&pBuf->p[pBuf->n], pData, nData);
pBuf->n += nData;
}
@@ -169,6 +170,7 @@ int sqlite3Fts5PoslistNext64(
i64 *piOff /* IN/OUT: Current offset */
){
int i = *pi;
assert( a!=0 || i==0 );
if( i>=n ){
/* EOF */
*piOff = -1;
@@ -176,6 +178,7 @@ int sqlite3Fts5PoslistNext64(
}else{
i64 iOff = *piOff;
u32 iVal;
assert( a!=0 );
fts5FastGetVarint32(a, i, iVal);
if( iVal<=1 ){
if( iVal==0 ){
+155 -42
View File
@@ -22,6 +22,8 @@
#define FTS5_DEFAULT_CRISISMERGE 16
#define FTS5_DEFAULT_HASHSIZE (1024*1024)
#define FTS5_DEFAULT_DELETE_AUTOMERGE 10 /* default 10% */
/* Maximum allowed page size */
#define FTS5_MAX_PAGE_SIZE (64*1024)
@@ -232,7 +234,6 @@ static int fts5ConfigSetEnum(
** eventually free any such error message using sqlite3_free().
*/
static int fts5ConfigParseSpecial(
Fts5Global *pGlobal,
Fts5Config *pConfig, /* Configuration object to update */
const char *zCmd, /* Special command to parse */
const char *zArg, /* Argument to parse */
@@ -296,12 +297,11 @@ static int fts5ConfigParseSpecial(
if( sqlite3_strnicmp("tokenize", zCmd, nCmd)==0 ){
const char *p = (const char*)zArg;
sqlite3_int64 nArg = strlen(zArg) + 1;
char **azArg = sqlite3Fts5MallocZero(&rc, sizeof(char*) * nArg);
char *pDel = sqlite3Fts5MallocZero(&rc, nArg * 2);
char *pSpace = pDel;
char **azArg = sqlite3Fts5MallocZero(&rc, (sizeof(char*) + 2) * nArg);
if( azArg && pSpace ){
if( pConfig->pTok ){
if( azArg ){
char *pSpace = (char*)&azArg[nArg];
if( pConfig->t.azArg ){
*pzErr = sqlite3_mprintf("multiple tokenize=... directives");
rc = SQLITE_ERROR;
}else{
@@ -324,16 +324,14 @@ static int fts5ConfigParseSpecial(
*pzErr = sqlite3_mprintf("parse error in tokenize directive");
rc = SQLITE_ERROR;
}else{
rc = sqlite3Fts5GetTokenizer(pGlobal,
(const char**)azArg, (int)nArg, pConfig,
pzErr
);
pConfig->t.azArg = (const char**)azArg;
pConfig->t.nArg = nArg;
azArg = 0;
}
}
}
sqlite3_free(azArg);
sqlite3_free(pDel);
return rc;
}
@@ -352,6 +350,16 @@ static int fts5ConfigParseSpecial(
return rc;
}
if( sqlite3_strnicmp("contentless_delete", zCmd, nCmd)==0 ){
if( (zArg[0]!='0' && zArg[0]!='1') || zArg[1]!='\0' ){
*pzErr = sqlite3_mprintf("malformed contentless_delete=... directive");
rc = SQLITE_ERROR;
}else{
pConfig->bContentlessDelete = (zArg[0]=='1');
}
return rc;
}
if( sqlite3_strnicmp("content_rowid", zCmd, nCmd)==0 ){
if( pConfig->zContentRowid ){
*pzErr = sqlite3_mprintf("multiple content_rowid=... directives");
@@ -372,6 +380,16 @@ static int fts5ConfigParseSpecial(
return rc;
}
if( sqlite3_strnicmp("locale", zCmd, nCmd)==0 ){
if( (zArg[0]!='0' && zArg[0]!='1') || zArg[1]!='\0' ){
*pzErr = sqlite3_mprintf("malformed locale=... directive");
rc = SQLITE_ERROR;
}else{
pConfig->bLocale = (zArg[0]=='1');
}
return rc;
}
if( sqlite3_strnicmp("detail", zCmd, nCmd)==0 ){
const Fts5Enum aDetail[] = {
{ "none", FTS5_DETAIL_NONE },
@@ -386,20 +404,20 @@ static int fts5ConfigParseSpecial(
return rc;
}
if( sqlite3_strnicmp("tokendata", zCmd, nCmd)==0 ){
if( (zArg[0]!='0' && zArg[0]!='1') || zArg[1]!='\0' ){
*pzErr = sqlite3_mprintf("malformed tokendata=... directive");
rc = SQLITE_ERROR;
}else{
pConfig->bTokendata = (zArg[0]=='1');
}
return rc;
}
*pzErr = sqlite3_mprintf("unrecognized option: \"%.*s\"", nCmd, zCmd);
return SQLITE_ERROR;
}
/*
** Allocate an instance of the default tokenizer ("simple") at
** Fts5Config.pTokenizer. Return SQLITE_OK if successful, or an SQLite error
** code if an error occurs.
*/
static int fts5ConfigDefaultTokenizer(Fts5Global *pGlobal, Fts5Config *pConfig){
assert( pConfig->pTok==0 && pConfig->pTokApi==0 );
return sqlite3Fts5GetTokenizer(pGlobal, 0, 0, pConfig, 0);
}
/*
** Gobble up the first bareword or quoted word from the input buffer zIn.
** Return a pointer to the character immediately following the last in
@@ -498,6 +516,15 @@ static int fts5ConfigMakeExprlist(Fts5Config *p){
}
}
}
if( p->eContent==FTS5_CONTENT_NORMAL && p->bLocale ){
for(i=0; i<p->nCol; i++){
if( p->abUnindexed[i]==0 ){
sqlite3Fts5BufferAppendPrintf(&rc, &buf, ", T.l%d", i);
}else{
sqlite3Fts5BufferAppendPrintf(&rc, &buf, ", NULL");
}
}
}
assert( p->zContentExprlist==0 );
p->zContentExprlist = (char*)buf.p;
@@ -532,6 +559,7 @@ int sqlite3Fts5ConfigParse(
*ppOut = pRet = (Fts5Config*)sqlite3_malloc(sizeof(Fts5Config));
if( pRet==0 ) return SQLITE_NOMEM;
memset(pRet, 0, sizeof(Fts5Config));
pRet->pGlobal = pGlobal;
pRet->db = db;
pRet->iCookie = -1;
@@ -550,6 +578,7 @@ int sqlite3Fts5ConfigParse(
rc = SQLITE_ERROR;
}
assert( (pRet->abUnindexed && pRet->azCol) || rc!=SQLITE_OK );
for(i=3; rc==SQLITE_OK && i<nArg; i++){
const char *zOrig = azArg[i];
const char *z;
@@ -579,7 +608,7 @@ int sqlite3Fts5ConfigParse(
rc = SQLITE_ERROR;
}else{
if( bOption ){
rc = fts5ConfigParseSpecial(pGlobal, pRet,
rc = fts5ConfigParseSpecial(pRet,
ALWAYS(zOne)?zOne:"",
zTwo?zTwo:"",
pzErr
@@ -595,11 +624,26 @@ int sqlite3Fts5ConfigParse(
sqlite3_free(zTwo);
}
/* If a tokenizer= option was successfully parsed, the tokenizer has
** already been allocated. Otherwise, allocate an instance of the default
** tokenizer (unicode61) now. */
if( rc==SQLITE_OK && pRet->pTok==0 ){
rc = fts5ConfigDefaultTokenizer(pGlobal, pRet);
/* We only allow contentless_delete=1 if the table is indeed contentless. */
if( rc==SQLITE_OK
&& pRet->bContentlessDelete
&& pRet->eContent!=FTS5_CONTENT_NONE
){
*pzErr = sqlite3_mprintf(
"contentless_delete=1 requires a contentless table"
);
rc = SQLITE_ERROR;
}
/* We only allow contentless_delete=1 if columnsize=0 is not present.
**
** This restriction may be removed at some point.
*/
if( rc==SQLITE_OK && pRet->bContentlessDelete && pRet->bColumnsize==0 ){
*pzErr = sqlite3_mprintf(
"contentless_delete=1 is incompatible with columnsize=0"
);
rc = SQLITE_ERROR;
}
/* If no zContent option was specified, fill in the default values. */
@@ -643,9 +687,14 @@ int sqlite3Fts5ConfigParse(
void sqlite3Fts5ConfigFree(Fts5Config *pConfig){
if( pConfig ){
int i;
if( pConfig->pTok ){
pConfig->pTokApi->xDelete(pConfig->pTok);
if( pConfig->t.pTok ){
if( pConfig->t.pApi1 ){
pConfig->t.pApi1->xDelete(pConfig->t.pTok);
}else{
pConfig->t.pApi2->xDelete(pConfig->t.pTok);
}
}
sqlite3_free((char*)pConfig->t.azArg);
sqlite3_free(pConfig->zDb);
sqlite3_free(pConfig->zName);
for(i=0; i<pConfig->nCol; i++){
@@ -720,10 +769,24 @@ int sqlite3Fts5Tokenize(
void *pCtx, /* Context passed to xToken() */
int (*xToken)(void*, int, const char*, int, int, int) /* Callback */
){
if( pText==0 ) return SQLITE_OK;
return pConfig->pTokApi->xTokenize(
pConfig->pTok, pCtx, flags, pText, nText, xToken
);
int rc = SQLITE_OK;
if( pText ){
if( pConfig->t.pTok==0 ){
rc = sqlite3Fts5LoadTokenizer(pConfig);
}
if( rc==SQLITE_OK ){
if( pConfig->t.pApi1 ){
rc = pConfig->t.pApi1->xTokenize(
pConfig->t.pTok, pCtx, flags, pText, nText, xToken
);
}else{
rc = pConfig->t.pApi2->xTokenize(pConfig->t.pTok, pCtx, flags,
pText, nText, pConfig->t.pLocale, pConfig->t.nLocale, xToken
);
}
}
}
return rc;
}
/*
@@ -889,6 +952,18 @@ int sqlite3Fts5ConfigSetValue(
}
}
else if( 0==sqlite3_stricmp(zKey, "deletemerge") ){
int nVal = -1;
if( SQLITE_INTEGER==sqlite3_value_numeric_type(pVal) ){
nVal = sqlite3_value_int(pVal);
}else{
*pbBadkey = 1;
}
if( nVal<0 ) nVal = FTS5_DEFAULT_DELETE_AUTOMERGE;
if( nVal>100 ) nVal = 0;
pConfig->nDeleteMerge = nVal;
}
else if( 0==sqlite3_stricmp(zKey, "rank") ){
const char *zIn = (const char*)sqlite3_value_text(pVal);
char *zRank;
@@ -903,6 +978,18 @@ int sqlite3Fts5ConfigSetValue(
rc = SQLITE_OK;
*pbBadkey = 1;
}
}
else if( 0==sqlite3_stricmp(zKey, "secure-delete") ){
int bVal = -1;
if( SQLITE_INTEGER==sqlite3_value_numeric_type(pVal) ){
bVal = sqlite3_value_int(pVal);
}
if( bVal<0 ){
*pbBadkey = 1;
}else{
pConfig->bSecureDelete = (bVal ? 1 : 0);
}
}else{
*pbBadkey = 1;
}
@@ -925,6 +1012,7 @@ int sqlite3Fts5ConfigLoad(Fts5Config *pConfig, int iCookie){
pConfig->nUsermerge = FTS5_DEFAULT_USERMERGE;
pConfig->nCrisisMerge = FTS5_DEFAULT_CRISISMERGE;
pConfig->nHashSize = FTS5_DEFAULT_HASHSIZE;
pConfig->nDeleteMerge = FTS5_DEFAULT_DELETE_AUTOMERGE;
zSql = sqlite3Fts5Mprintf(&rc, zSelect, pConfig->zDb, pConfig->zName);
if( zSql ){
@@ -947,15 +1035,17 @@ int sqlite3Fts5ConfigLoad(Fts5Config *pConfig, int iCookie){
rc = sqlite3_finalize(p);
}
if( rc==SQLITE_OK && iVersion!=FTS5_CURRENT_VERSION ){
if( rc==SQLITE_OK
&& iVersion!=FTS5_CURRENT_VERSION
&& iVersion!=FTS5_CURRENT_VERSION_SECUREDELETE
){
rc = SQLITE_ERROR;
if( pConfig->pzErrmsg ){
assert( 0==*pConfig->pzErrmsg );
*pConfig->pzErrmsg = sqlite3_mprintf(
"invalid fts5 file format (found %d, expected %d) - run 'rebuild'",
iVersion, FTS5_CURRENT_VERSION
);
}
sqlite3Fts5ConfigErrmsg(pConfig, "invalid fts5 file format "
"(found %d, expected %d or %d) - run 'rebuild'",
iVersion, FTS5_CURRENT_VERSION, FTS5_CURRENT_VERSION_SECUREDELETE
);
}else{
pConfig->iVersion = iVersion;
}
if( rc==SQLITE_OK ){
@@ -963,3 +1053,26 @@ int sqlite3Fts5ConfigLoad(Fts5Config *pConfig, int iCookie){
}
return rc;
}
/*
** Set (*pConfig->pzErrmsg) to point to an sqlite3_malloc()ed buffer
** containing the error message created using printf() style formatting
** string zFmt and its trailing arguments.
*/
void sqlite3Fts5ConfigErrmsg(Fts5Config *pConfig, const char *zFmt, ...){
va_list ap; /* ... printf arguments */
char *zMsg = 0;
va_start(ap, zFmt);
zMsg = sqlite3_vmprintf(zFmt, ap);
if( pConfig->pzErrmsg ){
assert( *pConfig->pzErrmsg==0 );
*pConfig->pzErrmsg = zMsg;
}else{
sqlite3_free(zMsg);
}
va_end(ap);
}
+263 -67
View File
@@ -17,6 +17,10 @@
#include "fts5Int.h"
#include "fts5parse.h"
#ifndef SQLITE_FTS5_MAX_EXPR_DEPTH
# define SQLITE_FTS5_MAX_EXPR_DEPTH 256
#endif
/*
** All token types in the generated fts5parse.h file are greater than 0.
*/
@@ -50,18 +54,28 @@ struct Fts5Expr {
/*
** eType:
** Expression node type. Always one of:
** Expression node type. Usually one of:
**
** FTS5_AND (nChild, apChild valid)
** FTS5_OR (nChild, apChild valid)
** FTS5_NOT (nChild, apChild valid)
** FTS5_STRING (pNear valid)
** FTS5_TERM (pNear valid)
**
** An expression node with eType==0 may also exist. It always matches zero
** rows. This is created when a phrase containing no tokens is parsed.
** e.g. "".
**
** iHeight:
** Distance from this node to furthest leaf. This is always 0 for nodes
** of type FTS5_STRING and FTS5_TERM. For all other nodes it is one
** greater than the largest child value.
*/
struct Fts5ExprNode {
int eType; /* Node type */
int bEof; /* True at EOF */
int bNomatch; /* True if entry is not a match */
int iHeight; /* Distance to tree leaf nodes */
/* Next method for this node. */
int (*xNext)(Fts5Expr*, Fts5ExprNode*, int, i64);
@@ -90,7 +104,9 @@ struct Fts5ExprNode {
struct Fts5ExprTerm {
u8 bPrefix; /* True for a prefix term */
u8 bFirst; /* True if token must be first in column */
char *zTerm; /* nul-terminated term */
char *pTerm; /* Term data */
int nQueryTerm; /* Effective size of term in bytes */
int nFullTerm; /* Size of term in bytes incl. tokendata */
Fts5IndexIter *pIter; /* Iterator for this term */
Fts5ExprTerm *pSynonym; /* Pointer to first in list of synonyms */
};
@@ -131,6 +147,31 @@ struct Fts5Parse {
int bPhraseToAnd; /* Convert "a+b" to "a AND b" */
};
/*
** Check that the Fts5ExprNode.iHeight variables are set correctly in
** the expression tree passed as the only argument.
*/
#ifndef NDEBUG
static void assert_expr_depth_ok(int rc, Fts5ExprNode *p){
if( rc==SQLITE_OK ){
if( p->eType==FTS5_TERM || p->eType==FTS5_STRING || p->eType==0 ){
assert( p->iHeight==0 );
}else{
int ii;
int iMaxChild = 0;
for(ii=0; ii<p->nChild; ii++){
Fts5ExprNode *pChild = p->apChild[ii];
iMaxChild = MAX(iMaxChild, pChild->iHeight);
assert_expr_depth_ok(SQLITE_OK, pChild);
}
assert( p->iHeight==iMaxChild+1 );
}
}
}
#else
# define assert_expr_depth_ok(rc, p)
#endif
void sqlite3Fts5ParseError(Fts5Parse *pParse, const char *zFmt, ...){
va_list ap;
va_start(ap, zFmt);
@@ -245,9 +286,12 @@ int sqlite3Fts5ExprNew(
}while( sParse.rc==SQLITE_OK && t!=FTS5_EOF );
sqlite3Fts5ParserFree(pEngine, fts5ParseFree);
assert( sParse.pExpr || sParse.rc!=SQLITE_OK );
assert_expr_depth_ok(sParse.rc, sParse.pExpr);
/* If the LHS of the MATCH expression was a user column, apply the
** implicit column-filter. */
if( iCol<pConfig->nCol && sParse.pExpr && sParse.rc==SQLITE_OK ){
if( sParse.rc==SQLITE_OK && iCol<pConfig->nCol ){
int n = sizeof(Fts5Colset);
Fts5Colset *pColset = (Fts5Colset*)sqlite3Fts5MallocZero(&sParse.rc, n);
if( pColset ){
@@ -264,15 +308,7 @@ int sqlite3Fts5ExprNew(
sParse.rc = SQLITE_NOMEM;
sqlite3Fts5ParseNodeFree(sParse.pExpr);
}else{
if( !sParse.pExpr ){
const int nByte = sizeof(Fts5ExprNode);
pNew->pRoot = (Fts5ExprNode*)sqlite3Fts5MallocZero(&sParse.rc, nByte);
if( pNew->pRoot ){
pNew->pRoot->bEof = 1;
}
}else{
pNew->pRoot = sParse.pExpr;
}
pNew->pRoot = sParse.pExpr;
pNew->pIndex = 0;
pNew->pConfig = pConfig;
pNew->apExprPhrase = sParse.apPhrase;
@@ -285,10 +321,27 @@ int sqlite3Fts5ExprNew(
}
sqlite3_free(sParse.apPhrase);
*pzErr = sParse.zErr;
if( 0==*pzErr ){
*pzErr = sParse.zErr;
}else{
sqlite3_free(sParse.zErr);
}
return sParse.rc;
}
/*
** Assuming that buffer z is at least nByte bytes in size and contains a
** valid utf-8 string, return the number of characters in the string.
*/
static int fts5ExprCountChar(const char *z, int nByte){
int nRet = 0;
int ii;
for(ii=0; ii<nByte; ii++){
if( (z[ii] & 0xC0)!=0x80 ) nRet++;
}
return nRet;
}
/*
** This function is only called when using the special 'trigram' tokenizer.
** Argument zText contains the text of a LIKE or GLOB pattern matched
@@ -326,7 +379,8 @@ int sqlite3Fts5ExprPattern(
if( i==nText
|| zText[i]==aSpec[0] || zText[i]==aSpec[1] || zText[i]==aSpec[2]
){
if( i-iFirst>=3 ){
if( fts5ExprCountChar(&zText[iFirst], i-iFirst)>=3 ){
int jj;
zExpr[iOut++] = '"';
for(jj=iFirst; jj<i; jj++){
@@ -393,7 +447,7 @@ int sqlite3Fts5ExprAnd(Fts5Expr **pp1, Fts5Expr *p2){
Fts5Parse sParse;
memset(&sParse, 0, sizeof(sParse));
if( *pp1 ){
if( *pp1 && p2 ){
Fts5Expr *p1 = *pp1;
int nPhrase = p1->nPhrase + p2->nPhrase;
@@ -418,7 +472,7 @@ int sqlite3Fts5ExprAnd(Fts5Expr **pp1, Fts5Expr *p2){
}
sqlite3_free(p2->apExprPhrase);
sqlite3_free(p2);
}else{
}else if( p2 ){
*pp1 = p2;
}
@@ -916,7 +970,7 @@ static int fts5ExprNearInitAll(
p->pIter = 0;
}
rc = sqlite3Fts5IndexQuery(
pExpr->pIndex, p->zTerm, (int)strlen(p->zTerm),
pExpr->pIndex, p->pTerm, p->nQueryTerm,
(pTerm->bPrefix ? FTS5INDEX_QUERY_PREFIX : 0) |
(pExpr->bDesc ? FTS5INDEX_QUERY_DESC : 0),
pNear->pColset,
@@ -1072,7 +1126,7 @@ static int fts5ExprNodeTest_STRING(
}
}else{
Fts5IndexIter *pIter = pPhrase->aTerm[j].pIter;
if( pIter->iRowid==iLast || pIter->bEof ) continue;
if( pIter->iRowid==iLast ) continue;
bMatch = 0;
if( fts5ExprAdvanceto(pIter, bDesc, &iLast, &rc, &pNode->bEof) ){
return rc;
@@ -1553,7 +1607,7 @@ static void fts5ExprPhraseFree(Fts5ExprPhrase *pPhrase){
Fts5ExprTerm *pSyn;
Fts5ExprTerm *pNext;
Fts5ExprTerm *pTerm = &pPhrase->aTerm[i];
sqlite3_free(pTerm->zTerm);
sqlite3_free(pTerm->pTerm);
sqlite3Fts5IterClose(pTerm->pIter);
for(pSyn=pTerm->pSynonym; pSyn; pSyn=pNext){
pNext = pSyn->pSynonym;
@@ -1594,9 +1648,6 @@ Fts5ExprNearset *sqlite3Fts5ParseNearset(
Fts5ExprNearset *pRet = 0;
if( pParse->rc==SQLITE_OK ){
if( pPhrase==0 ){
return pNear;
}
if( pNear==0 ){
sqlite3_int64 nByte;
nByte = sizeof(Fts5ExprNearset) + SZALLOC * sizeof(Fts5ExprPhrase*);
@@ -1651,6 +1702,7 @@ Fts5ExprNearset *sqlite3Fts5ParseNearset(
typedef struct TokenCtx TokenCtx;
struct TokenCtx {
Fts5ExprPhrase *pPhrase;
Fts5Config *pConfig;
int rc;
};
@@ -1684,8 +1736,12 @@ static int fts5ParseTokenize(
rc = SQLITE_NOMEM;
}else{
memset(pSyn, 0, (size_t)nByte);
pSyn->zTerm = ((char*)pSyn) + sizeof(Fts5ExprTerm) + sizeof(Fts5Buffer);
memcpy(pSyn->zTerm, pToken, nToken);
pSyn->pTerm = ((char*)pSyn) + sizeof(Fts5ExprTerm) + sizeof(Fts5Buffer);
pSyn->nFullTerm = pSyn->nQueryTerm = nToken;
if( pCtx->pConfig->bTokendata ){
pSyn->nQueryTerm = (int)strlen(pSyn->pTerm);
}
memcpy(pSyn->pTerm, pToken, nToken);
pSyn->pSynonym = pPhrase->aTerm[pPhrase->nTerm-1].pSynonym;
pPhrase->aTerm[pPhrase->nTerm-1].pSynonym = pSyn;
}
@@ -1710,7 +1766,11 @@ static int fts5ParseTokenize(
if( rc==SQLITE_OK ){
pTerm = &pPhrase->aTerm[pPhrase->nTerm++];
memset(pTerm, 0, sizeof(Fts5ExprTerm));
pTerm->zTerm = sqlite3Fts5Strndup(&rc, pToken, nToken);
pTerm->pTerm = sqlite3Fts5Strndup(&rc, pToken, nToken);
pTerm->nFullTerm = pTerm->nQueryTerm = nToken;
if( pCtx->pConfig->bTokendata && rc==SQLITE_OK ){
pTerm->nQueryTerm = (int)strlen(pTerm->pTerm);
}
}
}
@@ -1777,6 +1837,7 @@ Fts5ExprPhrase *sqlite3Fts5ParseTerm(
memset(&sCtx, 0, sizeof(TokenCtx));
sCtx.pPhrase = pAppend;
sCtx.pConfig = pConfig;
rc = fts5ParseStringFromToken(pToken, &z);
if( rc==SQLITE_OK ){
@@ -1808,6 +1869,7 @@ Fts5ExprPhrase *sqlite3Fts5ParseTerm(
}else if( sCtx.pPhrase->nTerm ){
sCtx.pPhrase->aTerm[sCtx.pPhrase->nTerm-1].bPrefix = (u8)bPrefix;
}
assert( pParse->apPhrase!=0 );
pParse->apPhrase[pParse->nPhrase-1] = sCtx.pPhrase;
}
@@ -1824,12 +1886,15 @@ int sqlite3Fts5ExprClonePhrase(
Fts5Expr **ppNew
){
int rc = SQLITE_OK; /* Return code */
Fts5ExprPhrase *pOrig; /* The phrase extracted from pExpr */
Fts5ExprPhrase *pOrig = 0; /* The phrase extracted from pExpr */
Fts5Expr *pNew = 0; /* Expression to return via *ppNew */
TokenCtx sCtx = {0,0}; /* Context object for fts5ParseTokenize */
pOrig = pExpr->apExprPhrase[iPhrase];
pNew = (Fts5Expr*)sqlite3Fts5MallocZero(&rc, sizeof(Fts5Expr));
TokenCtx sCtx = {0,0,0}; /* Context object for fts5ParseTokenize */
if( !pExpr || iPhrase<0 || iPhrase>=pExpr->nPhrase ){
rc = SQLITE_RANGE;
}else{
pOrig = pExpr->apExprPhrase[iPhrase];
pNew = (Fts5Expr*)sqlite3Fts5MallocZero(&rc, sizeof(Fts5Expr));
}
if( rc==SQLITE_OK ){
pNew->apExprPhrase = (Fts5ExprPhrase**)sqlite3Fts5MallocZero(&rc,
sizeof(Fts5ExprPhrase*));
@@ -1842,7 +1907,7 @@ int sqlite3Fts5ExprClonePhrase(
pNew->pRoot->pNear = (Fts5ExprNearset*)sqlite3Fts5MallocZero(&rc,
sizeof(Fts5ExprNearset) + sizeof(Fts5ExprPhrase*));
}
if( rc==SQLITE_OK ){
if( rc==SQLITE_OK && ALWAYS(pOrig!=0) ){
Fts5Colset *pColsetOrig = pOrig->pNode->pNear->pColset;
if( pColsetOrig ){
sqlite3_int64 nByte;
@@ -1856,26 +1921,27 @@ int sqlite3Fts5ExprClonePhrase(
}
}
if( pOrig->nTerm ){
int i; /* Used to iterate through phrase terms */
for(i=0; rc==SQLITE_OK && i<pOrig->nTerm; i++){
int tflags = 0;
Fts5ExprTerm *p;
for(p=&pOrig->aTerm[i]; p && rc==SQLITE_OK; p=p->pSynonym){
const char *zTerm = p->zTerm;
rc = fts5ParseTokenize((void*)&sCtx, tflags, zTerm, (int)strlen(zTerm),
0, 0);
tflags = FTS5_TOKEN_COLOCATED;
}
if( rc==SQLITE_OK ){
sCtx.pPhrase->aTerm[i].bPrefix = pOrig->aTerm[i].bPrefix;
sCtx.pPhrase->aTerm[i].bFirst = pOrig->aTerm[i].bFirst;
if( rc==SQLITE_OK ){
if( pOrig->nTerm ){
int i; /* Used to iterate through phrase terms */
sCtx.pConfig = pExpr->pConfig;
for(i=0; rc==SQLITE_OK && i<pOrig->nTerm; i++){
int tflags = 0;
Fts5ExprTerm *p;
for(p=&pOrig->aTerm[i]; p && rc==SQLITE_OK; p=p->pSynonym){
rc = fts5ParseTokenize((void*)&sCtx,tflags,p->pTerm,p->nFullTerm,0,0);
tflags = FTS5_TOKEN_COLOCATED;
}
if( rc==SQLITE_OK ){
sCtx.pPhrase->aTerm[i].bPrefix = pOrig->aTerm[i].bPrefix;
sCtx.pPhrase->aTerm[i].bFirst = pOrig->aTerm[i].bFirst;
}
}
}else{
/* This happens when parsing a token or quoted phrase that contains
** no token characters at all. (e.g ... MATCH '""'). */
sCtx.pPhrase = sqlite3Fts5MallocZero(&rc, sizeof(Fts5ExprPhrase));
}
}else{
/* This happens when parsing a token or quoted phrase that contains
** no token characters at all. (e.g ... MATCH '""'). */
sCtx.pPhrase = sqlite3Fts5MallocZero(&rc, sizeof(Fts5ExprPhrase));
}
if( rc==SQLITE_OK && ALWAYS(sCtx.pPhrase) ){
@@ -2191,7 +2257,11 @@ static void fts5ExprAssignXNext(Fts5ExprNode *pNode){
}
}
/*
** Add pSub as a child of p.
*/
static void fts5ExprAddChildren(Fts5ExprNode *p, Fts5ExprNode *pSub){
int ii = p->nChild;
if( p->eType!=FTS5_NOT && pSub->eType==p->eType ){
int nByte = sizeof(Fts5ExprNode*) * pSub->nChild;
memcpy(&p->apChild[p->nChild], pSub->apChild, nByte);
@@ -2200,6 +2270,9 @@ static void fts5ExprAddChildren(Fts5ExprNode *p, Fts5ExprNode *pSub){
}else{
p->apChild[p->nChild++] = pSub;
}
for( ; ii<p->nChild; ii++){
p->iHeight = MAX(p->iHeight, p->apChild[ii]->iHeight + 1);
}
}
/*
@@ -2230,6 +2303,7 @@ static Fts5ExprNode *fts5ParsePhraseToAnd(
if( pRet ){
pRet->eType = FTS5_AND;
pRet->nChild = nTerm;
pRet->iHeight = 1;
fts5ExprAssignXNext(pRet);
pParse->nPhrase--;
for(ii=0; ii<nTerm; ii++){
@@ -2240,11 +2314,13 @@ static Fts5ExprNode *fts5ParsePhraseToAnd(
if( parseGrowPhraseArray(pParse) ){
fts5ExprPhraseFree(pPhrase);
}else{
Fts5ExprTerm *p = &pNear->apPhrase[0]->aTerm[ii];
Fts5ExprTerm *pTo = &pPhrase->aTerm[0];
pParse->apPhrase[pParse->nPhrase++] = pPhrase;
pPhrase->nTerm = 1;
pPhrase->aTerm[0].zTerm = sqlite3Fts5Strndup(
&pParse->rc, pNear->apPhrase[0]->aTerm[ii].zTerm, -1
);
pTo->pTerm = sqlite3Fts5Strndup(&pParse->rc, p->pTerm, p->nFullTerm);
pTo->nQueryTerm = p->nQueryTerm;
pTo->nFullTerm = p->nFullTerm;
pRet->apChild[ii] = sqlite3Fts5ParseNode(pParse, FTS5_STRING,
0, 0, sqlite3Fts5ParseNearset(pParse, 0, pPhrase)
);
@@ -2328,13 +2404,25 @@ Fts5ExprNode *sqlite3Fts5ParseNode(
"fts5: %s queries are not supported (detail!=full)",
pNear->nPhrase==1 ? "phrase": "NEAR"
);
sqlite3_free(pRet);
sqlite3Fts5ParseNodeFree(pRet);
pRet = 0;
pNear = 0;
assert( pLeft==0 && pRight==0 );
}
}
}else{
assert( pNear==0 );
fts5ExprAddChildren(pRet, pLeft);
fts5ExprAddChildren(pRet, pRight);
pLeft = pRight = 0;
if( pRet->iHeight>SQLITE_FTS5_MAX_EXPR_DEPTH ){
sqlite3Fts5ParseError(pParse,
"fts5 expression tree is too large (maximum depth %d)",
SQLITE_FTS5_MAX_EXPR_DEPTH
);
sqlite3Fts5ParseNodeFree(pRet);
pRet = 0;
}
}
}
}
@@ -2370,6 +2458,7 @@ Fts5ExprNode *sqlite3Fts5ParseImplicitAnd(
assert( pRight->eType==FTS5_STRING
|| pRight->eType==FTS5_TERM
|| pRight->eType==FTS5_EOF
|| (pRight->eType==FTS5_AND && pParse->bPhraseToAnd)
);
if( pLeft->eType==FTS5_AND ){
@@ -2383,6 +2472,8 @@ Fts5ExprNode *sqlite3Fts5ParseImplicitAnd(
);
if( pRight->eType==FTS5_EOF ){
assert( pParse->apPhrase!=0 );
assert( pParse->nPhrase>0 );
assert( pParse->apPhrase[pParse->nPhrase-1]==pRight->pNear->apPhrase[0] );
sqlite3Fts5ParseNodeFree(pRight);
pRet = pLeft;
@@ -2413,7 +2504,7 @@ Fts5ExprNode *sqlite3Fts5ParseImplicitAnd(
return pRet;
}
#ifdef SQLITE_TEST
#if defined(SQLITE_TEST) || defined(SQLITE_FTS5_DEBUG)
static char *fts5ExprTermPrint(Fts5ExprTerm *pTerm){
sqlite3_int64 nByte = 0;
Fts5ExprTerm *p;
@@ -2421,16 +2512,17 @@ static char *fts5ExprTermPrint(Fts5ExprTerm *pTerm){
/* Determine the maximum amount of space required. */
for(p=pTerm; p; p=p->pSynonym){
nByte += (int)strlen(pTerm->zTerm) * 2 + 3 + 2;
nByte += pTerm->nQueryTerm * 2 + 3 + 2;
}
zQuoted = sqlite3_malloc64(nByte);
if( zQuoted ){
int i = 0;
for(p=pTerm; p; p=p->pSynonym){
char *zIn = p->zTerm;
char *zIn = p->pTerm;
char *zEnd = &zIn[p->nQueryTerm];
zQuoted[i++] = '"';
while( *zIn ){
while( zIn<zEnd ){
if( *zIn=='"' ) zQuoted[i++] = '"';
zQuoted[i++] = *zIn++;
}
@@ -2508,8 +2600,10 @@ static char *fts5ExprPrintTcl(
zRet = fts5PrintfAppend(zRet, " {");
for(iTerm=0; zRet && iTerm<pPhrase->nTerm; iTerm++){
char *zTerm = pPhrase->aTerm[iTerm].zTerm;
zRet = fts5PrintfAppend(zRet, "%s%s", iTerm==0?"":" ", zTerm);
Fts5ExprTerm *p = &pPhrase->aTerm[iTerm];
zRet = fts5PrintfAppend(zRet, "%s%.*s", iTerm==0?"":" ",
p->nQueryTerm, p->pTerm
);
if( pPhrase->aTerm[iTerm].bPrefix ){
zRet = fts5PrintfAppend(zRet, "*");
}
@@ -2519,6 +2613,8 @@ static char *fts5ExprPrintTcl(
if( zRet==0 ) return 0;
}
}else if( pExpr->eType==0 ){
zRet = sqlite3_mprintf("{}");
}else{
char const *zOp = 0;
int i;
@@ -2780,14 +2876,14 @@ static void fts5ExprFold(
sqlite3_result_int(pCtx, sqlite3Fts5UnicodeFold(iCode, bRemoveDiacritics));
}
}
#endif /* ifdef SQLITE_TEST */
#endif /* if SQLITE_TEST || SQLITE_FTS5_DEBUG */
/*
** This is called during initialization to register the fts5_expr() scalar
** UDF with the SQLite handle passed as the only argument.
*/
int sqlite3Fts5ExprInit(Fts5Global *pGlobal, sqlite3 *db){
#ifdef SQLITE_TEST
#if defined(SQLITE_TEST) || defined(SQLITE_FTS5_DEBUG)
struct Fts5ExprFunc {
const char *z;
void (*x)(sqlite3_context*,int,sqlite3_value**);
@@ -2908,6 +3004,17 @@ static int fts5ExprColsetTest(Fts5Colset *pColset, int iCol){
return 0;
}
/*
** pToken is a buffer nToken bytes in size that may or may not contain
** an embedded 0x00 byte. If it does, return the number of bytes in
** the buffer before the 0x00. If it does not, return nToken.
*/
static int fts5QueryTerm(const char *pToken, int nToken){
int ii;
for(ii=0; ii<nToken && pToken[ii]; ii++){}
return ii;
}
static int fts5ExprPopulatePoslistsCb(
void *pCtx, /* Copy of 2nd argument to xTokenize() */
int tflags, /* Mask of FTS5_TOKEN_* flags */
@@ -2919,22 +3026,33 @@ static int fts5ExprPopulatePoslistsCb(
Fts5ExprCtx *p = (Fts5ExprCtx*)pCtx;
Fts5Expr *pExpr = p->pExpr;
int i;
int nQuery = nToken;
i64 iRowid = pExpr->pRoot->iRowid;
UNUSED_PARAM2(iUnused1, iUnused2);
if( nToken>FTS5_MAX_TOKEN_SIZE ) nToken = FTS5_MAX_TOKEN_SIZE;
if( nQuery>FTS5_MAX_TOKEN_SIZE ) nQuery = FTS5_MAX_TOKEN_SIZE;
if( pExpr->pConfig->bTokendata ){
nQuery = fts5QueryTerm(pToken, nQuery);
}
if( (tflags & FTS5_TOKEN_COLOCATED)==0 ) p->iOff++;
for(i=0; i<pExpr->nPhrase; i++){
Fts5ExprTerm *pTerm;
Fts5ExprTerm *pT;
if( p->aPopulator[i].bOk==0 ) continue;
for(pTerm=&pExpr->apExprPhrase[i]->aTerm[0]; pTerm; pTerm=pTerm->pSynonym){
int nTerm = (int)strlen(pTerm->zTerm);
if( (nTerm==nToken || (nTerm<nToken && pTerm->bPrefix))
&& memcmp(pTerm->zTerm, pToken, nTerm)==0
for(pT=&pExpr->apExprPhrase[i]->aTerm[0]; pT; pT=pT->pSynonym){
if( (pT->nQueryTerm==nQuery || (pT->nQueryTerm<nQuery && pT->bPrefix))
&& memcmp(pT->pTerm, pToken, pT->nQueryTerm)==0
){
int rc = sqlite3Fts5PoslistWriterAppend(
&pExpr->apExprPhrase[i]->poslist, &p->aPopulator[i].writer, p->iOff
);
if( rc==SQLITE_OK && pExpr->pConfig->bTokendata && !pT->bPrefix ){
int iCol = p->iOff>>32;
int iTokOff = p->iOff & 0x7FFFFFFF;
rc = sqlite3Fts5IndexIterWriteTokendata(
pT->pIter, pToken, nToken, iRowid, iCol, iTokOff
);
}
if( rc ) return rc;
break;
}
@@ -2988,6 +3106,7 @@ static int fts5ExprCheckPoslists(Fts5ExprNode *pNode, i64 iRowid){
pNode->iRowid = iRowid;
pNode->bEof = 0;
switch( pNode->eType ){
case 0:
case FTS5_TERM:
case FTS5_STRING:
return (pNode->pNear->apPhrase[0]->poslist.n>0);
@@ -3069,3 +3188,80 @@ int sqlite3Fts5ExprPhraseCollist(
return rc;
}
/*
** Does the work of the fts5_api.xQueryToken() API method.
*/
int sqlite3Fts5ExprQueryToken(
Fts5Expr *pExpr,
int iPhrase,
int iToken,
const char **ppOut,
int *pnOut
){
Fts5ExprPhrase *pPhrase = 0;
if( iPhrase<0 || iPhrase>=pExpr->nPhrase ){
return SQLITE_RANGE;
}
pPhrase = pExpr->apExprPhrase[iPhrase];
if( iToken<0 || iToken>=pPhrase->nTerm ){
return SQLITE_RANGE;
}
*ppOut = pPhrase->aTerm[iToken].pTerm;
*pnOut = pPhrase->aTerm[iToken].nFullTerm;
return SQLITE_OK;
}
/*
** Does the work of the fts5_api.xInstToken() API method.
*/
int sqlite3Fts5ExprInstToken(
Fts5Expr *pExpr,
i64 iRowid,
int iPhrase,
int iCol,
int iOff,
int iToken,
const char **ppOut,
int *pnOut
){
Fts5ExprPhrase *pPhrase = 0;
Fts5ExprTerm *pTerm = 0;
int rc = SQLITE_OK;
if( iPhrase<0 || iPhrase>=pExpr->nPhrase ){
return SQLITE_RANGE;
}
pPhrase = pExpr->apExprPhrase[iPhrase];
if( iToken<0 || iToken>=pPhrase->nTerm ){
return SQLITE_RANGE;
}
pTerm = &pPhrase->aTerm[iToken];
if( pTerm->bPrefix==0 ){
if( pExpr->pConfig->bTokendata ){
rc = sqlite3Fts5IterToken(
pTerm->pIter, iRowid, iCol, iOff+iToken, ppOut, pnOut
);
}else{
*ppOut = pTerm->pTerm;
*pnOut = pTerm->nFullTerm;
}
}
return rc;
}
/*
** Clear the token mappings for all Fts5IndexIter objects mannaged by
** the expression passed as the only argument.
*/
void sqlite3Fts5ExprClearTokens(Fts5Expr *pExpr){
int ii;
for(ii=0; ii<pExpr->nPhrase; ii++){
Fts5ExprTerm *pT;
for(pT=&pExpr->apExprPhrase[ii]->aTerm[0]; pT; pT=pT->pSynonym){
sqlite3Fts5IndexIterClearTokendata(pT->pIter);
}
}
}
+54 -24
View File
@@ -36,10 +36,15 @@ struct Fts5Hash {
/*
** Each entry in the hash table is represented by an object of the
** following type. Each object, its key (a nul-terminated string) and
** its current data are stored in a single memory allocation. The
** key immediately follows the object in memory. The position list
** data immediately follows the key data in memory.
** following type. Each object, its key, and its current data are stored
** in a single memory allocation. The key immediately follows the object
** in memory. The position list data immediately follows the key data
** in memory.
**
** The key is Fts5HashEntry.nKey bytes in size. It consists of a single
** byte identifying the index (either the main term index or a prefix-index),
** followed by the term data. For example: "0token". There is no
** nul-terminator - in this case nKey=6.
**
** The data that follows the key is in a similar, but not identical format
** to the doclist data stored in the database. It is:
@@ -174,8 +179,7 @@ static int fts5HashResize(Fts5Hash *pHash){
unsigned int iHash;
Fts5HashEntry *p = apOld[i];
apOld[i] = p->pHashNext;
iHash = fts5HashKey(nNew, (u8*)fts5EntryKey(p),
(int)strlen(fts5EntryKey(p)));
iHash = fts5HashKey(nNew, (u8*)fts5EntryKey(p), p->nKey);
p->pHashNext = apNew[iHash];
apNew[iHash] = p;
}
@@ -259,7 +263,7 @@ int sqlite3Fts5HashWrite(
for(p=pHash->aSlot[iHash]; p; p=p->pHashNext){
char *zKey = fts5EntryKey(p);
if( zKey[0]==bByte
&& p->nKey==nToken
&& p->nKey==nToken+1
&& memcmp(&zKey[1], pToken, nToken)==0
){
break;
@@ -289,9 +293,9 @@ int sqlite3Fts5HashWrite(
zKey[0] = bByte;
memcpy(&zKey[1], pToken, nToken);
assert( iHash==fts5HashKey(pHash->nSlot, (u8*)zKey, nToken+1) );
p->nKey = nToken;
p->nKey = nToken+1;
zKey[nToken+1] = '\0';
p->nData = nToken+1 + 1 + sizeof(Fts5HashEntry);
p->nData = nToken+1 + sizeof(Fts5HashEntry);
p->pHashNext = pHash->aSlot[iHash];
pHash->aSlot[iHash] = p;
pHash->nEntry++;
@@ -408,12 +412,17 @@ static Fts5HashEntry *fts5HashEntryMerge(
*ppOut = p1;
p1 = 0;
}else{
int i = 0;
char *zKey1 = fts5EntryKey(p1);
char *zKey2 = fts5EntryKey(p2);
while( zKey1[i]==zKey2[i] ) i++;
int nMin = MIN(p1->nKey, p2->nKey);
if( ((u8)zKey1[i])>((u8)zKey2[i]) ){
int cmp = memcmp(zKey1, zKey2, nMin);
if( cmp==0 ){
cmp = p1->nKey - p2->nKey;
}
assert( cmp!=0 );
if( cmp>0 ){
/* p2 is smaller */
*ppOut = p2;
ppOut = &p2->pScanNext;
@@ -432,10 +441,8 @@ static Fts5HashEntry *fts5HashEntryMerge(
}
/*
** Extract all tokens from hash table iHash and link them into a list
** in sorted order. The hash table is cleared before returning. It is
** the responsibility of the caller to free the elements of the returned
** list.
** Link all tokens from hash table iHash into a list in sorted order. The
** tokens are not removed from the hash table.
*/
static int fts5HashEntrySort(
Fts5Hash *pHash,
@@ -457,7 +464,7 @@ static int fts5HashEntrySort(
Fts5HashEntry *pIter;
for(pIter=pHash->aSlot[iSlot]; pIter; pIter=pIter->pHashNext){
if( pTerm==0
|| (pIter->nKey+1>=nTerm && 0==memcmp(fts5EntryKey(pIter), pTerm, nTerm))
|| (pIter->nKey>=nTerm && 0==memcmp(fts5EntryKey(pIter), pTerm, nTerm))
){
Fts5HashEntry *pEntry = pIter;
pEntry->pScanNext = 0;
@@ -475,7 +482,6 @@ static int fts5HashEntrySort(
pList = fts5HashEntryMerge(pList, ap[i]);
}
pHash->nEntry = 0;
sqlite3_free(ap);
*ppSorted = pList;
return SQLITE_OK;
@@ -497,12 +503,11 @@ int sqlite3Fts5HashQuery(
for(p=pHash->aSlot[iHash]; p; p=p->pHashNext){
zKey = fts5EntryKey(p);
assert( p->nKey+1==(int)strlen(zKey) );
if( nTerm==p->nKey+1 && memcmp(zKey, pTerm, nTerm)==0 ) break;
if( nTerm==p->nKey && memcmp(zKey, pTerm, nTerm)==0 ) break;
}
if( p ){
int nHashPre = sizeof(Fts5HashEntry) + nTerm + 1;
int nHashPre = sizeof(Fts5HashEntry) + nTerm;
int nList = p->nData - nHashPre;
u8 *pRet = (u8*)(*ppOut = sqlite3_malloc64(nPre + nList + 10));
if( pRet ){
@@ -529,6 +534,28 @@ int sqlite3Fts5HashScanInit(
return fts5HashEntrySort(p, pTerm, nTerm, &p->pScan);
}
#ifdef SQLITE_DEBUG
static int fts5HashCount(Fts5Hash *pHash){
int nEntry = 0;
int ii;
for(ii=0; ii<pHash->nSlot; ii++){
Fts5HashEntry *p = 0;
for(p=pHash->aSlot[ii]; p; p=p->pHashNext){
nEntry++;
}
}
return nEntry;
}
#endif
/*
** Return true if the hash table is empty, false otherwise.
*/
int sqlite3Fts5HashIsEmpty(Fts5Hash *pHash){
assert( pHash->nEntry==fts5HashCount(pHash) );
return pHash->nEntry==0;
}
void sqlite3Fts5HashScanNext(Fts5Hash *p){
assert( !sqlite3Fts5HashScanEof(p) );
p->pScan = p->pScan->pScanNext;
@@ -541,19 +568,22 @@ int sqlite3Fts5HashScanEof(Fts5Hash *p){
void sqlite3Fts5HashScanEntry(
Fts5Hash *pHash,
const char **pzTerm, /* OUT: term (nul-terminated) */
int *pnTerm, /* OUT: Size of term in bytes */
const u8 **ppDoclist, /* OUT: pointer to doclist */
int *pnDoclist /* OUT: size of doclist in bytes */
){
Fts5HashEntry *p;
if( (p = pHash->pScan) ){
char *zKey = fts5EntryKey(p);
int nTerm = (int)strlen(zKey);
int nTerm = p->nKey;
fts5HashAddPoslistSize(pHash, p, 0);
*pzTerm = zKey;
*ppDoclist = (const u8*)&zKey[nTerm+1];
*pnDoclist = p->nData - (sizeof(Fts5HashEntry) + nTerm + 1);
*pnTerm = nTerm;
*ppDoclist = (const u8*)&zKey[nTerm];
*pnDoclist = p->nData - (sizeof(Fts5HashEntry) + nTerm);
}else{
*pzTerm = 0;
*pnTerm = 0;
*ppDoclist = 0;
*pnDoclist = 0;
}
+2583 -326
View File
File diff suppressed because it is too large Load Diff
+1047 -204
View File
File diff suppressed because it is too large Load Diff
+422 -95
View File
@@ -16,13 +16,40 @@
#include "fts5Int.h"
/*
** pSavedRow:
** SQL statement FTS5_STMT_LOOKUP2 is a copy of FTS5_STMT_LOOKUP, it
** does a by-rowid lookup to retrieve a single row from the %_content
** table or equivalent external-content table/view.
**
** However, FTS5_STMT_LOOKUP2 is only used when retrieving the original
** values for a row being UPDATEd. In that case, the SQL statement is
** not reset and pSavedRow is set to point at it. This is so that the
** insert operation that follows the delete may access the original
** row values for any new values for which sqlite3_value_nochange() returns
** true. i.e. if the user executes:
**
** CREATE VIRTUAL TABLE ft USING fts5(a, b, c, locale=1);
** ...
** UPDATE fts SET a=?, b=? WHERE rowid=?;
**
** then the value passed to the xUpdate() method of this table as the
** new.c value is an sqlite3_value_nochange() value. So in this case it
** must be read from the saved row stored in Fts5Storage.pSavedRow.
**
** This is necessary - using sqlite3_value_nochange() instead of just having
** SQLite pass the original value back via xUpdate() - so as not to discard
** any locale information associated with such values.
**
*/
struct Fts5Storage {
Fts5Config *pConfig;
Fts5Index *pIndex;
int bTotalsValid; /* True if nTotalRow/aTotalSize[] are valid */
i64 nTotalRow; /* Total number of rows in FTS table */
i64 *aTotalSize; /* Total sizes of each column */
sqlite3_stmt *aStmt[11];
sqlite3_stmt *pSavedRow;
sqlite3_stmt *aStmt[12];
};
@@ -36,14 +63,39 @@ struct Fts5Storage {
# error "FTS5_STMT_LOOKUP mismatch"
#endif
#define FTS5_STMT_INSERT_CONTENT 3
#define FTS5_STMT_REPLACE_CONTENT 4
#define FTS5_STMT_DELETE_CONTENT 5
#define FTS5_STMT_REPLACE_DOCSIZE 6
#define FTS5_STMT_DELETE_DOCSIZE 7
#define FTS5_STMT_LOOKUP_DOCSIZE 8
#define FTS5_STMT_REPLACE_CONFIG 9
#define FTS5_STMT_SCAN 10
#define FTS5_STMT_LOOKUP2 3
#define FTS5_STMT_INSERT_CONTENT 4
#define FTS5_STMT_REPLACE_CONTENT 5
#define FTS5_STMT_DELETE_CONTENT 6
#define FTS5_STMT_REPLACE_DOCSIZE 7
#define FTS5_STMT_DELETE_DOCSIZE 8
#define FTS5_STMT_LOOKUP_DOCSIZE 9
#define FTS5_STMT_REPLACE_CONFIG 10
#define FTS5_STMT_SCAN 11
/*
** Return a pointer to a buffer obtained from sqlite3_malloc() that contains
** nBind comma-separated question marks. e.g. if nBind is passed 5, this
** function returns "?,?,?,?,?".
**
** If *pRc is not SQLITE_OK when this function is called, it is a no-op and
** NULL is returned immediately. Or, if the attempt to malloc a buffer
** fails, then *pRc is set to SQLITE_NOMEM and NULL is returned. Otherwise,
** if it is SQLITE_OK when this function is called and the malloc() succeeds,
** *pRc is left unchanged.
*/
static char *fts5BindingsList(int *pRc, int nBind){
char *zBind = sqlite3Fts5MallocZero(pRc, 1 + nBind*2);
if( zBind ){
int ii;
for(ii=0; ii<nBind; ii++){
zBind[ii*2] = '?';
zBind[ii*2 + 1] = ',';
}
zBind[ii*2-1] = '\0';
}
return zBind;
}
/*
** Prepare the two insert statements - Fts5Storage.pInsertContent and
@@ -73,14 +125,15 @@ static int fts5StorageGetStmt(
"SELECT %s FROM %s T WHERE T.%Q >= ? AND T.%Q <= ? ORDER BY T.%Q ASC",
"SELECT %s FROM %s T WHERE T.%Q <= ? AND T.%Q >= ? ORDER BY T.%Q DESC",
"SELECT %s FROM %s T WHERE T.%Q=?", /* LOOKUP */
"SELECT %s FROM %s T WHERE T.%Q=?", /* LOOKUP2 */
"INSERT INTO %Q.'%q_content' VALUES(%s)", /* INSERT_CONTENT */
"REPLACE INTO %Q.'%q_content' VALUES(%s)", /* REPLACE_CONTENT */
"DELETE FROM %Q.'%q_content' WHERE id=?", /* DELETE_CONTENT */
"REPLACE INTO %Q.'%q_docsize' VALUES(?,?)", /* REPLACE_DOCSIZE */
"REPLACE INTO %Q.'%q_docsize' VALUES(?,?%s)", /* REPLACE_DOCSIZE */
"DELETE FROM %Q.'%q_docsize' WHERE id=?", /* DELETE_DOCSIZE */
"SELECT sz FROM %Q.'%q_docsize' WHERE id=?", /* LOOKUP_DOCSIZE */
"SELECT sz%s FROM %Q.'%q_docsize' WHERE id=?", /* LOOKUP_DOCSIZE */
"REPLACE INTO %Q.'%q_config' VALUES(?,?)", /* REPLACE_CONFIG */
"SELECT %s FROM %s AS T", /* SCAN */
@@ -88,6 +141,8 @@ static int fts5StorageGetStmt(
Fts5Config *pC = p->pConfig;
char *zSql = 0;
assert( ArraySize(azStmt)==ArraySize(p->aStmt) );
switch( eStmt ){
case FTS5_STMT_SCAN:
zSql = sqlite3_mprintf(azStmt[eStmt],
@@ -104,30 +159,45 @@ static int fts5StorageGetStmt(
break;
case FTS5_STMT_LOOKUP:
case FTS5_STMT_LOOKUP2:
zSql = sqlite3_mprintf(azStmt[eStmt],
pC->zContentExprlist, pC->zContent, pC->zContentRowid
);
break;
case FTS5_STMT_INSERT_CONTENT:
case FTS5_STMT_REPLACE_CONTENT: {
int nCol = pC->nCol + 1;
case FTS5_STMT_INSERT_CONTENT: {
int nCol = 0;
char *zBind;
int i;
zBind = sqlite3_malloc64(1 + nCol*2);
if( zBind ){
for(i=0; i<nCol; i++){
zBind[i*2] = '?';
zBind[i*2 + 1] = ',';
nCol = 1 + pC->nCol;
if( pC->bLocale ){
for(i=0; i<pC->nCol; i++){
if( pC->abUnindexed[i]==0 ) nCol++;
}
zBind[i*2-1] = '\0';
}
zBind = fts5BindingsList(&rc, nCol);
if( zBind ){
zSql = sqlite3_mprintf(azStmt[eStmt], pC->zDb, pC->zName, zBind);
sqlite3_free(zBind);
}
break;
}
case FTS5_STMT_REPLACE_DOCSIZE:
zSql = sqlite3_mprintf(azStmt[eStmt], pC->zDb, pC->zName,
(pC->bContentlessDelete ? ",?" : "")
);
break;
case FTS5_STMT_LOOKUP_DOCSIZE:
zSql = sqlite3_mprintf(azStmt[eStmt],
(pC->bContentlessDelete ? ",origin" : ""),
pC->zDb, pC->zName
);
break;
default:
zSql = sqlite3_mprintf(azStmt[eStmt], pC->zDb, pC->zName);
break;
@@ -137,7 +207,7 @@ static int fts5StorageGetStmt(
rc = SQLITE_NOMEM;
}else{
int f = SQLITE_PREPARE_PERSISTENT;
if( eStmt>FTS5_STMT_LOOKUP ) f |= SQLITE_PREPARE_NO_VTAB;
if( eStmt>FTS5_STMT_LOOKUP2 ) f |= SQLITE_PREPARE_NO_VTAB;
p->pConfig->bLock++;
rc = sqlite3_prepare_v3(pC->db, zSql, -1, f, &p->aStmt[eStmt], 0);
p->pConfig->bLock--;
@@ -299,7 +369,7 @@ int sqlite3Fts5StorageOpen(
if( bCreate ){
if( pConfig->eContent==FTS5_CONTENT_NORMAL ){
int nDefn = 32 + pConfig->nCol*10;
char *zDefn = sqlite3_malloc64(32 + (sqlite3_int64)pConfig->nCol * 10);
char *zDefn = sqlite3_malloc64(32 + (sqlite3_int64)pConfig->nCol * 20);
if( zDefn==0 ){
rc = SQLITE_NOMEM;
}else{
@@ -311,15 +381,25 @@ int sqlite3Fts5StorageOpen(
sqlite3_snprintf(nDefn-iOff, &zDefn[iOff], ", c%d", i);
iOff += (int)strlen(&zDefn[iOff]);
}
if( pConfig->bLocale ){
for(i=0; i<pConfig->nCol; i++){
if( pConfig->abUnindexed[i]==0 ){
sqlite3_snprintf(nDefn-iOff, &zDefn[iOff], ", l%d", i);
iOff += (int)strlen(&zDefn[iOff]);
}
}
}
rc = sqlite3Fts5CreateTable(pConfig, "content", zDefn, 0, pzErr);
}
sqlite3_free(zDefn);
}
if( rc==SQLITE_OK && pConfig->bColumnsize ){
rc = sqlite3Fts5CreateTable(
pConfig, "docsize", "id INTEGER PRIMARY KEY, sz BLOB", 0, pzErr
);
const char *zCols = "id INTEGER PRIMARY KEY, sz BLOB";
if( pConfig->bContentlessDelete ){
zCols = "id INTEGER PRIMARY KEY, sz BLOB, origin INTEGER";
}
rc = sqlite3Fts5CreateTable(pConfig, "docsize", zCols, 0, pzErr);
}
if( rc==SQLITE_OK ){
rc = sqlite3Fts5CreateTable(
@@ -384,57 +464,115 @@ static int fts5StorageInsertCallback(
return sqlite3Fts5IndexWrite(pIdx, pCtx->iCol, pCtx->szCol-1, pToken, nToken);
}
/*
** This function is used as part of an UPDATE statement that modifies the
** rowid of a row. In that case, this function is called first to set
** Fts5Storage.pSavedRow to point to a statement that may be used to
** access the original values of the row being deleted - iDel.
**
** SQLITE_OK is returned if successful, or an SQLite error code otherwise.
** It is not considered an error if row iDel does not exist. In this case
** pSavedRow is not set and SQLITE_OK returned.
*/
int sqlite3Fts5StorageFindDeleteRow(Fts5Storage *p, i64 iDel){
int rc = SQLITE_OK;
sqlite3_stmt *pSeek = 0;
assert( p->pSavedRow==0 );
rc = fts5StorageGetStmt(p, FTS5_STMT_LOOKUP+1, &pSeek, 0);
if( rc==SQLITE_OK ){
sqlite3_bind_int64(pSeek, 1, iDel);
if( sqlite3_step(pSeek)!=SQLITE_ROW ){
rc = sqlite3_reset(pSeek);
}else{
p->pSavedRow = pSeek;
}
}
return rc;
}
/*
** If a row with rowid iDel is present in the %_content table, add the
** delete-markers to the FTS index necessary to delete it. Do not actually
** remove the %_content row at this time though.
**
** If parameter bSaveRow is true, then Fts5Storage.pSavedRow is left
** pointing to a statement (FTS5_STMT_LOOKUP2) that may be used to access
** the original values of the row being deleted. This is used by UPDATE
** statements.
*/
static int fts5StorageDeleteFromIndex(
Fts5Storage *p,
i64 iDel,
sqlite3_value **apVal
sqlite3_value **apVal,
int bSaveRow /* True to set pSavedRow */
){
Fts5Config *pConfig = p->pConfig;
sqlite3_stmt *pSeek = 0; /* SELECT to read row iDel from %_data */
int rc; /* Return code */
int rc = SQLITE_OK; /* Return code */
int rc2; /* sqlite3_reset() return code */
int iCol;
Fts5InsertCtx ctx;
assert( bSaveRow==0 || apVal==0 );
assert( bSaveRow==0 || bSaveRow==1 );
assert( FTS5_STMT_LOOKUP2==FTS5_STMT_LOOKUP+1 );
if( apVal==0 ){
rc = fts5StorageGetStmt(p, FTS5_STMT_LOOKUP, &pSeek, 0);
if( rc!=SQLITE_OK ) return rc;
sqlite3_bind_int64(pSeek, 1, iDel);
if( sqlite3_step(pSeek)!=SQLITE_ROW ){
return sqlite3_reset(pSeek);
if( p->pSavedRow && bSaveRow ){
pSeek = p->pSavedRow;
p->pSavedRow = 0;
}else{
rc = fts5StorageGetStmt(p, FTS5_STMT_LOOKUP+bSaveRow, &pSeek, 0);
if( rc!=SQLITE_OK ) return rc;
sqlite3_bind_int64(pSeek, 1, iDel);
if( sqlite3_step(pSeek)!=SQLITE_ROW ){
return sqlite3_reset(pSeek);
}
}
}
ctx.pStorage = p;
ctx.iCol = -1;
rc = sqlite3Fts5IndexBeginWrite(p->pIndex, 1, iDel);
for(iCol=1; rc==SQLITE_OK && iCol<=pConfig->nCol; iCol++){
if( pConfig->abUnindexed[iCol-1]==0 ){
const char *zText;
int nText;
sqlite3_value *pVal = 0;
const char *pText = 0;
int nText = 0;
const char *pLoc = 0;
int nLoc = 0;
assert( pSeek==0 || apVal==0 );
assert( pSeek!=0 || apVal!=0 );
if( pSeek ){
zText = (const char*)sqlite3_column_text(pSeek, iCol);
nText = sqlite3_column_bytes(pSeek, iCol);
}else if( ALWAYS(apVal) ){
zText = (const char*)sqlite3_value_text(apVal[iCol-1]);
nText = sqlite3_value_bytes(apVal[iCol-1]);
pVal = sqlite3_column_value(pSeek, iCol);
}else{
continue;
pVal = apVal[iCol-1];
}
ctx.szCol = 0;
rc = sqlite3Fts5Tokenize(pConfig, FTS5_TOKENIZE_DOCUMENT,
zText, nText, (void*)&ctx, fts5StorageInsertCallback
);
p->aTotalSize[iCol-1] -= (i64)ctx.szCol;
if( p->aTotalSize[iCol-1]<0 ){
rc = FTS5_CORRUPT;
if( pConfig->bLocale && sqlite3Fts5IsLocaleValue(pConfig, pVal) ){
rc = sqlite3Fts5DecodeLocaleValue(pVal, &pText, &nText, &pLoc, &nLoc);
}else{
pText = (const char*)sqlite3_value_text(pVal);
nText = sqlite3_value_bytes(pVal);
if( pConfig->bLocale && pSeek ){
pLoc = (const char*)sqlite3_column_text(pSeek, iCol + pConfig->nCol);
nLoc = sqlite3_column_bytes(pSeek, iCol + pConfig->nCol);
}
}
if( rc==SQLITE_OK ){
sqlite3Fts5SetLocale(pConfig, pLoc, nLoc);
ctx.szCol = 0;
rc = sqlite3Fts5Tokenize(pConfig, FTS5_TOKENIZE_DOCUMENT,
pText, nText, (void*)&ctx, fts5StorageInsertCallback
);
p->aTotalSize[iCol-1] -= (i64)ctx.szCol;
if( rc==SQLITE_OK && p->aTotalSize[iCol-1]<0 ){
rc = FTS5_CORRUPT;
}
sqlite3Fts5ClearLocale(pConfig);
}
}
}
@@ -444,11 +582,60 @@ static int fts5StorageDeleteFromIndex(
p->nTotalRow--;
}
rc2 = sqlite3_reset(pSeek);
if( rc==SQLITE_OK ) rc = rc2;
if( rc==SQLITE_OK && bSaveRow ){
assert( p->pSavedRow==0 );
p->pSavedRow = pSeek;
}else{
rc2 = sqlite3_reset(pSeek);
if( rc==SQLITE_OK ) rc = rc2;
}
return rc;
}
/*
** Reset any saved statement pSavedRow. Zero pSavedRow as well. This
** should be called by the xUpdate() method of the fts5 table before
** returning from any operation that may have set Fts5Storage.pSavedRow.
*/
void sqlite3Fts5StorageReleaseDeleteRow(Fts5Storage *pStorage){
assert( pStorage->pSavedRow==0
|| pStorage->pSavedRow==pStorage->aStmt[FTS5_STMT_LOOKUP2]
);
sqlite3_reset(pStorage->pSavedRow);
pStorage->pSavedRow = 0;
}
/*
** This function is called to process a DELETE on a contentless_delete=1
** table. It adds the tombstone required to delete the entry with rowid
** iDel. If successful, SQLITE_OK is returned. Or, if an error occurs,
** an SQLite error code.
*/
static int fts5StorageContentlessDelete(Fts5Storage *p, i64 iDel){
i64 iOrigin = 0;
sqlite3_stmt *pLookup = 0;
int rc = SQLITE_OK;
assert( p->pConfig->bContentlessDelete );
assert( p->pConfig->eContent==FTS5_CONTENT_NONE );
/* Look up the origin of the document in the %_docsize table. Store
** this in stack variable iOrigin. */
rc = fts5StorageGetStmt(p, FTS5_STMT_LOOKUP_DOCSIZE, &pLookup, 0);
if( rc==SQLITE_OK ){
sqlite3_bind_int64(pLookup, 1, iDel);
if( SQLITE_ROW==sqlite3_step(pLookup) ){
iOrigin = sqlite3_column_int64(pLookup, 1);
}
rc = sqlite3_reset(pLookup);
}
if( rc==SQLITE_OK && iOrigin!=0 ){
rc = sqlite3Fts5IndexContentlessDelete(p->pIndex, iOrigin, iDel);
}
return rc;
}
/*
** Insert a record into the %_docsize table. Specifically, do:
@@ -469,6 +656,13 @@ static int fts5StorageInsertDocsize(
rc = fts5StorageGetStmt(p, FTS5_STMT_REPLACE_DOCSIZE, &pReplace, 0);
if( rc==SQLITE_OK ){
sqlite3_bind_int64(pReplace, 1, iRowid);
if( p->pConfig->bContentlessDelete ){
i64 iOrigin = 0;
rc = sqlite3Fts5IndexGetOrigin(p->pIndex, &iOrigin);
sqlite3_bind_int64(pReplace, 3, iOrigin);
}
}
if( rc==SQLITE_OK ){
sqlite3_bind_blob(pReplace, 2, pBuf->p, pBuf->n, SQLITE_STATIC);
sqlite3_step(pReplace);
rc = sqlite3_reset(pReplace);
@@ -526,7 +720,12 @@ static int fts5StorageSaveTotals(Fts5Storage *p){
/*
** Remove a row from the FTS table.
*/
int sqlite3Fts5StorageDelete(Fts5Storage *p, i64 iDel, sqlite3_value **apVal){
int sqlite3Fts5StorageDelete(
Fts5Storage *p, /* Storage object */
i64 iDel, /* Rowid to delete from table */
sqlite3_value **apVal, /* Optional - values to remove from index */
int bSaveRow /* If true, set pSavedRow for deleted row */
){
Fts5Config *pConfig = p->pConfig;
int rc;
sqlite3_stmt *pDel = 0;
@@ -536,7 +735,15 @@ int sqlite3Fts5StorageDelete(Fts5Storage *p, i64 iDel, sqlite3_value **apVal){
/* Delete the index records */
if( rc==SQLITE_OK ){
rc = fts5StorageDeleteFromIndex(p, iDel, apVal);
rc = sqlite3Fts5IndexBeginWrite(p->pIndex, 1, iDel);
}
if( rc==SQLITE_OK ){
if( p->pConfig->bContentlessDelete ){
rc = fts5StorageContentlessDelete(p, iDel);
}else{
rc = fts5StorageDeleteFromIndex(p, iDel, apVal, bSaveRow);
}
}
/* Delete the %_docsize record */
@@ -613,7 +820,7 @@ int sqlite3Fts5StorageRebuild(Fts5Storage *p){
}
if( rc==SQLITE_OK ){
rc = fts5StorageGetStmt(p, FTS5_STMT_SCAN, &pScan, 0);
rc = fts5StorageGetStmt(p, FTS5_STMT_SCAN, &pScan, pConfig->pzErrmsg);
}
while( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pScan) ){
@@ -624,14 +831,36 @@ int sqlite3Fts5StorageRebuild(Fts5Storage *p){
for(ctx.iCol=0; rc==SQLITE_OK && ctx.iCol<pConfig->nCol; ctx.iCol++){
ctx.szCol = 0;
if( pConfig->abUnindexed[ctx.iCol]==0 ){
const char *zText = (const char*)sqlite3_column_text(pScan, ctx.iCol+1);
int nText = sqlite3_column_bytes(pScan, ctx.iCol+1);
rc = sqlite3Fts5Tokenize(pConfig,
FTS5_TOKENIZE_DOCUMENT,
zText, nText,
(void*)&ctx,
fts5StorageInsertCallback
);
int nText = 0; /* Size of pText in bytes */
const char *pText = 0; /* Pointer to buffer containing text value */
int nLoc = 0; /* Size of pLoc in bytes */
const char *pLoc = 0; /* Pointer to buffer containing text value */
sqlite3_value *pVal = sqlite3_column_value(pScan, ctx.iCol+1);
if( pConfig->eContent==FTS5_CONTENT_EXTERNAL
&& sqlite3Fts5IsLocaleValue(pConfig, pVal)
){
rc = sqlite3Fts5DecodeLocaleValue(pVal, &pText, &nText, &pLoc, &nLoc);
}else{
pText = (const char*)sqlite3_value_text(pVal);
nText = sqlite3_value_bytes(pVal);
if( pConfig->bLocale ){
int iCol = ctx.iCol + 1 + pConfig->nCol;
pLoc = (const char*)sqlite3_column_text(pScan, iCol);
nLoc = sqlite3_column_bytes(pScan, iCol);
}
}
if( rc==SQLITE_OK ){
sqlite3Fts5SetLocale(pConfig, pLoc, nLoc);
rc = sqlite3Fts5Tokenize(pConfig,
FTS5_TOKENIZE_DOCUMENT,
pText, nText,
(void*)&ctx,
fts5StorageInsertCallback
);
sqlite3Fts5ClearLocale(pConfig);
}
}
sqlite3Fts5BufferAppendVarint(&rc, &buf, ctx.szCol);
p->aTotalSize[ctx.iCol] += (i64)ctx.szCol;
@@ -713,9 +942,49 @@ int sqlite3Fts5StorageContentInsert(
}else{
sqlite3_stmt *pInsert = 0; /* Statement to write %_content table */
int i; /* Counter variable */
int nIndexed = 0; /* Number indexed columns seen */
rc = fts5StorageGetStmt(p, FTS5_STMT_INSERT_CONTENT, &pInsert, 0);
for(i=1; rc==SQLITE_OK && i<=pConfig->nCol+1; i++){
rc = sqlite3_bind_value(pInsert, i, apVal[i]);
if( pInsert ) sqlite3_clear_bindings(pInsert);
/* Bind the rowid value */
sqlite3_bind_value(pInsert, 1, apVal[1]);
/* Loop through values for user-defined columns. i=2 is the leftmost
** user-defined column. As is column 1 of pSavedRow. */
for(i=2; rc==SQLITE_OK && i<=pConfig->nCol+1; i++){
int bUnindexed = pConfig->abUnindexed[i-2];
sqlite3_value *pVal = apVal[i];
nIndexed += !bUnindexed;
if( sqlite3_value_nochange(pVal) && p->pSavedRow ){
/* This is an UPDATE statement, and column (i-2) was not modified.
** Retrieve the value from Fts5Storage.pSavedRow instead. */
pVal = sqlite3_column_value(p->pSavedRow, i-1);
if( pConfig->bLocale && bUnindexed==0 ){
sqlite3_bind_value(pInsert, pConfig->nCol + 1 + nIndexed,
sqlite3_column_value(p->pSavedRow, pConfig->nCol + i - 1)
);
}
}else if( sqlite3Fts5IsLocaleValue(pConfig, pVal) ){
const char *pText = 0;
const char *pLoc = 0;
int nText = 0;
int nLoc = 0;
assert( pConfig->bLocale );
rc = sqlite3Fts5DecodeLocaleValue(pVal, &pText, &nText, &pLoc, &nLoc);
if( rc==SQLITE_OK ){
sqlite3_bind_text(pInsert, i, pText, nText, SQLITE_TRANSIENT);
if( bUnindexed==0 ){
int iLoc = pConfig->nCol + 1 + nIndexed;
sqlite3_bind_text(pInsert, iLoc, pLoc, nLoc, SQLITE_TRANSIENT);
}
}
continue;
}
rc = sqlite3_bind_value(pInsert, i, pVal);
}
if( rc==SQLITE_OK ){
sqlite3_step(pInsert);
@@ -750,14 +1019,38 @@ int sqlite3Fts5StorageIndexInsert(
for(ctx.iCol=0; rc==SQLITE_OK && ctx.iCol<pConfig->nCol; ctx.iCol++){
ctx.szCol = 0;
if( pConfig->abUnindexed[ctx.iCol]==0 ){
const char *zText = (const char*)sqlite3_value_text(apVal[ctx.iCol+2]);
int nText = sqlite3_value_bytes(apVal[ctx.iCol+2]);
rc = sqlite3Fts5Tokenize(pConfig,
FTS5_TOKENIZE_DOCUMENT,
zText, nText,
(void*)&ctx,
fts5StorageInsertCallback
);
int nText = 0; /* Size of pText in bytes */
const char *pText = 0; /* Pointer to buffer containing text value */
int nLoc = 0; /* Size of pText in bytes */
const char *pLoc = 0; /* Pointer to buffer containing text value */
sqlite3_value *pVal = apVal[ctx.iCol+2];
if( p->pSavedRow && sqlite3_value_nochange(pVal) ){
pVal = sqlite3_column_value(p->pSavedRow, ctx.iCol+1);
if( pConfig->eContent==FTS5_CONTENT_NORMAL && pConfig->bLocale ){
int iCol = ctx.iCol + 1 + pConfig->nCol;
pLoc = (const char*)sqlite3_column_text(p->pSavedRow, iCol);
nLoc = sqlite3_column_bytes(p->pSavedRow, iCol);
}
}else{
pVal = apVal[ctx.iCol+2];
}
if( pConfig->bLocale && sqlite3Fts5IsLocaleValue(pConfig, pVal) ){
rc = sqlite3Fts5DecodeLocaleValue(pVal, &pText, &nText, &pLoc, &nLoc);
}else{
pText = (const char*)sqlite3_value_text(pVal);
nText = sqlite3_value_bytes(pVal);
}
if( rc==SQLITE_OK ){
sqlite3Fts5SetLocale(pConfig, pLoc, nLoc);
rc = sqlite3Fts5Tokenize(pConfig,
FTS5_TOKENIZE_DOCUMENT, pText, nText, (void*)&ctx,
fts5StorageInsertCallback
);
sqlite3Fts5ClearLocale(pConfig);
}
}
sqlite3Fts5BufferAppendVarint(&rc, &buf, ctx.szCol);
p->aTotalSize[ctx.iCol] += (i64)ctx.szCol;
@@ -921,29 +1214,61 @@ int sqlite3Fts5StorageIntegrity(Fts5Storage *p, int iArg){
rc = sqlite3Fts5TermsetNew(&ctx.pTermset);
}
for(i=0; rc==SQLITE_OK && i<pConfig->nCol; i++){
if( pConfig->abUnindexed[i] ) continue;
ctx.iCol = i;
ctx.szCol = 0;
if( pConfig->eDetail==FTS5_DETAIL_COLUMNS ){
rc = sqlite3Fts5TermsetNew(&ctx.pTermset);
}
if( rc==SQLITE_OK ){
const char *zText = (const char*)sqlite3_column_text(pScan, i+1);
int nText = sqlite3_column_bytes(pScan, i+1);
rc = sqlite3Fts5Tokenize(pConfig,
FTS5_TOKENIZE_DOCUMENT,
zText, nText,
(void*)&ctx,
fts5StorageIntegrityCallback
);
}
if( rc==SQLITE_OK && pConfig->bColumnsize && ctx.szCol!=aColSize[i] ){
rc = FTS5_CORRUPT;
}
aTotalSize[i] += ctx.szCol;
if( pConfig->eDetail==FTS5_DETAIL_COLUMNS ){
sqlite3Fts5TermsetFree(ctx.pTermset);
ctx.pTermset = 0;
if( pConfig->abUnindexed[i]==0 ){
const char *pText = 0;
int nText = 0;
const char *pLoc = 0;
int nLoc = 0;
sqlite3_value *pVal = sqlite3_column_value(pScan, i+1);
if( pConfig->eContent==FTS5_CONTENT_EXTERNAL
&& sqlite3Fts5IsLocaleValue(pConfig, pVal)
){
rc = sqlite3Fts5DecodeLocaleValue(
pVal, &pText, &nText, &pLoc, &nLoc
);
}else{
if( pConfig->eContent==FTS5_CONTENT_NORMAL && pConfig->bLocale ){
int iCol = i + 1 + pConfig->nCol;
pLoc = (const char*)sqlite3_column_text(pScan, iCol);
nLoc = sqlite3_column_bytes(pScan, iCol);
}
pText = (const char*)sqlite3_value_text(pVal);
nText = sqlite3_value_bytes(pVal);
}
ctx.iCol = i;
ctx.szCol = 0;
if( rc==SQLITE_OK && pConfig->eDetail==FTS5_DETAIL_COLUMNS ){
rc = sqlite3Fts5TermsetNew(&ctx.pTermset);
}
if( rc==SQLITE_OK ){
sqlite3Fts5SetLocale(pConfig, pLoc, nLoc);
rc = sqlite3Fts5Tokenize(pConfig,
FTS5_TOKENIZE_DOCUMENT,
pText, nText,
(void*)&ctx,
fts5StorageIntegrityCallback
);
sqlite3Fts5ClearLocale(pConfig);
}
/* If this is not a columnsize=0 database, check that the number
** of tokens in the value matches the aColSize[] value read from
** the %_docsize table. */
if( rc==SQLITE_OK
&& pConfig->bColumnsize
&& ctx.szCol!=aColSize[i]
){
rc = FTS5_CORRUPT;
}
aTotalSize[i] += ctx.szCol;
if( pConfig->eDetail==FTS5_DETAIL_COLUMNS ){
sqlite3Fts5TermsetFree(ctx.pTermset);
ctx.pTermset = 0;
}
}
}
sqlite3Fts5TermsetFree(ctx.pTermset);
@@ -1124,7 +1449,9 @@ int sqlite3Fts5StorageSync(Fts5Storage *p){
i64 iLastRowid = sqlite3_last_insert_rowid(p->pConfig->db);
if( p->bTotalsValid ){
rc = fts5StorageSaveTotals(p);
p->bTotalsValid = 0;
if( rc==SQLITE_OK ){
p->bTotalsValid = 0;
}
}
if( rc==SQLITE_OK ){
rc = sqlite3Fts5IndexSync(p->pIndex);
+565 -53
View File
@@ -14,14 +14,7 @@
#ifdef SQLITE_TEST
#if defined(INCLUDE_SQLITE_TCL_H)
# include "sqlite_tcl.h"
#else
# include "tcl.h"
# ifndef SQLITE_TCLAPI
# define SQLITE_TCLAPI
# endif
#endif
#include "tclsqlite.h"
#ifdef SQLITE_ENABLE_FTS5
@@ -103,14 +96,14 @@ static int SQLITE_TCLAPI f5tDbAndApi(
rc = sqlite3_prepare_v2(db, "SELECT fts5(?1)", -1, &pStmt, 0);
if( rc!=SQLITE_OK ){
Tcl_AppendResult(interp, "error: ", sqlite3_errmsg(db), 0);
Tcl_AppendResult(interp, "error: ", sqlite3_errmsg(db), (char*)0);
return TCL_ERROR;
}
sqlite3_bind_pointer(pStmt, 1, (void*)&pApi, "fts5_api_ptr", 0);
sqlite3_step(pStmt);
if( sqlite3_finalize(pStmt)!=SQLITE_OK ){
Tcl_AppendResult(interp, "error: ", sqlite3_errmsg(db), 0);
Tcl_AppendResult(interp, "error: ", sqlite3_errmsg(db), (char*)0);
return TCL_ERROR;
}
@@ -244,6 +237,10 @@ static int SQLITE_TCLAPI xF5tApi(
{ "xGetAuxdataInt", 1, "CLEAR" }, /* 15 */
{ "xPhraseForeach", 4, "IPHRASE COLVAR OFFVAR SCRIPT" }, /* 16 */
{ "xPhraseColumnForeach", 3, "IPHRASE COLVAR SCRIPT" }, /* 17 */
{ "xQueryToken", 2, "IPHRASE ITERM" }, /* 18 */
{ "xInstToken", 2, "IDX ITERM" }, /* 19 */
{ "xColumnLocale", 1, "COL" }, /* 20 */
{ 0, 0, 0}
};
@@ -294,12 +291,12 @@ static int SQLITE_TCLAPI xF5tApi(
break;
}
CASE(3, "xTokenize") {
int nText;
Tcl_Size nText;
char *zText = Tcl_GetStringFromObj(objv[2], &nText);
F5tFunction ctx;
ctx.interp = interp;
ctx.pScript = objv[3];
rc = p->pApi->xTokenize(p->pFts, zText, nText, &ctx, xTokenizeCb);
rc = p->pApi->xTokenize(p->pFts, zText, (int)nText, &ctx, xTokenizeCb);
if( rc==SQLITE_OK ){
Tcl_ResetResult(interp);
}
@@ -395,7 +392,7 @@ static int SQLITE_TCLAPI xF5tApi(
CASE(12, "xSetAuxdata") {
F5tAuxData *pData = (F5tAuxData*)sqlite3_malloc(sizeof(F5tAuxData));
if( pData==0 ){
Tcl_AppendResult(interp, "out of memory", 0);
Tcl_AppendResult(interp, "out of memory", (char*)0);
return TCL_ERROR;
}
pData->pObj = objv[2];
@@ -455,7 +452,7 @@ static int SQLITE_TCLAPI xF5tApi(
rc = p->pApi->xPhraseFirst(p->pFts, iPhrase, &iter, &iCol, &iOff);
if( rc!=SQLITE_OK ){
Tcl_AppendResult(interp, sqlite3ErrName(rc), 0);
Tcl_AppendResult(interp, sqlite3ErrName(rc), (char*)0);
return TCL_ERROR;
}
for( ;iCol>=0; p->pApi->xPhraseNext(p->pFts, &iter, &iCol, &iOff) ){
@@ -500,6 +497,52 @@ static int SQLITE_TCLAPI xF5tApi(
break;
}
CASE(18, "xQueryToken") {
const char *pTerm = 0;
int nTerm = 0;
int iPhrase = 0;
int iTerm = 0;
if( Tcl_GetIntFromObj(interp, objv[2], &iPhrase) ) return TCL_ERROR;
if( Tcl_GetIntFromObj(interp, objv[3], &iTerm) ) return TCL_ERROR;
rc = p->pApi->xQueryToken(p->pFts, iPhrase, iTerm, &pTerm, &nTerm);
if( rc==SQLITE_OK ){
Tcl_SetObjResult(interp, Tcl_NewStringObj(pTerm, nTerm));
}
break;
}
CASE(19, "xInstToken") {
const char *pTerm = 0;
int nTerm = 0;
int iIdx = 0;
int iTerm = 0;
if( Tcl_GetIntFromObj(interp, objv[2], &iIdx) ) return TCL_ERROR;
if( Tcl_GetIntFromObj(interp, objv[3], &iTerm) ) return TCL_ERROR;
rc = p->pApi->xInstToken(p->pFts, iIdx, iTerm, &pTerm, &nTerm);
if( rc==SQLITE_OK ){
Tcl_SetObjResult(interp, Tcl_NewStringObj(pTerm, nTerm));
}
break;
}
CASE(20, "xColumnLocale") {
const char *z = 0;
int n = 0;
int iCol;
if( Tcl_GetIntFromObj(interp, objv[2], &iCol) ){
return TCL_ERROR;
}
rc = p->pApi->xColumnLocale(p->pFts, iCol, &z, &n);
if( rc==SQLITE_OK && z ){
Tcl_SetObjResult(interp, Tcl_NewStringObj(z, n));
}
break;
}
default:
assert( 0 );
break;
@@ -570,15 +613,16 @@ static void xF5tFunction(
sqlite3_result_error(pCtx, Tcl_GetStringResult(p->interp), -1);
}else{
Tcl_Obj *pVar = Tcl_GetObjResult(p->interp);
int n;
const char *zType = (pVar->typePtr ? pVar->typePtr->name : "");
char c = zType[0];
if( c=='b' && strcmp(zType,"bytearray")==0 && pVar->bytes==0 ){
/* Only return a BLOB type if the Tcl variable is a bytearray and
** has no string representation. */
unsigned char *data = Tcl_GetByteArrayFromObj(pVar, &n);
sqlite3_result_blob(pCtx, data, n, SQLITE_TRANSIENT);
Tcl_Size nn;
unsigned char *data = Tcl_GetByteArrayFromObj(pVar, &nn);
sqlite3_result_blob(pCtx, data, (int)nn, SQLITE_TRANSIENT);
}else if( c=='b' && strcmp(zType,"boolean")==0 ){
int n;
Tcl_GetIntFromObj(0, pVar, &n);
sqlite3_result_int(pCtx, n);
}else if( c=='d' && strcmp(zType,"double")==0 ){
@@ -591,8 +635,9 @@ static void xF5tFunction(
Tcl_GetWideIntFromObj(0, pVar, &v);
sqlite3_result_int64(pCtx, v);
}else{
unsigned char *data = (unsigned char *)Tcl_GetStringFromObj(pVar, &n);
sqlite3_result_text(pCtx, (char *)data, n, SQLITE_TRANSIENT);
Tcl_Size nn;
unsigned char *data = (unsigned char *)Tcl_GetStringFromObj(pVar, &nn);
sqlite3_result_text(pCtx, (char*)data, (int)nn, SQLITE_TRANSIENT);
}
}
}
@@ -638,7 +683,7 @@ static int SQLITE_TCLAPI f5tCreateFunction(
pApi, zName, (void*)pCtx, xF5tFunction, xF5tDestroy
);
if( rc!=SQLITE_OK ){
Tcl_AppendResult(interp, "error: ", sqlite3_errmsg(db), 0);
Tcl_AppendResult(interp, "error: ", sqlite3_errmsg(db), (char*)0);
return TCL_ERROR;
}
@@ -685,7 +730,7 @@ static int SQLITE_TCLAPI f5tTokenize(
Tcl_Obj *CONST objv[]
){
char *zText;
int nText;
Tcl_Size nText;
sqlite3 *db = 0;
fts5_api *pApi = 0;
Fts5Tokenizer *pTok = 0;
@@ -694,7 +739,7 @@ static int SQLITE_TCLAPI f5tTokenize(
void *pUserdata;
int rc;
int nArg;
Tcl_Size nArg;
const char **azArg;
F5tTokenizeCtx ctx;
@@ -705,7 +750,7 @@ static int SQLITE_TCLAPI f5tTokenize(
if( objc==5 ){
char *zOpt = Tcl_GetString(objv[1]);
if( strcmp("-subst", zOpt) ){
Tcl_AppendResult(interp, "unrecognized option: ", zOpt, 0);
Tcl_AppendResult(interp, "unrecognized option: ", zOpt, (char*)0);
return TCL_ERROR;
}
}
@@ -714,7 +759,7 @@ static int SQLITE_TCLAPI f5tTokenize(
return TCL_ERROR;
}
if( nArg==0 ){
Tcl_AppendResult(interp, "no such tokenizer: ", 0);
Tcl_AppendResult(interp, "no such tokenizer: ", (char*)0);
Tcl_Free((void*)azArg);
return TCL_ERROR;
}
@@ -722,13 +767,13 @@ static int SQLITE_TCLAPI f5tTokenize(
rc = pApi->xFindTokenizer(pApi, azArg[0], &pUserdata, &tokenizer);
if( rc!=SQLITE_OK ){
Tcl_AppendResult(interp, "no such tokenizer: ", azArg[0], 0);
Tcl_AppendResult(interp, "no such tokenizer: ", azArg[0], (char*)0);
return TCL_ERROR;
}
rc = tokenizer.xCreate(pUserdata, &azArg[1], nArg-1, &pTok);
rc = tokenizer.xCreate(pUserdata, &azArg[1], (int)(nArg-1), &pTok);
if( rc!=SQLITE_OK ){
Tcl_AppendResult(interp, "error in tokenizer.xCreate()", 0);
Tcl_AppendResult(interp, "error in tokenizer.xCreate()", (char*)0);
return TCL_ERROR;
}
@@ -738,11 +783,11 @@ static int SQLITE_TCLAPI f5tTokenize(
ctx.pRet = pRet;
ctx.zInput = zText;
rc = tokenizer.xTokenize(
pTok, (void*)&ctx, FTS5_TOKENIZE_DOCUMENT, zText, nText, xTokenizeCb2
pTok, (void*)&ctx, FTS5_TOKENIZE_DOCUMENT, zText,(int)nText, xTokenizeCb2
);
tokenizer.xDelete(pTok);
if( rc!=SQLITE_OK ){
Tcl_AppendResult(interp, "error in tokenizer.xTokenize()", 0);
Tcl_AppendResult(interp, "error in tokenizer.xTokenize()", (char*)0);
Tcl_DecrRefCount(pRet);
return TCL_ERROR;
}
@@ -766,18 +811,32 @@ typedef struct F5tTokenizerInstance F5tTokenizerInstance;
struct F5tTokenizerContext {
void *pCtx;
int (*xToken)(void*, int, const char*, int, int, int);
F5tTokenizerInstance *pInst;
};
struct F5tTokenizerModule {
Tcl_Interp *interp;
Tcl_Obj *pScript;
void *pParentCtx;
fts5_tokenizer_v2 parent_v2;
fts5_tokenizer parent;
F5tTokenizerContext *pContext;
};
/*
** zLocale:
** Within a call to xTokenize_v2(), pLocale/nLocale store the locale
** passed to the call by fts5. This can be retrieved by a Tcl tokenize
** script using [sqlite3_fts5_locale].
*/
struct F5tTokenizerInstance {
Tcl_Interp *interp;
Tcl_Obj *pScript;
F5tTokenizerModule *pModule;
Fts5Tokenizer *pParent;
F5tTokenizerContext *pContext;
const char *pLocale;
int nLocale;
};
static int f5tTokenizerCreate(
@@ -786,11 +845,20 @@ static int f5tTokenizerCreate(
int nArg,
Fts5Tokenizer **ppOut
){
Fts5Tokenizer *pParent = 0;
F5tTokenizerModule *pMod = (F5tTokenizerModule*)pCtx;
Tcl_Obj *pEval;
int rc = TCL_OK;
int i;
assert( pMod->parent_v2.xCreate==0 || pMod->parent.xCreate==0 );
if( pMod->parent_v2.xCreate ){
rc = pMod->parent_v2.xCreate(pMod->pParentCtx, 0, 0, &pParent);
}
if( pMod->parent.xCreate ){
rc = pMod->parent.xCreate(pMod->pParentCtx, 0, 0, &pParent);
}
pEval = Tcl_DuplicateObj(pMod->pScript);
Tcl_IncrRefCount(pEval);
for(i=0; rc==TCL_OK && i<nArg; i++){
@@ -810,6 +878,8 @@ static int f5tTokenizerCreate(
pInst->interp = pMod->interp;
pInst->pScript = Tcl_GetObjResult(pMod->interp);
pInst->pContext = pMod->pContext;
pInst->pParent = pParent;
pInst->pModule = pMod;
Tcl_IncrRefCount(pInst->pScript);
*ppOut = (Fts5Tokenizer*)pInst;
}
@@ -820,11 +890,21 @@ static int f5tTokenizerCreate(
static void f5tTokenizerDelete(Fts5Tokenizer *p){
F5tTokenizerInstance *pInst = (F5tTokenizerInstance*)p;
Tcl_DecrRefCount(pInst->pScript);
ckfree((char *)pInst);
if( pInst ){
if( pInst->pParent ){
if( pInst->pModule->parent_v2.xDelete ){
pInst->pModule->parent_v2.xDelete(pInst->pParent);
}else{
pInst->pModule->parent.xDelete(pInst->pParent);
}
}
Tcl_DecrRefCount(pInst->pScript);
ckfree((char *)pInst);
}
}
static int f5tTokenizerTokenize(
static int f5tTokenizerReallyTokenize(
Fts5Tokenizer *p,
void *pCtx,
int flags,
@@ -832,6 +912,7 @@ static int f5tTokenizerTokenize(
int (*xToken)(void*, int, const char*, int, int, int)
){
F5tTokenizerInstance *pInst = (F5tTokenizerInstance*)p;
F5tTokenizerInstance *pOldInst = 0;
void *pOldCtx;
int (*xOldToken)(void*, int, const char*, int, int, int);
Tcl_Obj *pEval;
@@ -840,9 +921,11 @@ static int f5tTokenizerTokenize(
pOldCtx = pInst->pContext->pCtx;
xOldToken = pInst->pContext->xToken;
pOldInst = pInst->pContext->pInst;
pInst->pContext->pCtx = pCtx;
pInst->pContext->xToken = xToken;
pInst->pContext->pInst = pInst;
assert(
flags==FTS5_TOKENIZE_DOCUMENT
@@ -878,9 +961,105 @@ static int f5tTokenizerTokenize(
pInst->pContext->pCtx = pOldCtx;
pInst->pContext->xToken = xOldToken;
pInst->pContext->pInst = pOldInst;
return rc;
}
typedef struct CallbackCtx CallbackCtx;
struct CallbackCtx {
Fts5Tokenizer *p;
void *pCtx;
int flags;
int (*xToken)(void*, int, const char*, int, int, int);
};
static int f5tTokenizeCallback(
void *pCtx,
int tflags,
const char *z, int n,
int iStart, int iEnd
){
CallbackCtx *p = (CallbackCtx*)pCtx;
return f5tTokenizerReallyTokenize(p->p, p->pCtx, p->flags, z, n, p->xToken);
}
static int f5tTokenizerTokenize_v2(
Fts5Tokenizer *p,
void *pCtx,
int flags,
const char *pText, int nText,
const char *pLoc, int nLoc,
int (*xToken)(void*, int, const char*, int, int, int)
){
int rc = SQLITE_OK;
F5tTokenizerInstance *pInst = (F5tTokenizerInstance*)p;
pInst->pLocale = pLoc;
pInst->nLocale = nLoc;
if( pInst->pParent ){
CallbackCtx ctx;
ctx.p = p;
ctx.pCtx = pCtx;
ctx.flags = flags;
ctx.xToken = xToken;
if( pInst->pModule->parent_v2.xTokenize ){
rc = pInst->pModule->parent_v2.xTokenize(
pInst->pParent, (void*)&ctx, flags, pText, nText,
pLoc, nLoc, f5tTokenizeCallback
);
}else{
rc = pInst->pModule->parent.xTokenize(
pInst->pParent, (void*)&ctx, flags, pText, nText, f5tTokenizeCallback
);
}
}else{
rc = f5tTokenizerReallyTokenize(p, pCtx, flags, pText, nText, xToken);
}
pInst->pLocale = 0;
pInst->nLocale = 0;
return rc;
}
static int f5tTokenizerTokenize(
Fts5Tokenizer *p,
void *pCtx,
int flags,
const char *pText, int nText,
int (*xToken)(void*, int, const char*, int, int, int)
){
return f5tTokenizerTokenize_v2(p, pCtx, flags, pText, nText, 0, 0, xToken);
}
/*
** sqlite3_fts5_locale
*/
static int SQLITE_TCLAPI f5tTokenizerLocale(
void * clientData,
Tcl_Interp *interp,
int objc,
Tcl_Obj *CONST objv[]
){
F5tTokenizerContext *p = (F5tTokenizerContext*)clientData;
if( objc!=1 ){
Tcl_WrongNumArgs(interp, 1, objv, "");
return TCL_ERROR;
}
if( p->xToken==0 ){
Tcl_AppendResult(interp,
"sqlite3_fts5_locale may only be used by tokenizer callback", (char*)0
);
return TCL_ERROR;
}
Tcl_SetObjResult(interp,
Tcl_NewStringObj(p->pInst->pLocale, p->pInst->nLocale)
);
return TCL_OK;
}
/*
** sqlite3_fts5_token ?-colocated? TEXT START END
*/
@@ -893,13 +1072,13 @@ static int SQLITE_TCLAPI f5tTokenizerReturn(
F5tTokenizerContext *p = (F5tTokenizerContext*)clientData;
int iStart;
int iEnd;
int nToken;
Tcl_Size nToken;
int tflags = 0;
char *zToken;
int rc;
if( objc==5 ){
int nArg;
Tcl_Size nArg;
char *zArg = Tcl_GetStringFromObj(objv[1], &nArg);
if( nArg<=10 && nArg>=2 && memcmp("-colocated", zArg, nArg)==0 ){
tflags |= FTS5_TOKEN_COLOCATED;
@@ -919,12 +1098,12 @@ static int SQLITE_TCLAPI f5tTokenizerReturn(
if( p->xToken==0 ){
Tcl_AppendResult(interp,
"sqlite3_fts5_token may only be used by tokenizer callback", 0
"sqlite3_fts5_token may only be used by tokenizer callback", (char*)0
);
return TCL_ERROR;
}
rc = p->xToken(p->pCtx, tflags, zToken, nToken, iStart, iEnd);
rc = p->xToken(p->pCtx, tflags, zToken, (int)nToken, iStart, iEnd);
Tcl_SetResult(interp, (char*)sqlite3ErrName(rc), TCL_VOLATILE);
return rc==SQLITE_OK ? TCL_OK : TCL_ERROR;
@@ -966,32 +1145,112 @@ static int SQLITE_TCLAPI f5tCreateTokenizer(
fts5_api *pApi;
char *zName;
Tcl_Obj *pScript;
fts5_tokenizer t;
F5tTokenizerModule *pMod;
int rc;
int rc = SQLITE_OK;
int bV2 = 0; /* True to use _v2 API */
int iVersion = 2; /* Value for _v2.iVersion */
const char *zParent = 0; /* Name of parent tokenizer, if any */
int ii = 0;
if( objc!=4 ){
Tcl_WrongNumArgs(interp, 1, objv, "DB NAME SCRIPT");
if( objc<4 ){
Tcl_WrongNumArgs(interp, 1, objv, "?OPTIONS? DB NAME SCRIPT");
return TCL_ERROR;
}
if( f5tDbAndApi(interp, objv[1], &db, &pApi) ){
/* Parse any options. Set stack variables bV2 and zParent. */
for(ii=1; ii<objc-3; ii++){
int iOpt = 0;
const char *azOpt[] = { "-v2", "-parent", "-version", 0 };
if( Tcl_GetIndexFromObj(interp, objv[ii], azOpt, "OPTION", 0, &iOpt) ){
return TCL_ERROR;
}
switch( iOpt ){
case 0: /* -v2 */ {
bV2 = 1;
break;
}
case 1: /* -parent */ {
ii++;
if( ii==objc-3 ){
Tcl_AppendResult(
interp, "option requires an argument: -parent", (char*)0
);
return TCL_ERROR;
}
zParent = Tcl_GetString(objv[ii]);
break;
}
case 2: /* -version */ {
ii++;
if( ii==objc-3 ){
Tcl_AppendResult(
interp, "option requires an argument: -version", (char*)0
);
return TCL_ERROR;
}
if( Tcl_GetIntFromObj(interp, objv[ii], &iVersion) ){
return TCL_ERROR;
}
break;
}
default:
assert( 0 );
break;
}
}
if( f5tDbAndApi(interp, objv[objc-3], &db, &pApi) ){
return TCL_ERROR;
}
zName = Tcl_GetString(objv[2]);
pScript = objv[3];
t.xCreate = f5tTokenizerCreate;
t.xTokenize = f5tTokenizerTokenize;
t.xDelete = f5tTokenizerDelete;
zName = Tcl_GetString(objv[objc-2]);
pScript = objv[objc-1];
pMod = (F5tTokenizerModule*)ckalloc(sizeof(F5tTokenizerModule));
memset(pMod, 0, sizeof(F5tTokenizerModule));
pMod->interp = interp;
pMod->pScript = pScript;
pMod->pContext = pContext;
Tcl_IncrRefCount(pScript);
rc = pApi->xCreateTokenizer(pApi, zName, (void*)pMod, &t, f5tDelTokenizer);
pMod->pContext = pContext;
if( zParent ){
if( bV2 ){
fts5_tokenizer_v2 *pParent = 0;
rc = pApi->xFindTokenizer_v2(pApi, zParent, &pMod->pParentCtx, &pParent);
if( rc==SQLITE_OK ){
memcpy(&pMod->parent_v2, pParent, sizeof(fts5_tokenizer_v2));
pMod->parent_v2.xDelete(0);
}
}else{
rc = pApi->xFindTokenizer(pApi, zParent, &pMod->pParentCtx,&pMod->parent);
if( rc==SQLITE_OK ){
pMod->parent.xDelete(0);
}
}
}
if( rc==SQLITE_OK ){
void *pModCtx = (void*)pMod;
if( bV2==0 ){
fts5_tokenizer t;
t.xCreate = f5tTokenizerCreate;
t.xTokenize = f5tTokenizerTokenize;
t.xDelete = f5tTokenizerDelete;
rc = pApi->xCreateTokenizer(pApi, zName, pModCtx, &t, f5tDelTokenizer);
}else{
fts5_tokenizer_v2 t2;
memset(&t2, 0, sizeof(t2));
t2.iVersion = iVersion;
t2.xCreate = f5tTokenizerCreate;
t2.xTokenize = f5tTokenizerTokenize_v2;
t2.xDelete = f5tTokenizerDelete;
rc = pApi->xCreateTokenizer_v2(pApi, zName, pModCtx, &t2,f5tDelTokenizer);
}
}
if( rc!=SQLITE_OK ){
Tcl_AppendResult(interp, "error in fts5_api.xCreateTokenizer()", 0);
Tcl_AppendResult(interp, (
bV2 ? "error in fts5_api.xCreateTokenizer_v2()"
: "error in fts5_api.xCreateTokenizer()"
), (char*)0);
return TCL_ERROR;
}
@@ -1048,7 +1307,7 @@ static int SQLITE_TCLAPI f5tTokenHash(
Tcl_Obj *CONST objv[]
){
char *z;
int n;
Tcl_Size n;
unsigned int iVal;
int nSlot;
@@ -1061,7 +1320,7 @@ static int SQLITE_TCLAPI f5tTokenHash(
}
z = Tcl_GetStringFromObj(objv[2], &n);
iVal = f5t_fts5HashKey(nSlot, z, n);
iVal = f5t_fts5HashKey(nSlot, z, (int)n);
Tcl_SetObjResult(interp, Tcl_NewIntObj(iVal));
return TCL_OK;
}
@@ -1117,6 +1376,256 @@ static int SQLITE_TCLAPI f5tRegisterTok(
return TCL_OK;
}
typedef struct OriginTextCtx OriginTextCtx;
struct OriginTextCtx {
sqlite3 *db;
fts5_api *pApi;
};
typedef struct OriginTextTokenizer OriginTextTokenizer;
struct OriginTextTokenizer {
Fts5Tokenizer *pTok; /* Underlying tokenizer object */
fts5_tokenizer tokapi; /* API implementation for pTok */
};
/*
** Delete the OriginTextCtx object indicated by the only argument.
*/
static void f5tOrigintextTokenizerDelete(void *pCtx){
OriginTextCtx *p = (OriginTextCtx*)pCtx;
ckfree((char*)p);
}
static int f5tOrigintextCreate(
void *pCtx,
const char **azArg,
int nArg,
Fts5Tokenizer **ppOut
){
OriginTextCtx *p = (OriginTextCtx*)pCtx;
OriginTextTokenizer *pTok = 0;
void *pTokCtx = 0;
int rc = SQLITE_OK;
pTok = (OriginTextTokenizer*)sqlite3_malloc(sizeof(OriginTextTokenizer));
if( pTok==0 ){
rc = SQLITE_NOMEM;
}else if( nArg<1 ){
rc = SQLITE_ERROR;
}else{
/* Locate the underlying tokenizer */
rc = p->pApi->xFindTokenizer(p->pApi, azArg[0], &pTokCtx, &pTok->tokapi);
}
/* Create the new tokenizer instance */
if( rc==SQLITE_OK ){
rc = pTok->tokapi.xCreate(pTokCtx, &azArg[1], nArg-1, &pTok->pTok);
}
if( rc!=SQLITE_OK ){
sqlite3_free(pTok);
pTok = 0;
}
*ppOut = (Fts5Tokenizer*)pTok;
return rc;
}
static void f5tOrigintextDelete(Fts5Tokenizer *pTokenizer){
OriginTextTokenizer *p = (OriginTextTokenizer*)pTokenizer;
if( p->pTok ){
p->tokapi.xDelete(p->pTok);
}
sqlite3_free(p);
}
typedef struct OriginTextCb OriginTextCb;
struct OriginTextCb {
void *pCtx;
const char *pText;
int nText;
int (*xToken)(void *, int, const char *, int, int, int);
char *aBuf; /* Buffer to use */
int nBuf; /* Allocated size of aBuf[] */
};
static int xOriginToken(
void *pCtx, /* Copy of 2nd argument to xTokenize() */
int tflags, /* Mask of FTS5_TOKEN_* flags */
const char *pToken, /* Pointer to buffer containing token */
int nToken, /* Size of token in bytes */
int iStart, /* Byte offset of token within input text */
int iEnd /* Byte offset of end of token within input */
){
OriginTextCb *p = (OriginTextCb*)pCtx;
int ret = 0;
if( nToken==(iEnd-iStart) && 0==memcmp(pToken, &p->pText[iStart], nToken) ){
/* Token exactly matches document text. Pass it through as is. */
ret = p->xToken(p->pCtx, tflags, pToken, nToken, iStart, iEnd);
}else{
int nReq = nToken + 1 + (iEnd-iStart);
if( nReq>p->nBuf ){
sqlite3_free(p->aBuf);
p->aBuf = sqlite3_malloc(nReq*2);
if( p->aBuf==0 ) return SQLITE_NOMEM;
p->nBuf = nReq*2;
}
memcpy(p->aBuf, pToken, nToken);
p->aBuf[nToken] = '\0';
memcpy(&p->aBuf[nToken+1], &p->pText[iStart], iEnd-iStart);
ret = p->xToken(p->pCtx, tflags, p->aBuf, nReq, iStart, iEnd);
}
return ret;
}
static int f5tOrigintextTokenize(
Fts5Tokenizer *pTokenizer,
void *pCtx,
int flags, /* Mask of FTS5_TOKENIZE_* flags */
const char *pText, int nText,
int (*xToken)(void *, int, const char *, int, int, int)
){
OriginTextTokenizer *p = (OriginTextTokenizer*)pTokenizer;
OriginTextCb cb;
int ret;
memset(&cb, 0, sizeof(cb));
cb.pCtx = pCtx;
cb.pText = pText;
cb.nText = nText;
cb.xToken = xToken;
ret = p->tokapi.xTokenize(p->pTok,(void*)&cb,flags,pText,nText,xOriginToken);
sqlite3_free(cb.aBuf);
return ret;
}
/*
** sqlite3_fts5_register_origintext DB
**
** Description...
*/
static int SQLITE_TCLAPI f5tRegisterOriginText(
void * clientData,
Tcl_Interp *interp,
int objc,
Tcl_Obj *CONST objv[]
){
sqlite3 *db = 0;
fts5_api *pApi = 0;
int rc;
fts5_tokenizer tok = {0, 0, 0};
OriginTextCtx *pCtx = 0;
if( objc!=2 ){
Tcl_WrongNumArgs(interp, 1, objv, "DB");
return TCL_ERROR;
}
if( f5tDbAndApi(interp, objv[1], &db, &pApi) ) return TCL_ERROR;
pCtx = (OriginTextCtx*)ckalloc(sizeof(OriginTextCtx));
pCtx->db = db;
pCtx->pApi = pApi;
tok.xCreate = f5tOrigintextCreate;
tok.xDelete = f5tOrigintextDelete;
tok.xTokenize = f5tOrigintextTokenize;
rc = pApi->xCreateTokenizer(
pApi, "origintext", (void*)pCtx, &tok, f5tOrigintextTokenizerDelete
);
Tcl_ResetResult(interp);
if( rc!=SQLITE_OK ){
Tcl_AppendResult(interp, "error: ", sqlite3_errmsg(db), (void*)0);
return TCL_ERROR;
}
return TCL_OK;
}
/*
** This function is used to DROP an fts5 table. It works even if the data
** structures fts5 stores within the database are corrupt, which sometimes
** prevents a straight "DROP TABLE" command from succeeding.
**
** The first parameter is the database handle to use for the DROP TABLE
** operation. The second is the name of the database to drop the fts5 table
** from (i.e. "main", "temp" or the name of an attached database). The
** third parameter is the name of the fts5 table to drop.
**
** SQLITE_OK is returned if the table is successfully dropped. Or, if an
** error occurs, an SQLite error code.
*/
static int sqlite3_fts5_drop_corrupt_table(
sqlite3 *db, /* Database handle */
const char *zDb, /* Database name ("main", "temp" etc.) */
const char *zTab /* Name of fts5 table to drop */
){
int rc = SQLITE_OK;
int bDef = 0;
rc = sqlite3_db_config(db, SQLITE_DBCONFIG_DEFENSIVE, -1, &bDef);
if( rc==SQLITE_OK ){
char *zScript = sqlite3_mprintf(
"DELETE FROM %Q.'%q_data';"
"DELETE FROM %Q.'%q_config';"
"INSERT INTO %Q.'%q_data' VALUES(10, X'0000000000');"
"INSERT INTO %Q.'%q_config' VALUES('version', 4);"
"DROP TABLE %Q.'%q';",
zDb, zTab, zDb, zTab, zDb, zTab, zDb, zTab, zDb, zTab
);
if( zScript==0 ){
rc = SQLITE_NOMEM;
}else{
if( bDef ) sqlite3_db_config(db, SQLITE_DBCONFIG_DEFENSIVE, 0, 0);
rc = sqlite3_exec(db, zScript, 0, 0, 0);
if( bDef ) sqlite3_db_config(db, SQLITE_DBCONFIG_DEFENSIVE, 1, 0);
sqlite3_free(zScript);
}
}
return rc;
}
/*
** sqlite3_fts5_drop_corrupt_table DB DATABASE TABLE
**
** Description...
*/
static int SQLITE_TCLAPI f5tDropCorruptTable(
void * clientData,
Tcl_Interp *interp,
int objc,
Tcl_Obj *CONST objv[]
){
sqlite3 *db = 0;
const char *zDb = 0;
const char *zTab = 0;
int rc = SQLITE_OK;
if( objc!=4 ){
Tcl_WrongNumArgs(interp, 1, objv, "DB DATABASE TABLE");
return TCL_ERROR;
}
if( f5tDbPointer(interp, objv[1], &db) ){
return TCL_ERROR;
}
zDb = Tcl_GetString(objv[2]);
zTab = Tcl_GetString(objv[3]);
rc = sqlite3_fts5_drop_corrupt_table(db, zDb, zTab);
if( rc!=SQLITE_OK ){
Tcl_AppendResult(interp, "error: ", sqlite3_errmsg(db), (void*)0);
return TCL_ERROR;
}
return TCL_OK;
}
/*
** Entry point.
*/
@@ -1128,12 +1637,15 @@ int Fts5tcl_Init(Tcl_Interp *interp){
} aCmd[] = {
{ "sqlite3_fts5_create_tokenizer", f5tCreateTokenizer, 1 },
{ "sqlite3_fts5_token", f5tTokenizerReturn, 1 },
{ "sqlite3_fts5_locale", f5tTokenizerLocale, 1 },
{ "sqlite3_fts5_tokenize", f5tTokenize, 0 },
{ "sqlite3_fts5_create_function", f5tCreateFunction, 0 },
{ "sqlite3_fts5_may_be_corrupt", f5tMayBeCorrupt, 0 },
{ "sqlite3_fts5_token_hash", f5tTokenHash, 0 },
{ "sqlite3_fts5_register_matchinfo", f5tRegisterMatchinfo, 0 },
{ "sqlite3_fts5_register_fts5tokenize", f5tRegisterTok, 0 }
{ "sqlite3_fts5_register_fts5tokenize", f5tRegisterTok, 0 },
{ "sqlite3_fts5_register_origintext",f5tRegisterOriginText, 0 },
{ "sqlite3_fts5_drop_corrupt_table", f5tDropCorruptTable, 0 }
};
int i;
F5tTokenizerContext *pContext;
+2 -1
View File
@@ -472,7 +472,8 @@ int sqlite3Fts5TestRegisterTok(sqlite3 *db, fts5_api *pApi){
0, /* xSavepoint */
0, /* xRelease */
0, /* xRollbackTo */
0 /* xShadowName */
0, /* xShadowName */
0 /* xIntegrity */
};
int rc; /* Return code */
+125 -53
View File
@@ -229,6 +229,12 @@ static const unsigned char sqlite3Utf8Trans1[] = {
#endif /* ifndef SQLITE_AMALGAMATION */
#define FTS5_SKIP_UTF8(zIn) { \
if( ((unsigned char)(*(zIn++)))>=0xc0 ){ \
while( (((unsigned char)*zIn) & 0xc0)==0x80 ){ zIn++; } \
} \
}
typedef struct Unicode61Tokenizer Unicode61Tokenizer;
struct Unicode61Tokenizer {
unsigned char aTokenChar[128]; /* ASCII range token characters */
@@ -380,7 +386,6 @@ static int fts5UnicodeCreate(
zCat = azArg[i+1];
}
}
if( rc==SQLITE_OK ){
rc = unicodeSetCategories(p, zCat);
}
@@ -410,7 +415,6 @@ static int fts5UnicodeCreate(
rc = SQLITE_ERROR;
}
}
}else{
rc = SQLITE_NOMEM;
}
@@ -549,7 +553,7 @@ static int fts5UnicodeTokenize(
typedef struct PorterTokenizer PorterTokenizer;
struct PorterTokenizer {
fts5_tokenizer tokenizer; /* Parent tokenizer module */
fts5_tokenizer_v2 tokenizer_v2; /* Parent tokenizer module */
Fts5Tokenizer *pTokenizer; /* Parent tokenizer instance */
char aBuf[FTS5_PORTER_MAX_TOKEN + 64];
};
@@ -561,7 +565,7 @@ static void fts5PorterDelete(Fts5Tokenizer *pTok){
if( pTok ){
PorterTokenizer *p = (PorterTokenizer*)pTok;
if( p->pTokenizer ){
p->tokenizer.xDelete(p->pTokenizer);
p->tokenizer_v2.xDelete(p->pTokenizer);
}
sqlite3_free(p);
}
@@ -580,6 +584,7 @@ static int fts5PorterCreate(
PorterTokenizer *pRet;
void *pUserdata = 0;
const char *zBase = "unicode61";
fts5_tokenizer_v2 *pV2 = 0;
if( nArg>0 ){
zBase = azArg[0];
@@ -588,14 +593,15 @@ static int fts5PorterCreate(
pRet = (PorterTokenizer*)sqlite3_malloc(sizeof(PorterTokenizer));
if( pRet ){
memset(pRet, 0, sizeof(PorterTokenizer));
rc = pApi->xFindTokenizer(pApi, zBase, &pUserdata, &pRet->tokenizer);
rc = pApi->xFindTokenizer_v2(pApi, zBase, &pUserdata, &pV2);
}else{
rc = SQLITE_NOMEM;
}
if( rc==SQLITE_OK ){
int nArg2 = (nArg>0 ? nArg-1 : 0);
const char **azArg2 = (nArg2 ? &azArg[1] : 0);
rc = pRet->tokenizer.xCreate(pUserdata, azArg2, nArg2, &pRet->pTokenizer);
const char **az2 = (nArg2 ? &azArg[1] : 0);
memcpy(&pRet->tokenizer_v2, pV2, sizeof(fts5_tokenizer_v2));
rc = pRet->tokenizer_v2.xCreate(pUserdata, az2, nArg2, &pRet->pTokenizer);
}
if( rc!=SQLITE_OK ){
@@ -1246,6 +1252,7 @@ static int fts5PorterTokenize(
void *pCtx,
int flags,
const char *pText, int nText,
const char *pLoc, int nLoc,
int (*xToken)(void*, int, const char*, int nToken, int iStart, int iEnd)
){
PorterTokenizer *p = (PorterTokenizer*)pTokenizer;
@@ -1253,8 +1260,8 @@ static int fts5PorterTokenize(
sCtx.xToken = xToken;
sCtx.pCtx = pCtx;
sCtx.aBuf = p->aBuf;
return p->tokenizer.xTokenize(
p->pTokenizer, (void*)&sCtx, flags, pText, nText, fts5PorterCb
return p->tokenizer_v2.xTokenize(
p->pTokenizer, (void*)&sCtx, flags, pText, nText, pLoc, nLoc, fts5PorterCb
);
}
@@ -1264,6 +1271,7 @@ static int fts5PorterTokenize(
typedef struct TrigramTokenizer TrigramTokenizer;
struct TrigramTokenizer {
int bFold; /* True to fold to lower-case */
int iFoldParam; /* Parameter to pass to Fts5UnicodeFold() */
};
/*
@@ -1283,28 +1291,46 @@ static int fts5TriCreate(
Fts5Tokenizer **ppOut
){
int rc = SQLITE_OK;
TrigramTokenizer *pNew = (TrigramTokenizer*)sqlite3_malloc(sizeof(*pNew));
TrigramTokenizer *pNew = 0;
UNUSED_PARAM(pUnused);
if( pNew==0 ){
rc = SQLITE_NOMEM;
if( nArg%2 ){
rc = SQLITE_ERROR;
}else{
int i;
pNew->bFold = 1;
for(i=0; rc==SQLITE_OK && i<nArg; i+=2){
const char *zArg = azArg[i+1];
if( 0==sqlite3_stricmp(azArg[i], "case_sensitive") ){
if( (zArg[0]!='0' && zArg[0]!='1') || zArg[1] ){
rc = SQLITE_ERROR;
pNew = (TrigramTokenizer*)sqlite3_malloc(sizeof(*pNew));
if( pNew==0 ){
rc = SQLITE_NOMEM;
}else{
pNew->bFold = 1;
pNew->iFoldParam = 0;
for(i=0; rc==SQLITE_OK && i<nArg; i+=2){
const char *zArg = azArg[i+1];
if( 0==sqlite3_stricmp(azArg[i], "case_sensitive") ){
if( (zArg[0]!='0' && zArg[0]!='1') || zArg[1] ){
rc = SQLITE_ERROR;
}else{
pNew->bFold = (zArg[0]=='0');
}
}else if( 0==sqlite3_stricmp(azArg[i], "remove_diacritics") ){
if( (zArg[0]!='0' && zArg[0]!='1' && zArg[0]!='2') || zArg[1] ){
rc = SQLITE_ERROR;
}else{
pNew->iFoldParam = (zArg[0]!='0') ? 2 : 0;
}
}else{
pNew->bFold = (zArg[0]=='0');
rc = SQLITE_ERROR;
}
}else{
}
if( pNew->iFoldParam!=0 && pNew->bFold==0 ){
rc = SQLITE_ERROR;
}
}
if( rc!=SQLITE_OK ){
fts5TriDelete((Fts5Tokenizer*)pNew);
pNew = 0;
if( rc!=SQLITE_OK ){
fts5TriDelete((Fts5Tokenizer*)pNew);
pNew = 0;
}
}
}
*ppOut = (Fts5Tokenizer*)pNew;
@@ -1324,40 +1350,62 @@ static int fts5TriTokenize(
TrigramTokenizer *p = (TrigramTokenizer*)pTok;
int rc = SQLITE_OK;
char aBuf[32];
char *zOut = aBuf;
int ii;
const unsigned char *zIn = (const unsigned char*)pText;
const unsigned char *zEof = &zIn[nText];
u32 iCode;
int aStart[3]; /* Input offset of each character in aBuf[] */
UNUSED_PARAM(unusedFlags);
while( 1 ){
char *zOut = aBuf;
int iStart = zIn - (const unsigned char*)pText;
const unsigned char *zNext;
READ_UTF8(zIn, zEof, iCode);
if( iCode==0 ) break;
zNext = zIn;
if( zIn<zEof ){
if( p->bFold ) iCode = sqlite3Fts5UnicodeFold(iCode, 0);
WRITE_UTF8(zOut, iCode);
/* Populate aBuf[] with the characters for the first trigram. */
for(ii=0; ii<3; ii++){
do {
aStart[ii] = zIn - (const unsigned char*)pText;
READ_UTF8(zIn, zEof, iCode);
if( iCode==0 ) return SQLITE_OK;
if( p->bFold ) iCode = sqlite3Fts5UnicodeFold(iCode, p->iFoldParam);
}while( iCode==0 );
WRITE_UTF8(zOut, iCode);
}
/* At the start of each iteration of this loop:
**
** aBuf: Contains 3 characters. The 3 characters of the next trigram.
** zOut: Points to the byte following the last character in aBuf.
** aStart[3]: Contains the byte offset in the input text corresponding
** to the start of each of the three characters in the buffer.
*/
assert( zIn<=zEof );
while( 1 ){
int iNext; /* Start of character following current tri */
const char *z1;
/* Read characters from the input up until the first non-diacritic */
do {
iNext = zIn - (const unsigned char*)pText;
READ_UTF8(zIn, zEof, iCode);
if( iCode==0 ) break;
}else{
break;
}
if( zIn<zEof ){
if( p->bFold ) iCode = sqlite3Fts5UnicodeFold(iCode, 0);
WRITE_UTF8(zOut, iCode);
READ_UTF8(zIn, zEof, iCode);
if( iCode==0 ) break;
if( p->bFold ) iCode = sqlite3Fts5UnicodeFold(iCode, 0);
WRITE_UTF8(zOut, iCode);
}else{
break;
}
rc = xToken(pCtx, 0, aBuf, zOut-aBuf, iStart, iStart + zOut-aBuf);
if( rc!=SQLITE_OK ) break;
zIn = zNext;
if( p->bFold ) iCode = sqlite3Fts5UnicodeFold(iCode, p->iFoldParam);
}while( iCode==0 );
/* Pass the current trigram back to fts5 */
rc = xToken(pCtx, 0, aBuf, zOut-aBuf, aStart[0], iNext);
if( iCode==0 || rc!=SQLITE_OK ) break;
/* Remove the first character from buffer aBuf[]. Append the character
** with codepoint iCode. */
z1 = aBuf;
FTS5_SKIP_UTF8(z1);
memmove(aBuf, z1, zOut - z1);
zOut -= (z1 - aBuf);
WRITE_UTF8(zOut, iCode);
/* Update the aStart[] array */
aStart[0] = aStart[1];
aStart[1] = aStart[2];
aStart[2] = iNext;
}
return rc;
@@ -1380,11 +1428,23 @@ int sqlite3Fts5TokenizerPattern(
){
if( xCreate==fts5TriCreate ){
TrigramTokenizer *p = (TrigramTokenizer*)pTok;
return p->bFold ? FTS5_PATTERN_LIKE : FTS5_PATTERN_GLOB;
if( p->iFoldParam==0 ){
return p->bFold ? FTS5_PATTERN_LIKE : FTS5_PATTERN_GLOB;
}
}
return FTS5_PATTERN_NONE;
}
/*
** Return true if the tokenizer described by p->azArg[] is the trigram
** tokenizer. This tokenizer needs to be loaded before xBestIndex is
** called for the first time in order to correctly handle LIKE/GLOB.
*/
int sqlite3Fts5TokenizerPreload(Fts5TokenizerConfig *p){
return (p->nArg>=1 && 0==sqlite3_stricmp(p->azArg[0], "trigram"));
}
/*
** Register all built-in tokenizers with FTS5.
*/
@@ -1395,7 +1455,6 @@ int sqlite3Fts5TokenizerInit(fts5_api *pApi){
} aBuiltin[] = {
{ "unicode61", {fts5UnicodeCreate, fts5UnicodeDelete, fts5UnicodeTokenize}},
{ "ascii", {fts5AsciiCreate, fts5AsciiDelete, fts5AsciiTokenize }},
{ "porter", {fts5PorterCreate, fts5PorterDelete, fts5PorterTokenize }},
{ "trigram", {fts5TriCreate, fts5TriDelete, fts5TriTokenize}},
};
@@ -1410,6 +1469,19 @@ int sqlite3Fts5TokenizerInit(fts5_api *pApi){
0
);
}
if( rc==SQLITE_OK ){
fts5_tokenizer_v2 sPorter = {
2,
fts5PorterCreate,
fts5PorterDelete,
fts5PorterTokenize
};
rc = pApi->xCreateTokenizer_v2(pApi,
"porter",
(void*)pApi,
&sPorter,
0
);
}
return rc;
}
+3
View File
@@ -364,6 +364,9 @@ int sqlite3Fts5UnicodeCatParse(const char *zCat, u8 *aArray){
default: return 1; }
break;
default:
return 1;
}
return 0;
}
+27 -10
View File
@@ -64,6 +64,7 @@ struct Fts5VocabCursor {
int nLeTerm; /* Size of zLeTerm in bytes */
char *zLeTerm; /* (term <= $zLeTerm) paramater, or NULL */
int colUsed; /* Copy of sqlite3_index_info.colUsed */
/* These are used by 'col' tables only */
int iCol;
@@ -90,9 +91,11 @@ struct Fts5VocabCursor {
/*
** Bits for the mask used as the idxNum value by xBestIndex/xFilter.
*/
#define FTS5_VOCAB_TERM_EQ 0x01
#define FTS5_VOCAB_TERM_GE 0x02
#define FTS5_VOCAB_TERM_LE 0x04
#define FTS5_VOCAB_TERM_EQ 0x0100
#define FTS5_VOCAB_TERM_GE 0x0200
#define FTS5_VOCAB_TERM_LE 0x0400
#define FTS5_VOCAB_COLUSED_MASK 0xFF
/*
@@ -269,11 +272,13 @@ static int fts5VocabBestIndexMethod(
int iTermEq = -1;
int iTermGe = -1;
int iTermLe = -1;
int idxNum = 0;
int idxNum = (int)pInfo->colUsed;
int nArg = 0;
UNUSED_PARAM(pUnused);
assert( (pInfo->colUsed & FTS5_VOCAB_COLUSED_MASK)==pInfo->colUsed );
for(i=0; i<pInfo->nConstraint; i++){
struct sqlite3_index_constraint *p = &pInfo->aConstraint[i];
if( p->usable==0 ) continue;
@@ -365,7 +370,7 @@ static int fts5VocabOpenMethod(
if( rc==SQLITE_OK ){
pVTab->zErrMsg = sqlite3_mprintf(
"no such fts5 table: %s.%s", pTab->zFts5Db, pTab->zFts5Tbl
);
);
rc = SQLITE_ERROR;
}
}else{
@@ -525,9 +530,19 @@ static int fts5VocabNextMethod(sqlite3_vtab_cursor *pCursor){
switch( pTab->eType ){
case FTS5_VOCAB_ROW:
if( eDetail==FTS5_DETAIL_FULL ){
while( 0==sqlite3Fts5PoslistNext64(pPos, nPos, &iOff, &iPos) ){
pCsr->aCnt[0]++;
/* Do not bother counting the number of instances if the "cnt"
** column is not being read (according to colUsed). */
if( eDetail==FTS5_DETAIL_FULL && (pCsr->colUsed & 0x04) ){
while( iPos<nPos ){
u32 ii;
fts5FastGetVarint32(pPos, iPos, ii);
if( ii==1 ){
/* New column in the position list */
fts5FastGetVarint32(pPos, iPos, ii);
}else{
/* An instance - increment pCsr->aCnt[] */
pCsr->aCnt[0]++;
}
}
}
pCsr->aDoc[0]++;
@@ -625,11 +640,12 @@ static int fts5VocabFilterMethod(
if( idxNum & FTS5_VOCAB_TERM_EQ ) pEq = apVal[iVal++];
if( idxNum & FTS5_VOCAB_TERM_GE ) pGe = apVal[iVal++];
if( idxNum & FTS5_VOCAB_TERM_LE ) pLe = apVal[iVal++];
pCsr->colUsed = (idxNum & FTS5_VOCAB_COLUSED_MASK);
if( pEq ){
zTerm = (const char *)sqlite3_value_text(pEq);
nTerm = sqlite3_value_bytes(pEq);
f = 0;
f = FTS5INDEX_QUERY_NOTOKENDATA;
}else{
if( pGe ){
zTerm = (const char *)sqlite3_value_text(pGe);
@@ -783,7 +799,8 @@ int sqlite3Fts5VocabInit(Fts5Global *pGlobal, sqlite3 *db){
/* xSavepoint */ 0,
/* xRelease */ 0,
/* xRollbackTo */ 0,
/* xShadowName */ 0
/* xShadowName */ 0,
/* xIntegrity */ 0
};
void *p = (void*)pGlobal;
+61
View File
@@ -51,6 +51,10 @@ proc fts5_test_poslist2 {cmd} {
sort_poslist $res
}
proc fts5_test_insttoken {cmd iInst iToken} {
$cmd xInstToken $iInst $iToken
}
proc fts5_test_collist {cmd} {
set res [list]
@@ -61,6 +65,12 @@ proc fts5_test_collist {cmd} {
set res
}
proc fts5_collist {cmd iPhrase} {
set res [list]
$cmd xPhraseColumnForeach $iPhrase c { lappend res $c }
set res
}
proc fts5_test_columnsize {cmd} {
set res [list]
for {set i 0} {$i < [$cmd xColumnCount]} {incr i} {
@@ -69,6 +79,13 @@ proc fts5_test_columnsize {cmd} {
set res
}
proc fts5_columntext {cmd iCol} {
$cmd xColumnText $iCol
}
proc fts5_columnlocale {cmd iCol} {
$cmd xColumnLocale $iCol
}
proc fts5_test_columntext {cmd} {
set res [list]
for {set i 0} {$i < [$cmd xColumnCount]} {incr i} {
@@ -77,6 +94,14 @@ proc fts5_test_columntext {cmd} {
set res
}
proc fts5_test_columnlocale {cmd} {
set res [list]
for {set i 0} {$i < [$cmd xColumnCount]} {incr i} {
lappend res [$cmd xColumnLocale $i]
}
set res
}
proc fts5_test_columntotalsize {cmd} {
set res [list]
for {set i 0} {$i < [$cmd xColumnCount]} {incr i} {
@@ -104,6 +129,10 @@ proc fts5_test_rowcount {cmd} {
$cmd xRowCount
}
proc fts5_test_rowid {cmd} {
$cmd xRowid
}
proc test_queryphrase_cb {cnt cmd} {
upvar $cnt L
for {set i 0} {$i < [$cmd xInstCount]} {incr i} {
@@ -125,6 +154,13 @@ proc fts5_test_queryphrase {cmd} {
set res
}
proc fts5_queryphrase {cmd iPhrase} {
set cnt [list]
for {set j 0} {$j < [$cmd xColumnCount]} {incr j} { lappend cnt 0 }
$cmd xQueryPhrase $iPhrase [list test_queryphrase_cb cnt]
set cnt
}
proc fts5_test_phrasecount {cmd} {
$cmd xPhraseCount
}
@@ -144,16 +180,23 @@ proc fts5_aux_test_functions {db} {
foreach f {
fts5_test_columnsize
fts5_test_columntext
fts5_test_columnlocale
fts5_test_columntotalsize
fts5_test_poslist
fts5_test_poslist2
fts5_test_collist
fts5_test_insttoken
fts5_test_tokenize
fts5_test_rowcount
fts5_test_rowid
fts5_test_all
fts5_test_queryphrase
fts5_test_phrasecount
fts5_columntext
fts5_columnlocale
fts5_queryphrase
fts5_collist
} {
sqlite3_fts5_create_function $db $f $f
}
@@ -438,6 +481,20 @@ proc detail_is_none {} { detail_check ; expr {$::detail == "none"} }
proc detail_is_col {} { detail_check ; expr {$::detail == "col" } }
proc detail_is_full {} { detail_check ; expr {$::detail == "full"} }
proc foreach_tokenizer_mode {prefix script} {
set saved $::testprefix
foreach {d mapping} {
"" {}
"-origintext" {, tokenize="origintext unicode61", tokendata=1}
} {
set s [string map [list %TOKENIZER% $mapping] $script]
set ::testprefix "$prefix$d"
reset_db
sqlite3_fts5_register_origintext db
uplevel $s
}
set ::testprefix $saved
}
#-------------------------------------------------------------------------
# Convert a poslist of the type returned by fts5_test_poslist() to a
@@ -594,6 +651,10 @@ proc nearset_rc {aCol args} {
list
}
proc dump {tname} {
execsql_pp "SELECT * FROM ${tname}_idx"
execsql_pp "SELECT id, quote(block), fts5_decode(id,block) FROM ${tname}_data"
}
#-------------------------------------------------------------------------
# Code for a simple Tcl tokenizer that supports synonyms at query time.
+51 -28
View File
@@ -22,6 +22,7 @@ ifcapable !fts5 {
}
foreach_detail_mode $::testprefix {
foreach_tokenizer_mode $::testprefix {
do_execsql_test 1.0 {
CREATE VIRTUAL TABLE t1 USING fts5(a, b, c);
@@ -44,7 +45,7 @@ do_execsql_test 1.1 {
#
do_execsql_test 2.0 {
CREATE VIRTUAL TABLE t1 USING fts5(x, y, detail=%DETAIL%);
CREATE VIRTUAL TABLE t1 USING fts5(x, y, detail=%DETAIL% %TOKENIZER%);
}
do_execsql_test 2.1 {
INSERT INTO t1 VALUES('a b c', 'd e f');
@@ -65,14 +66,17 @@ foreach w {a b c d e f} {
do_execsql_test 2.4 {
INSERT INTO t1(t1) VALUES('integrity-check');
}
PRAGMA integrity_check;
PRAGMA integrity_check(t1);
} {ok ok}
#-------------------------------------------------------------------------
#
reset_db
sqlite3_fts5_register_origintext db
do_execsql_test 3.0 {
CREATE VIRTUAL TABLE t1 USING fts5(x,y, detail=%DETAIL%);
CREATE VIRTUAL TABLE t1 USING fts5(x,y, detail=%DETAIL% %TOKENIZER%);
}
foreach {i x y} {
1 {g f d b f} {h h e i a}
@@ -88,14 +92,16 @@ foreach {i x y} {
} {
do_execsql_test 3.$i.1 { INSERT INTO t1 VALUES($x, $y) }
do_execsql_test 3.$i.2 { INSERT INTO t1(t1) VALUES('integrity-check') }
do_execsql_test 3.$i.3 { PRAGMA integrity_check(t1) } ok
if {[set_test_counter errors]} break
}
#-------------------------------------------------------------------------
#
reset_db
sqlite3_fts5_register_origintext db
do_execsql_test 4.0 {
CREATE VIRTUAL TABLE t1 USING fts5(x,y, detail=%DETAIL%);
CREATE VIRTUAL TABLE t1 USING fts5(x,y, detail=%DETAIL% %TOKENIZER%);
INSERT INTO t1(t1, rank) VALUES('pgsz', 32);
}
foreach {i x y} {
@@ -118,8 +124,9 @@ foreach {i x y} {
#-------------------------------------------------------------------------
#
reset_db
sqlite3_fts5_register_origintext db
do_execsql_test 5.0 {
CREATE VIRTUAL TABLE t1 USING fts5(x,y, detail=%DETAIL%);
CREATE VIRTUAL TABLE t1 USING fts5(x,y, detail=%DETAIL% %TOKENIZER%);
INSERT INTO t1(t1, rank) VALUES('pgsz', 32);
}
foreach {i x y} {
@@ -135,15 +142,16 @@ foreach {i x y} {
10 {ddd abcde dddd dd c} {dddd c c d abcde}
} {
do_execsql_test 5.$i.1 { INSERT INTO t1 VALUES($x, $y) }
do_execsql_test 5.$i.2 { INSERT INTO t1(t1) VALUES('integrity-check') }
do_execsql_test 5.$i.2 { PRAGMA integrity_check(t1) } ok
if {[set_test_counter errors]} break
}
#-------------------------------------------------------------------------
#
reset_db
sqlite3_fts5_register_origintext db
do_execsql_test 6.0 {
CREATE VIRTUAL TABLE t1 USING fts5(x,y, detail=%DETAIL%);
CREATE VIRTUAL TABLE t1 USING fts5(x,y, detail=%DETAIL% %TOKENIZER%);
INSERT INTO t1(t1, rank) VALUES('pgsz', 32);
}
@@ -178,6 +186,7 @@ do_execsql_test 6.6 {
#-------------------------------------------------------------------------
#
reset_db
sqlite3_fts5_register_origintext db
expr srand(0)
do_execsql_test 7.0 {
CREATE VIRTUAL TABLE t1 USING fts5(x,y,z);
@@ -219,6 +228,7 @@ for {set i 1} {$i <= 10} {incr i} {
#-------------------------------------------------------------------------
#
reset_db
sqlite3_fts5_register_origintext db
do_execsql_test 8.0 {
CREATE VIRTUAL TABLE t1 USING fts5(x, prefix="1,2,3");
INSERT INTO t1(t1, rank) VALUES('pgsz', 32);
@@ -233,6 +243,7 @@ do_execsql_test 8.1 {
#-------------------------------------------------------------------------
#
reset_db
sqlite3_fts5_register_origintext db
expr srand(0)
@@ -277,8 +288,9 @@ for {set i 1} {$i <= 10} {incr i} {
#-------------------------------------------------------------------------
#
reset_db
sqlite3_fts5_register_origintext db
do_execsql_test 10.0 {
CREATE VIRTUAL TABLE t1 USING fts5(x,y, detail=%DETAIL%);
CREATE VIRTUAL TABLE t1 USING fts5(x,y, detail=%DETAIL% %TOKENIZER%);
}
set d10 {
1 {g f d b f} {h h e i a}
@@ -311,19 +323,19 @@ do_execsql_test 10.4.2 { INSERT INTO t1(t1) VALUES('integrity-check') }
#-------------------------------------------------------------------------
#
do_catchsql_test 11.1 {
CREATE VIRTUAL TABLE t2 USING fts5(a, b, c, rank, detail=%DETAIL%);
CREATE VIRTUAL TABLE t2 USING fts5(a, b, c, rank, detail=%DETAIL% %TOKENIZER%);
} {1 {reserved fts5 column name: rank}}
do_catchsql_test 11.2 {
CREATE VIRTUAL TABLE rank USING fts5(a, b, c, detail=%DETAIL%);
CREATE VIRTUAL TABLE rank USING fts5(a, b, c, detail=%DETAIL% %TOKENIZER%);
} {1 {reserved fts5 table name: rank}}
do_catchsql_test 11.3 {
CREATE VIRTUAL TABLE t2 USING fts5(a, b, c, rowid, detail=%DETAIL%);
CREATE VIRTUAL TABLE t2 USING fts5(a, b, c, rowid, detail=%DETAIL% %TOKENIZER%);
} {1 {reserved fts5 column name: rowid}}
#-------------------------------------------------------------------------
#
do_execsql_test 12.1 {
CREATE VIRTUAL TABLE t2 USING fts5(x,y, detail=%DETAIL%);
CREATE VIRTUAL TABLE t2 USING fts5(x,y, detail=%DETAIL% %TOKENIZER%);
} {}
do_catchsql_test 12.2 {
@@ -338,8 +350,9 @@ do_test 12.3 {
#-------------------------------------------------------------------------
#
reset_db
sqlite3_fts5_register_origintext db
do_execsql_test 13.1 {
CREATE VIRTUAL TABLE t1 USING fts5(x, detail=%DETAIL%);
CREATE VIRTUAL TABLE t1 USING fts5(x, detail=%DETAIL% %TOKENIZER%);
INSERT INTO t1(rowid, x) VALUES(1, 'o n e'), (2, 't w o');
} {}
@@ -362,8 +375,9 @@ do_execsql_test 13.6 {
#-------------------------------------------------------------------------
#
reset_db
sqlite3_fts5_register_origintext db
do_execsql_test 14.1 {
CREATE VIRTUAL TABLE t1 USING fts5(x, y, detail=%DETAIL%);
CREATE VIRTUAL TABLE t1 USING fts5(x, y, detail=%DETAIL% %TOKENIZER%);
INSERT INTO t1(t1, rank) VALUES('pgsz', 32);
WITH d(x,y) AS (
SELECT NULL, 'xyz xyz xyz xyz xyz xyz'
@@ -426,7 +440,7 @@ do_execsql_test 16.1 {
proc funk {} {
db eval { UPDATE n1_config SET v=50 WHERE k='version' }
set fd [db incrblob main n1_data block 10]
fconfigure $fd -encoding binary -translation binary
fconfigure $fd -translation binary
# puts -nonewline $fd "\x44\x45"
close $fd
}
@@ -439,15 +453,16 @@ db func funk funk
# statement no longer fails.
#
do_catchsql_test 16.2 {
SELECT funk(), bm25(n1), funk() FROM n1 WHERE n1 MATCH 'a+b+c+d'
SELECT funk(), format('%g',bm25(n1)), funk() FROM n1 WHERE n1 MATCH 'a+b+c+d'
} {0 {{} -1e-06 {}}}
# {1 {SQL logic error}}
#-------------------------------------------------------------------------
#
reset_db
sqlite3_fts5_register_origintext db
do_execsql_test 17.1 {
CREATE VIRTUAL TABLE b2 USING fts5(x, detail=%DETAIL%);
CREATE VIRTUAL TABLE b2 USING fts5(x, detail=%DETAIL% %TOKENIZER%);
INSERT INTO b2 VALUES('a');
INSERT INTO b2 VALUES('b');
INSERT INTO b2 VALUES('c');
@@ -463,8 +478,9 @@ do_test 17.2 {
if {[string match n* %DETAIL%]==0} {
reset_db
sqlite3_fts5_register_origintext db
do_execsql_test 17.3 {
CREATE VIRTUAL TABLE c2 USING fts5(x, y, detail=%DETAIL%);
CREATE VIRTUAL TABLE c2 USING fts5(x, y, detail=%DETAIL% %TOKENIZER%);
INSERT INTO c2 VALUES('x x x', 'x x x');
SELECT rowid FROM c2 WHERE c2 MATCH 'y:x';
} {1}
@@ -473,8 +489,9 @@ if {[string match n* %DETAIL%]==0} {
#-------------------------------------------------------------------------
#
reset_db
sqlite3_fts5_register_origintext db
do_execsql_test 17.1 {
CREATE VIRTUAL TABLE uio USING fts5(ttt, detail=%DETAIL%);
CREATE VIRTUAL TABLE uio USING fts5(ttt, detail=%DETAIL% %TOKENIZER%);
INSERT INTO uio VALUES(NULL);
INSERT INTO uio SELECT NULL FROM uio;
INSERT INTO uio SELECT NULL FROM uio;
@@ -521,8 +538,8 @@ do_execsql_test 17.9 {
#--------------------------------------------------------------------
#
do_execsql_test 18.1 {
CREATE VIRTUAL TABLE t1 USING fts5(a, b, detail=%DETAIL%);
CREATE VIRTUAL TABLE t2 USING fts5(c, d, detail=%DETAIL%);
CREATE VIRTUAL TABLE t1 USING fts5(a, b, detail=%DETAIL% %TOKENIZER%);
CREATE VIRTUAL TABLE t2 USING fts5(c, d, detail=%DETAIL% %TOKENIZER%);
INSERT INTO t1 VALUES('abc*', NULL);
INSERT INTO t2 VALUES(1, 'abcdefg');
}
@@ -537,8 +554,9 @@ do_execsql_test 18.3 {
# fts5 table in the temp schema.
#
reset_db
sqlite3_fts5_register_origintext db
do_execsql_test 19.0 {
CREATE VIRTUAL TABLE temp.t1 USING fts5(x, detail=%DETAIL%);
CREATE VIRTUAL TABLE temp.t1 USING fts5(x, detail=%DETAIL% %TOKENIZER%);
INSERT INTO t1 VALUES('x y z');
INSERT INTO t1 VALUES('w x 1');
SELECT rowid FROM t1 WHERE t1 MATCH 'x';
@@ -548,8 +566,9 @@ do_execsql_test 19.0 {
# Test that 6 and 7 byte varints can be read.
#
reset_db
sqlite3_fts5_register_origintext db
do_execsql_test 20.0 {
CREATE VIRTUAL TABLE temp.tmp USING fts5(x, detail=%DETAIL%);
CREATE VIRTUAL TABLE temp.tmp USING fts5(x, detail=%DETAIL% %TOKENIZER%);
}
set ::ids [list \
0 [expr 1<<36] [expr 2<<36] [expr 1<<43] [expr 2<<43]
@@ -567,7 +586,7 @@ do_test 20.1 {
#
do_execsql_test 21.0 {
CREATE TEMP TABLE t8(a, b);
CREATE VIRTUAL TABLE ft USING fts5(x, detail=%DETAIL%);
CREATE VIRTUAL TABLE ft USING fts5(x, detail=%DETAIL% %TOKENIZER%);
}
do_execsql_test 21.1 {
@@ -578,7 +597,7 @@ do_execsql_test 21.1 {
}
do_execsql_test 22.0 {
CREATE VIRTUAL TABLE t9 USING fts5(x, detail=%DETAIL%);
CREATE VIRTUAL TABLE t9 USING fts5(x, detail=%DETAIL% %TOKENIZER%);
INSERT INTO t9(rowid, x) VALUES(2, 'bbb');
BEGIN;
INSERT INTO t9(rowid, x) VALUES(1, 'aaa');
@@ -593,7 +612,7 @@ do_execsql_test 22.1 {
#-------------------------------------------------------------------------
do_execsql_test 23.0 {
CREATE VIRTUAL TABLE t10 USING fts5(x, detail=%DETAIL%);
CREATE VIRTUAL TABLE t10 USING fts5(x, detail=%DETAIL% %TOKENIZER%);
CREATE TABLE t11(x);
}
do_execsql_test 23.1 {
@@ -605,7 +624,7 @@ do_execsql_test 23.2 {
#-------------------------------------------------------------------------
do_execsql_test 24.0 {
CREATE VIRTUAL TABLE t12 USING fts5(x, detail=%DETAIL%);
CREATE VIRTUAL TABLE t12 USING fts5(x, detail=%DETAIL% %TOKENIZER%);
INSERT INTO t12 VALUES('aaaa');
}
do_execsql_test 24.1 {
@@ -615,6 +634,9 @@ do_execsql_test 24.1 {
INSERT INTO t12 VALUES('aaaa');
END;
}
execsql_pp {
SELECT rowid, hex(block) FROM t12_data
}
do_execsql_test 24.2 {
INSERT INTO t12(t12) VALUES('integrity-check');
}
@@ -624,7 +646,7 @@ do_execsql_test 24.3 {
#-------------------------------------------------------------------------
do_execsql_test 25.0 {
CREATE VIRTUAL TABLE t13 USING fts5(x, detail=%DETAIL%);
CREATE VIRTUAL TABLE t13 USING fts5(x, detail=%DETAIL% %TOKENIZER%);
}
do_execsql_test 25.1 {
BEGIN;
@@ -635,6 +657,7 @@ SELECT * FROM t13('BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
}
}
}
expand_all_sql db
+6 -2
View File
@@ -16,7 +16,7 @@
source [file join [file dirname [info script]] fts5_common.tcl]
set testprefix fts5ab
# If SQLITE_ENABLE_FTS5 is defined, omit this file.
# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
ifcapable !fts5 {
finish_test
return
@@ -180,7 +180,11 @@ if {[detail_is_full]} {
} {1 2}
}
do_execsql_test 4.5 {
do_execsql_test 4.5.1 {
SELECT rowid FROM s1 WHERE s1 MATCH 'a AND x'
} {1 2}
do_execsql_test 4.5.2 {
SELECT rowid FROM s1 WHERE s1 MATCH 'a x'
} {1 2}
+1 -1
View File
@@ -16,7 +16,7 @@
source [file join [file dirname [info script]] fts5_common.tcl]
set testprefix fts5ac
# If SQLITE_ENABLE_FTS5 is defined, omit this file.
# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
ifcapable !fts5 {
finish_test
return
+1 -1
View File
@@ -18,7 +18,7 @@
source [file join [file dirname [info script]] fts5_common.tcl]
set testprefix fts5ad
# If SQLITE_ENABLE_FTS5 is defined, omit this file.
# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
ifcapable !fts5 {
finish_test
return
+1 -1
View File
@@ -16,7 +16,7 @@
source [file join [file dirname [info script]] fts5_common.tcl]
set testprefix fts5ae
# If SQLITE_ENABLE_FTS5 is defined, omit this file.
# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
ifcapable !fts5 {
finish_test
return
+31 -1
View File
@@ -18,7 +18,7 @@
source [file join [file dirname [info script]] fts5_common.tcl]
set testprefix fts5af
# If SQLITE_ENABLE_FTS5 is defined, omit this file.
# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
ifcapable !fts5 {
finish_test
return
@@ -193,4 +193,34 @@ do_execsql_test 5.6 {
} ;# foreach_detail_mode
reset_db
do_execsql_test 6.0 {
CREATE VIRTUAL TABLE t1 USING fts5(colA, colB);
INSERT INTO t1 VALUES('A B C', 'D E F');
}
do_execsql_test 6.1 {
SELECT colA, colB, snippet(t1,0,'[', ']','...',1) FROM t1 WHERE t1 MATCH 'B';
} {{A B C} {D E F} ...[B]...}
breakpoint
do_execsql_test 6.2 {
SELECT colA, colB, snippet(t1, 1,'[',']','...',2) FROM t1 WHERE t1 MATCH 'B';
} {{A B C} {D E F} {D E...}}
do_execsql_test 6.3 {
SELECT colA, colB, snippet(t1, 1,'[',']','...',1) FROM t1 WHERE t1 MATCH 'B';
} {{A B C} {D E F} {D...}}
do_execsql_test 6.1 {
SELECT colA, colB, snippet(t1,0,'[', ']','...',1) FROM t1 WHERE t1 MATCH 'A';
} {{A B C} {D E F} [A]...}
breakpoint
do_execsql_test 6.2 {
SELECT colA, colB, snippet(t1, 1,'[',']','...',2) FROM t1 WHERE t1 MATCH 'A';
} {{A B C} {D E F} {D E...}}
do_execsql_test 6.3 {
SELECT colA, colB, snippet(t1, 1,'[',']','...',1) FROM t1 WHERE t1 MATCH 'A';
} {{A B C} {D E F} {D...}}
finish_test

Some files were not shown because too many files have changed in this diff Show More