paulhorn/sqlite
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
32,251 Commits 1,696 Branches 387 Tags
begin-concurrent
Commit Graph

738 Commits

Author SHA1 Message Date
drh 56363a8d07 Merge the latest trunk fixes and enhancements into the begin-concurrent branch. 
FossilOrigin-Name: 330387a9c6a0ce6716e62f920d874bbea8b2e38f549fc03dd72d537284f1a01d
 2026-05-28 20:11:47 +00:00
stephan 6f842b8da4 For the various OPFS VFS db-import routines, in SEE builds (only) relax the is-this-a-db check to allow for SEE dbs not having the usual header string in cleartext. Addresses [https://sqlite.org/see/forumpost/f84bef3552|SEE forum post f84bef3552]. 
FossilOrigin-Name: a5a9ac7ad6622b5b07193aa279deefde7d59ca17f81aba9dd78ee90718446993
 2026-05-05 08:51:10 +00:00
drh 1768ec5ba7 Merge recent trunk enhancements into the begin-concurrent branch. 
FossilOrigin-Name: 7f954a9e2fa4203b55825dfd70a46ffde7c985a4c8b940208d74d97441f3fd04
 2026-04-22 19:50:38 +00:00
stephan d3abfd77f5 Reformulate how the OPFS async proxy Worker is loaded for the bundler-friendly builds to work around a bug in Vite in which it incompatibly rewrites the Worker's URL arguments. Addresses the discussion at [https://github.com/sqlite/sqlite-wasm/pull/159|npm package PR 159] and patch validated by Jure Rotar. 
FossilOrigin-Name: ec877a30f5c5db30b9154745140744c5cc80d89673d79e50901d7377599cf2b4
 2026-04-21 10:19:38 +00:00
stephan 9d950aa13b A protype of .prompt support in fiddle but its utility there is in doubt because .open does not work there, so there's very little dynamic info worth showing in a prompt. Also, ANSI colorization can't work there, nor can we colorize the prompts using HTML because of limitations in jquery.terminal. 
FossilOrigin-Name: 3b17500dc6338df23d1ed0572cd71593da5a9b048731f73242e27b9a76553a27
 2026-04-10 23:05:41 +00:00
stephan 35f8abcfd0 API doc correction for sqlite3_js_retry_busy(). No code changes. 
FossilOrigin-Name: 749c537812b907164b7e7ad1fddc1aee8106c59875d2686fe83d602cd0e07611
 2026-04-10 17:55:31 +00:00
stephan d4b74b9fac Add some comments to the OPFS async proxy about potential future experimentation. No code changes. 
FossilOrigin-Name: 9c14ee74305a5db3be95dbd9ae8d1db0abf39b323001be4e2f02b7226e6ca0d3
 2026-04-04 08:07:25 +00:00
drh 7693d9d087 Merge all the latest trunk enhancements into the begin-concurrent branch. 
FossilOrigin-Name: 51cda102d2c8e682076c962f7517b4d546392e31e50de4748d4647799c25c5bc
 2026-04-02 11:52:29 +00:00
stephan eb8edc0bac Doc cleanups in opfs-wl. No code changes. 
FossilOrigin-Name: 009b576dcd5fccaaa52459dbfaa1b2a6fcf6d372e4b8d63553dfa88aab9a759c
 2026-04-01 11:39:13 +00:00
drh e17ca27afe Merge all the latest trunk enhancements into the begin-concurrent branch 
FossilOrigin-Name: 8f4b9a2157d55baffe201b1e2c9a9a34faaeafcde01d8d9411d6d60c3f86f59a
 2026-03-21 17:47:43 +00:00
stephan 46a692c08b Upgrade ext/wasm/c-pp-lite.c to its newer sibling because we've reached the older one's limits. This renames c-pp-lite.c to libcmpp.c to maintain both the SCM- and code lineage but that may end up looking a bit weird because the diff between the two is vast. 
FossilOrigin-Name: 2e2339bd9e4293bad04ece7673a3048b99c2143cf9573ade2ec082d95744b981
 2026-03-08 16:03:08 +00:00
stephan 02cbc96d09 JS doc updates. 
FossilOrigin-Name: 699295f84ef2c4da363f13126abde6af07697d0959896cd868b14513d4278964
 2026-03-08 12:42:34 +00:00
stephan 976b50620a Merge opfs-wl into trunk, not so much for the "opfs-wl" VFS, which is stil tentative (working fine but with little perceived benefit), but for the "opfs" VFS cleanups which necessarily happened around it. 
FossilOrigin-Name: bcaa876c6dafee0995d239f59b6ce7810b6f5faa219b9c392966211ae3dec17a
 2026-03-08 08:32:22 +00:00
stephan 9358070cb3 Expose sqlite3_bind_zeroblob() to JS/WASM. 
FossilOrigin-Name: 0c5d4fe0bd322b4c8dbd0800001317a21cd511b00244de74b4db3fd051153879
 2026-03-08 05:53:47 +00:00
stephan 237fce8205 Extend the JS pre-bootstrapping configuration to include an option to disable inclusion of any given extension VFS (not the default VFSes, like memdb). The primary motivation for this is to give people who don't use OPFS to a way to keep the OPFS VFSes from loading their proxy workers, which are expensive. 
FossilOrigin-Name: 9a07eb7a941479510891d1444aacbeb440efaad3e9a13f186833618d8b60a8da
 2026-03-07 06:10:29 +00:00
stephan 78649bb421 Fix the mangling of the opfs async proxy script's URI caused by both OPFS VFSes modifying it to accound for sqlite3.dir. Add a link to speedtest1 with opfw-wl to index.html. 
FossilOrigin-Name: c1cb61f473a11a325ef421ba8edfb20257203688565f7db62309044be183af5f
 2026-03-07 04:19:08 +00:00
stephan 7d10030d75 Cleanups and docs in the opfs vfs and its concurrency tester. Experimentally add sqlite3.capi.sqlite3_js_retry_busy(), which runs a callback repeatedly until it _stops_ returning (or throwing) an SQLITE_BUSY error. 
FossilOrigin-Name: 45c02ed21635f7ef45214ab5ec6230b6b8bd89e35ef5889db318523e1c679fea
 2026-03-07 03:32:17 +00:00
stephan 9be458e611 Cleanups and docs in the new opfs code structure. 
FossilOrigin-Name: 3b470c4c7a1fcc710e6b9eae32134c7f6c3f6008b24c7351257f66f5e8f70311
 2026-03-07 02:19:23 +00:00
stephan 57d0fe254a For backwards compatibility, ensure that the "opfs" VFS does not specifically require Atomics.waitAsync() (a new requirement of "opfs-wl"), but make use of it if available. Only apply jitter to the concurrency test runs at random intervals. 
FossilOrigin-Name: f2175f526c00cfe562e8f332eb197b5ef2c3d6be1fff2aab1566c2c533a293ac
 2026-03-07 01:01:13 +00:00
stephan ac63b524ac Add some jitter and duration info to the opfs concurrency tester. Ensure that Atomics.waitAsync() is available for opfs-wl. Further testing has shown that Web opfs-wl is consistently fairer about doling out contested locks but it's dog slow compared to the original VFS under moderate concurrency. In single-connection use they're effectively on par. 
FossilOrigin-Name: a9aecc987512d60f2663973f43c769cf086fc14149edfbcb18c0aec9f5aa3dbf
 2026-03-06 23:50:44 +00:00
stephan 2251f0c4cc Get "opfs" and "opfs-wl" properly split. Speedtest1 and the concurrency tester are happy with opfs-wl but it is not yet plugged in to tester1. 
FossilOrigin-Name: 12dcd5425967b6306f3e3178babeac130962edb78e6298d545a429a1a20457cc
 2026-03-06 22:21:06 +00:00
stephan a85d09f1a4 Minor cleanups and docs. Teach the OPFS concurrency tester to deal with SQLITE_BUSY instead of failing. 
FossilOrigin-Name: 247ffed141f66a6a5a396a3e002995a9f00c70333271199200530066e77956c4
 2026-03-06 19:33:25 +00:00
stephan a98f278f58 This one reliably runs 5 workers. Checking in before subsequent cleanups and debug output removal break it. 
FossilOrigin-Name: ba81d95febc5fd0f9bbb2685fef5b1b10f9991751f2bdfafba80c15877af1cef
 2026-03-06 17:10:28 +00:00
stephan c008b8fe3e Correct an internal doc falsehood. 
FossilOrigin-Name: 53aa080e357d7a2ffeab68a3584fda43d51ecef3dc8a1d46dd32392ae4f9740c
 2026-03-06 16:15:32 +00:00
stephan 7b9cd03c0a This Web Lock impl can reliably run a single OPFS connection but rather unreliably 'loses' workers with higher counts, presumably due to deadlock or deadly embrace (how _all_ of them can deadlock at once is unclear, but clearly a bug). 
FossilOrigin-Name: d4e8583e2e80665adfe4e814adb6c219936af1dcac4105795045cb1a7b1e4864
 2026-03-06 16:04:21 +00:00
stephan ad2fe531d0 Expose SQLITE_{DBCONFIG_FP_DIGITS,LIMIT_PARSER_DEPTH,PREPARE_FROM_DDL} to WASM and add support for DBCONFIG_FP_DIGITS to the variadic wrapper for sqlite3_db_config(). These unfortunately went overlooked before the 3.52 release. 
FossilOrigin-Name: 837069635e53849cbca0aa876bad7c7ff44e17578ae492e07acf354067b7e16d
 2026-03-06 15:41:17 +00:00
stephan 544ef18bbd Remove an extraneous OPFS metrics increment. 
FossilOrigin-Name: bf3548a37712e848c7a9cadfdc1669a2be572ea0a0c28d84c157ab30f8c30c44
 2026-03-06 11:49:36 +00:00
stephan 891f5a176b A slight simplification/optimization in the OPFS proxy's waitLoop(). 
FossilOrigin-Name: 521bb140b7ed237c118ac9094732d06907229a6ff385502e850c679bd623fd58
 2026-03-06 09:11:47 +00:00
stephan 831e233202 Reimplement the OPFS async proxy's wait-forever loop to be more async-friendly for upcoming changes. 
FossilOrigin-Name: fd775772fbb50b04c8c37977b90a708784eefed403e3668196c9d95559d3a5e1
 2026-03-06 09:06:24 +00:00
stephan 14b5eadefe Strip the opfs-wl "back to formula", removing the current false starts so that this can be tried again without tripping over any cruft. The current impl is subject, with no obvious way out of it, to starvation in the async proxy. 
FossilOrigin-Name: d022f1f5e74dedae044801330eb099022498f359f408e69b3574885641b312f5
 2026-03-05 11:51:27 +00:00
stephan cef3ff74cd The opfs-wl lock hang has been traced to starvation between the WebLock and the tight wait-on-VFS-calls Atomics.wait() loop. This can reportedly be resolved with another level of indirection in which the WebLock takes over the wait-on-VFS-calls part until it's unlocked, returning to the global loop when it's done. That exceeds this morning's ambitions but is next to try out. 
FossilOrigin-Name: 113bd910e12fea17f9f4a0a3baf706f15627c08cfa6b47a960a83eee761ef4dd
 2026-03-05 06:30:57 +00:00
stephan 671a6f83f4 Add some debugging output to opfs and track down the breakage to the initial WebLock request, which is never reaching its callback. Still broken, but this is progress. 
FossilOrigin-Name: 62fc8b35aeec75f5648b3daa24162c638999d447aa874bdfcbac1431c5c97b6f
 2026-03-04 21:16:36 +00:00
stephan 232446b532 Get opfs-wl plugged in to the concurrency tester. Somewhat ironically, all competing workers fail with locking errors while worker 1 is busy running off the rails somewhere. Stashing for closer investigation later. 
FossilOrigin-Name: 3b27310aa29ea84f459974981a600301abac5c705029a289d3872ecacf231da3
 2026-03-04 20:33:21 +00:00
stephan b85067c568 Consolidate the last 200 lines of common OPFS VFS code. "opfs" still works, "opfs-wl" registers fine but is still otherwise untested. 
FossilOrigin-Name: 5978ee4902e4223fed6b95bd2d8f489bb300af8b762650e7113d1f3e97519d88
 2026-03-04 19:21:09 +00:00
stephan 080da5f56d Factor out about 300 lines of common OPFS VFS bootstrapping code. 
FossilOrigin-Name: 57adecbab71795b62b1c2e4570ff504f35681e81dd8c94f78ad8e05ef39d36fd
 2026-03-04 17:54:02 +00:00
stephan 691494a93b Consolidate the OPFS VFS's metrics-tracking code. 
FossilOrigin-Name: b71c79ef9672c77a72a976ffcd7cbebfaf0ff314dff97b274f7d092de6a7773f
 2026-03-04 16:30:51 +00:00
stephan 6ff9fcfad2 Baby steps in consolidating common OPFS VFS code. 
FossilOrigin-Name: b0dd23299e97ff975f213cb3a8b051f4d7b785b29def82e01f53427fdf77ecb6
 2026-03-04 14:33:33 +00:00
stephan 96daad10fa Consolidate much of the OPFS utility code into a new file for use by two of the OPFS VFSes. 
FossilOrigin-Name: db19a6e9663c3a44996178cb8c35dc4ccd60f48cb4b81b6c214411a56c57def7
 2026-03-04 11:37:39 +00:00
stephan 6d6bf30735 A potential fix for the probable breakage of bundler-friendly builds in the previous check-in. Pending review from someone who uses those tools. 
FossilOrigin-Name: 8ea85776116521526d684f221d67e288126e62931d4a0ea7fc7f164cd2d5b2ec
 2026-03-04 00:48:03 +00:00
stephan 9ac95d9ce1 opfs-wl is now loading and registering but it's still untested. 
FossilOrigin-Name: 9a471f7491a371052bf1785098ba966dd0d03503e7d8b9fbcd65f07b038e5021
 2026-03-04 00:37:55 +00:00
stephan a2d3a54f80 An initial attempt to plugging opfs-wl into the build but its initial handshake with the async half collides with the opfs VFS's handshake, causing bootstrapping to fail miserably. We'll need to either devise a handshake which can differentiate between the two instances or we'll need to preprocess sqlite3-opfs-async-proxy into two copies. Move the (now) three copies of some common code shared by the opfs pieces into a preprocessor #include. 
FossilOrigin-Name: 1e0b72631aecb0bb72f4089116da221e6c4abf962db589de08132cd52c2be0e2
 2026-03-03 23:43:40 +00:00
drh 5281d98a2a Merge the latest trunk enhancements into the begin-concurrent branch. 
FossilOrigin-Name: 81bde19bb9bb0be9eddb5c4c564a1a6ee3f612b41ff5289466f09ae1d62e9206
 2026-03-03 23:01:51 +00:00
stephan 0b7b802d83 Add a missing reset of the lock handshake SharedArrayBuffer slot. 
FossilOrigin-Name: 5910ed7ac843aa1fdb103ddcfaf6270d12cb39199205fc0b9f721fbf7fa2f851
 2026-03-03 21:46:44 +00:00
stephan e2309eebd7 Implementations of WebLock-based opfs::xLock() and xUnlock(). Still completely untested and not yet integrated into the build. 
FossilOrigin-Name: 3343e3aabe465f4ab91dd148dfc5a60346e9afb560c10d1f92aeae98763ec3ce
 2026-03-03 21:07:46 +00:00
stephan d016fad5a8 Merge trunk into the opfs-wl branch. 
FossilOrigin-Name: 81bc4b3b6abc19f98d1f3b1065c4b39a42620a0d7abebe98605dca62dd2d6f9a
 2026-03-03 18:34:59 +00:00
stephan a2fef84b19 Expose sqlite3_bind_zeroblob() to JS/WASM, per request in [forum:e7acf225f9|forum post e7acf225f9]. This is currently missing tests and is not planned for merge until the 3.53 dev cycle starts. 
FossilOrigin-Name: b0c2ef6046b53b6133fed6a890d2ab5e47eeebd7b0b32849b5f69ae54a03014c
 2026-03-03 01:48:18 +00:00
stephan 57d7a20ce7 Another nano-optimization in kvvfs v2. 
FossilOrigin-Name: a9d60190d693504742a894918f145111908d7d6cb9e853ba7a2356529046372a
 2026-02-27 06:33:28 +00:00
stephan d22d0edc48 JS: slight performance improvements in KVVfsStorage. 
FossilOrigin-Name: 56edf2addb155dbb60269fe563ecb5f6132672c6c61f6f9d18ebdff72111298d
 2026-02-27 03:21:47 +00:00
stephan 8c6bf948b7 An initial copy/paste stub for experimenting with using WebLocks for a new copy of the OPFS vfs. This is not yet functional, or even loading, but needs stashing to avoid potential loss. 
FossilOrigin-Name: 372f18ae909bafd7167b70051935832e3107ede72aaeccbf5c042db7ba791912
 2026-02-27 03:11:40 +00:00
drh 9331d99ca6 Merge the latest trunk enhancements into the begin-concurrent branch. 
FossilOrigin-Name: 8c043820d33c19ed8621e26de70655d1bc4c13103427a36f056f439a7fbb98b3
 2026-02-17 16:16:27 +00:00