Commit Graph

1057 Commits

Author SHA1 Message Date
stephan 3262f769ee In fiddle, remove the default .dot-commands it runs, as they do not reflect current shell defaults.
FossilOrigin-Name: 27b6aeda5d70e3480b1448b06277a19395b30a205f3f3bac5ce0e4102e3849ed
2025-12-10 16:20:44 +00:00
stephan 567dcbb5db Update the added-in-version stamp to some kvvfs JS APIs.
FossilOrigin-Name: 939275c6c574a3354e0626164aab1a005998f811c53a66b078249d6b2fcbd68e
2025-12-08 13:11:17 +00:00
stephan 91479c6b10 Introducing kvvfs v2 for the JS bindings. Summary: no longer hard-coded to session/localStorage, available outside of the main UI thread (non-persistently), a simpler-to-use import/export API, and adds an asynchronous event interface intended for creating page-by-page db backups.
FossilOrigin-Name: ec866b04d088e53b09764d00f5403902785328fd8f3cbf80d3affa166477711d
2025-12-08 13:06:27 +00:00
stephan b6306586ec kvvfs: make the filename suffixes -wal and -shm illegal and teach xAccess() to silently not fail for illegal names (else it breaks things when a -wal file is checked for).
FossilOrigin-Name: 2ea29c77a85236ca4126c05c7fd1d45a80dfe3f653af9b3ed86d6e62877ec5af
2025-12-02 18:28:22 +00:00
stephan b58b45d48e Get kvvfs::xAccess() fleshed out.
FossilOrigin-Name: d947e9bed5affee3f6e54b0fe3e2b1a0ac3771865b93c06a4087318f26d87f7f
2025-12-02 18:10:53 +00:00
stephan 6993f52d94 Consolidate kvvfs's page size workaround into a central flag and add some logging to facilitate testing and debugging.
FossilOrigin-Name: 8d683130d2f2aaf6300a8c858802eb671e831ed520621af59c9e830e87a9d753
2025-12-02 16:29:28 +00:00
stephan 22b1bc9997 Add more logging to kvvfs to try to trace down why it cannot recover from a page size change.
FossilOrigin-Name: 0bf0b8a98a2cc5128aa0e510ef2fe411a6859ce807d6159175f5eaf3bc14183b
2025-12-02 15:47:32 +00:00
stephan 9d6b913098 Begrudingly allow sqlite3_js_kvvfs_clear() to work for opened storage only for the local/session storage cases and only (one of the demo apps reminds me) for backwards compatiblity. Likewise, the JsStorageDb.clearStorage() inherited method is now deprecated because it requires an opened db (but continues to work for those two stores), whereas its 'static' method of the same name is unaffected. Rename kvvfs.size() to kvvfs.estimateSize().
FossilOrigin-Name: 09e9255828ed6a7ccbe18f701f4a88b859cfdbd1ddca7ac5dac09744542c25fa
2025-12-01 22:20:54 +00:00
stephan f1d5e9e913 Harden a change to SqliteTestUtil.assert() in the previous checkin to handle a null second argument.
FossilOrigin-Name: 9cbd5be803aba28916440cfa70c54588e7891d2073caebfa2b4dfcdd434b5c49
2025-12-01 21:14:11 +00:00
stephan 2463c80b54 Minor JS docs and cleanups. Add some #if'd-out kvvfs and oo1 pieces to experiment with later.
FossilOrigin-Name: 3db29b00db8d75c6b6a909a46a5c71515d3407af27ff9cb79091f293e4005ce3
2025-12-01 21:10:16 +00:00
stephan 4b6e26c29f Fix a potential UAF in sqlite3.oo1.DB.exec()'s teardown pieces.
FossilOrigin-Name: 7e99e93cddeba555836206a278c5dcfd8565cc2a486a83cffab64dad168e9464
2025-12-01 19:19:06 +00:00
stephan db94b3c0d7 Relax the name limits on kvvfs dbs.
FossilOrigin-Name: 9901cf8e4a00ea9a199a3fb54bd58bd66cff4d02c55433d55f2b417e188e49e0
2025-12-01 16:28:55 +00:00
stephan c281beded3 Reformulate sqlite3-wasm.c's exports so that they export properly with the wask-sdk compiler.
FossilOrigin-Name: d71849958aabdb05225be18d6bc46699cfda9de67c7105b11c3f79d1d01f47d4
2025-12-01 16:25:53 +00:00
stephan b8f8d021c9 Teach kvvfs to handle a NULL db file name by generating a random name, and to honor the SQLITE_OPEN_CREATE flag.
FossilOrigin-Name: 104291469169bef0c2ec5aee9c1cc505447541801bc822e6d5fe5360aef6a2e4
2025-12-01 15:49:31 +00:00
stephan 7c74a3fba5 Fix filtering of the 64-bit speedtest1 wasm test pages to again load the 64-bit wasm files.
FossilOrigin-Name: f507a8af41228d4ca332ab27e842767016ca18adda92e27a1e02e78f12c9be79
2025-11-30 18:59:44 +00:00
stephan 8ff58b4f41 kvvfs docs and 64-bit fixes.
FossilOrigin-Name: cf58e17fa2dc2e183a6ea1d41795c701efb303c9b378aa9b90953c9b568c621a
2025-11-30 18:43:59 +00:00
stephan 4694b91988 Optimize out a kvvfs event notification call in the common case where there are no listeners.
FossilOrigin-Name: 8405c19d32f1e8b7273953a038f8bdfd4ea1a6548300bac5421cdf6fc6840285
2025-11-30 16:37:47 +00:00
stephan 316020580e Rename kvvfs's 'localThread' storage object to '.'. Swap kvvfs.listen()'s elideJournal(=false) option with includeJournal(=false) (i.e. opt in instead of opt out).
FossilOrigin-Name: 7f0eca9d0aeb49c86a785ae930d235902bbd0f15877cc8f6083daac8efb2d1c1
2025-11-30 08:32:52 +00:00
stephan 9dd6979f3e Add sync events to kvvfs.listen() so that streaming can maybe get a better sense of when it's okay to process the stream.
FossilOrigin-Name: a4b6dadffa1b43aadebfa06a1de14d763efd18b16c4a3e87f1bd039fbff2524d
2025-11-30 07:08:45 +00:00
stephan 3dbe1f2f96 More work on the kvvfs events. Demonstrate re-importing a streamed-out set of db pages.
FossilOrigin-Name: b8fee839b0199d1d2714c99612ea4b4d943d8b0033d742eaa66e1575ce46f0d2
2025-11-30 06:20:28 +00:00
stephan b4397ec831 Give kvvfs.unlisten() a useful result value.
FossilOrigin-Name: bbd08b67ceeaae2a3a82655da5a3983f4e9057424de4223f1505ba74c36238af
2025-11-30 05:25:44 +00:00
stephan 9f83f4c563 Extend the kvvfs.listen() config to enable posting of raw binary db pages instead of the kvvfs-encoding. This is much more expensive but was added to... Demonstrate basic async streaming of kvvfs db page-level changes via logging of kvvfs write/delete ops.
FossilOrigin-Name: 0f2bad285577c26f1185dcafd3b8ca2f16e74aa9dc40e6e23867150bccee4602
2025-11-30 05:20:17 +00:00
stephan 768c364aaa Extend kvvfs export to optionally export the raw binary db pages as a list of Uint8Array instead of kvvfs-encoded strings. This is typically much larger but the pages can then be used as-is.
FossilOrigin-Name: a4f59496a53a079f8f73e4cde68f47dbd13d2d74de2ad11bc716e7e5c00f1ec0
2025-11-30 03:02:06 +00:00
stephan e8416526f0 Fix kvvfs event notification to notify _all_ listeners and optimize better for the common case of no listeners. Squelch some debug output.
FossilOrigin-Name: 989d097b1324cf712107bb81697fa8e9044aea0e7feacf0e6b6561e216f07989
2025-11-29 23:29:30 +00:00
stephan f2ec1d2560 Add a WASM-side export for kvvfsDecode() for pending private use in the kvvfs export bits and ensure that it consistently returns -1 for malformed input.
FossilOrigin-Name: cd81cb70525edeb25c4ab287cd5fd03db08ec45fd9e4001cad5c3eadf11505ae
2025-11-29 23:29:07 +00:00
stephan 4719c67753 More tests for the kvvfs vtab.
FossilOrigin-Name: 0358a99eedb503aaf8eff32a3143a2591081dfd3fd9eeecc51c601c98c714d6f
2025-11-29 21:19:35 +00:00
stephan facbf34cbf Add a kvvfs vtab for inspecting the storage state. This is for testing and my own edification, not part of the public API.
FossilOrigin-Name: eaff2e8b7f63575e85fb9be800a2c59163e402d6a25fc6a955b9df15afe13b92
2025-11-29 16:41:27 +00:00
stephan c4a22d6c14 Merge trunk into the kvvfs-v2 branch.
FossilOrigin-Name: 0e1a6ca0f1123da680918027b16608bf87593a9d91f7b1f3fc0c3a454185740d
2025-11-28 22:17:02 +00:00
stephan 79837b6ee5 Correct a doc falsehood - jquery.terminal does not require pre-building.
FossilOrigin-Name: 4384c9a108b58a0b8c38c51678aad871f088358b9bff3922299cc7ddb3d247ce
2025-11-28 17:56:22 +00:00
stephan 2c881d45dd Fiddle build: if JQTERM is set to the dir name of a built jquery.terminal checkout then copy the pieces fiddle needs to enable that input mode. By default it looks in ~/src/jquery.terminal.
FossilOrigin-Name: 9dd16f8e3b8e181ff138b4061c9dbc116cbc6f85ee867a97cd8af6e9e874c7d1
2025-11-28 17:50:29 +00:00
stephan 961665682b Minor doc updates.
FossilOrigin-Name: 66b0f28958311204270053ef1d5e66c0ad04e3a52f2570c543b6d319d75798fa
2025-11-28 16:23:24 +00:00
stephan f26c99005a Rename c-pp::ARGV to c-pp::argv and remove its use from the license header (where it doesn't get applied).
FossilOrigin-Name: a2e18d299c8fd7360de246fdc3a2f92025bf321e06c5baf6032ba98854733e60
2025-11-28 02:34:08 +00:00
stephan 973340c681 Minor kvvfs error handling cleanup.
FossilOrigin-Name: 5ec20ebe057af297f5e0ccc0d7184ab0ed720f91db0f470a7cf827df0f2aa530
2025-11-26 20:20:40 +00:00
stephan 154e1082f1 Teach kvvf's xClose() and xFileControl() to return more specific result codes for exceptions.
FossilOrigin-Name: 6b48375972e6e4112efd51ffea0e54d33a4245bf0155b7fa3ca1e3f147eb043c
2025-11-26 20:18:11 +00:00
stephan a4cb56e425 kvvfs: silently reject attempts to change the page size, as changing it corrupts this vfs for reasons not yet understood. Add more kvvfs tests.
FossilOrigin-Name: 65272adfab333565719bde4f6c5bbe4f0b5415c17a8de43e6b2d7b0518cca4c7
2025-11-26 20:12:29 +00:00
stephan a5f1fc25e3 Factor out an obsolete malloc failure check and an snprintf() for keys in transient kvvfs storage.
FossilOrigin-Name: e2e94f9094f8bbe93cdf7d2a2e72e92462b94e18319603c5a364f183cb780be1
2025-11-26 17:19:44 +00:00
stephan 2348682e4e Fix tests broken by the previous commit.
FossilOrigin-Name: 430d4d668fc2d151f20a02ddbbac7f884258af7fd1268d3aafb5f9e8eadf844b
2025-11-26 16:40:23 +00:00
stephan 4ad869aa3f Remove some dead code. Resolve the remaining kvvfs v1/v2 API compatibility confusion by eliding the v1 API from Workers like we've always done. Add some tests.
FossilOrigin-Name: 450b4bcca6038be58a73ff15ff18b9d70df661fe0bd9777273e0db6fbce5f296
2025-11-26 16:27:11 +00:00
stephan 0e4ea1821f Wasm speedtest1: replace some 'self' with 'globalThis' and expose the sqlite3 object to the global scope for inspection via the dev console (for inspecting resulting kvvfs storage objects).
FossilOrigin-Name: c18154bc9a7cf07d3902d74d27b4ca696929aae9e1abe031679bd5a42d759046
2025-11-26 16:26:27 +00:00
stephan 0baab04f8b Minor optimization in kvvfs event notification.
FossilOrigin-Name: fd85b96b12d164f7700624dba478d35e886b23f5c31c853715f783639ea95f23
2025-11-26 14:22:51 +00:00
stephan a781caa9ad Internal kvvfs docs and cleanups. Wasm speedtest1: remove the hard-coded --size=2 limit for kvvfs and use a transient storage object instead of session storage (which is limited to --size=2). For small job sizes kvvfs is roughly 3x the speed of memdb and that gap grows as the --size increases, with kvvfs _really_ bogging down on the --size=50 jsonb tests.
FossilOrigin-Name: 43173267849f7f4b77a9ee15ab586e9deb0e290071149857fe18421fb948a715
2025-11-26 13:55:20 +00:00
stephan 6003505a39 Cherry-pick JS fixes/cleanups/docs from [206275292217], [256c1dd2c367], and [dec0c6073cc1] from the kvvfs-v2 branch into trunk.
FossilOrigin-Name: cdd3e700f59e5c30241cd5c6d9ea31f6a5f1c7130c6341b4e0b6dc32b06f3c79
2025-11-25 19:29:23 +00:00
stephan f37923e723 Guard access to a piece of Emscripten-related scaffolding so that sqlite3-api.js does not hit it when imported into other builds. Reported in [forum:6eacbb9c447a5665|forum post 6eacbb9c447a5665].
FossilOrigin-Name: cb9cf028fcefa8ec51415b5dac886c2443b2ed207fa388dbc3d725dcc85fe092
2025-11-25 19:20:15 +00:00
stephan 1da8e7ac4f Remove some debug output.
FossilOrigin-Name: 9fc6b811d6b0e268be18b768b6a69381606557a1702064b29f0ff4ca7c505cac
2025-11-25 19:06:05 +00:00
stephan 20e1f8a289 Remove the ostensible support for empty kvvfs names, as those names apparently never reach kvvfs.xOpen (the empty name is special). That actually simplifies things with regards to backwards compatibility.
FossilOrigin-Name: a5c61fa56770a6fffcb4c76db2256c71b2d5f0beee6fc2c4800c40bf7ef995c8
2025-11-25 19:05:13 +00:00
stephan f0ca498c61 Improve kvvfs file name validation. Add sqlite3.kvvfs.unlink().
FossilOrigin-Name: 0dfdf4681cf63541de971a20be21b33d0d3b38e8281f302d20aca9492df3da42
2025-11-25 18:52:08 +00:00
stephan e62055f274 Add the sqlite3.kvvfs namespace for the new kvvfs APIs instead of adding more sqlite3_js_kvvfs_...() methods. Reinstate that clearing kvvfs storage is illegal when db handles are opened, solely for sanity's sake (they can actually recover from that but supporting such use feels ill-advised).
FossilOrigin-Name: 02793c5905e6b99379cd5ad6bfe1eb6cccf839da081fc174dce7b06245e212fb
2025-11-25 16:41:56 +00:00
stephan 27aec2d1c1 Add an explicit rollback test for kvvfs.
FossilOrigin-Name: 2bf31ef8027a3e15887d4dcd26fe09463b5f8852c5ce443f7d07c23d29c37311
2025-11-25 16:07:05 +00:00
stephan 30f9c25d9a A more compact formulation of [ac8af73c16ab821] which uncovered two remaining direct calls to kvrecordRead(), now replaced with calls through the thunk. Fix a typo which caused the storage object name to not be passed on to event listener.
FossilOrigin-Name: d391f1ce691fca4504c21a7e3e05a029f247da23d432ee34b0dae14be51d909c
2025-11-25 15:59:58 +00:00
stephan 68230ef670 kvvfs internal cleanups. Experimentally add async event listeners to kvvfs to explore backing up a kvvfs a page at a time.
FossilOrigin-Name: f355fd484947a645206c9b9c2fd6fe691455dece7fb1aa5b72cb51a86b39474f
2025-11-25 05:44:03 +00:00