paulhorn/sqlite
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
24,660 Commits 1,696 Branches 387 Tags
5e8bb0aa13628208b7baedd720ba4aaaac2cc239
Commit Graph

3228 Commits

Author SHA1 Message Date
stephan 5e8bb0aa13 An alternative messaging strategy for the OPFS VFS proxy which uses only SharedArrayBuffer and Atomics, instead of worker messages, for communication (only the initial one-time handshake during initialization uses worker messages). It runs speedtest1 approx. 15-20% faster but still 20-ish% slower than WASMFS. 
FossilOrigin-Name: a83ee3082d89439ea3ad5737e63e25bebb0f91895aca006ce5fecf5b93a2651a
 2022-09-20 08:27:57 +00:00
stephan 138647a552 OPFS VFS: further internal refactoring towards experimenting with a new comms model. 
FossilOrigin-Name: 5ca412ced24b4e3af5f467e710a597ed440badf7b8335346aade11d3cad3d1a1
 2022-09-20 03:31:02 +00:00
stephan c4b87be3e8 OPFS VFS: moved i/o buffer from per-file to the VFS, and related refactoring, in prep for experimentation with a new inter-worker comms model. 
FossilOrigin-Name: d4d63e4580ad8d497310608175308c03c517e051d7865cb66aa0b10356612d7d
 2022-09-20 01:28:47 +00:00
stephan aec046a264 Further metrics and buffer-copy optimizations in the OPFS proxy, but with little effect. 
FossilOrigin-Name: d1f1fe6f1c60640f7770dfb9245c459a09b8d24ec2ddf664dff77c810bd51f96
 2022-09-19 18:22:29 +00:00
stephan f815011a08 Cut the speedtest1 runtime of the OPFS VFS proxy by approximately 3/4ths via xRead/xWrite buffer-copying optimizations. Still slower than the WASMFS impl by approx. 1/5th. 
FossilOrigin-Name: fb7f287310d74a3e236125ae9c49b859f9263c29ae85161c1bcf9dd0778d8a51
 2022-09-19 17:09:09 +00:00
stephan 61418d5a67 Remove an obsolete file. Update wasmfs build for today's changes. Fix the case that OPFS.xTruncate() is passed a BigInt instead of a Number. 
FossilOrigin-Name: 976053925013bf1975f5f9222e28ba648af28e305bb6bdae600eb24d0e136bec
 2022-09-19 14:56:13 +00:00
stephan 0e0687ccfc Add sqlite3_web_vfs_list() to JS API. Corrected OPFS VFS's registering itself as the default VFS. speedtest1-worker now uses the xDelete() of both the default VFS and OPFS, to avoid that it starts up with a persistent OPFS test db (the native app calls unlink(), but that unlink call operates on a different virtual filesystem than the OPFS VFS). 
FossilOrigin-Name: 2ec7e09139a510b9fd29e4c97283b20740a00f369193c6fecbb734f187e81b48
 2022-09-19 13:44:23 +00:00
stephan b5ae85eca2 Merge kv-vfs branch into fiddle-opfs. Adjust various JS APIs and apps to deal with the new method of handling kvvfs. Adjust speedtest1 JS build to include sqlite3-api.js so that it can use kvvfs and opfs VFSes. Permit passing of the vfs as a URL parameter to certain demo/test apps. Milestone: speedtest-worker.html?vfs=opfs runs with the standalone OPFS impl. 
FossilOrigin-Name: ec09f32f7ae2249aaf27388ad2062982afa8bbbb5f88d236d6d9068bf33ad93d
 2022-09-19 13:16:35 +00:00
stephan ba0b157583 Minor doc improvements in demo-123.js. 
FossilOrigin-Name: 9370ce94780e974da03d5c74cf9a5983eed333e7b3bc54e605c5f30b3fa99287
 2022-09-19 10:13:01 +00:00
stephan 509f40526e Generic cleanups in the OPFS VFS proxies. 
FossilOrigin-Name: 7d5f6adc3b964413fc96ad8d2735312c3e58348024cabdd2099682cbf696eaf7
 2022-09-19 09:58:01 +00:00
stephan 862281fc47 Correct OPFS VFS xRead() to copy the result buffer if the result code is SQLITE_IOERR_SHORT_READ. 
FossilOrigin-Name: 56668f9902c6e896b6c63621a444444c6f58ee20f88a5feae97f1699be35892d
 2022-09-19 09:25:25 +00:00
stephan 8766fd20d1 Replace OPFS VFS xSleep() impl with a more efficient one (no Worker round-trip needed). 
FossilOrigin-Name: b9773f164878b0a1b7c88cc7a6d1374ea95f64920065e8b2b178a1afffd82fe5
 2022-09-19 05:19:04 +00:00
stephan ac51eb7754 Rename demo-oo1.* to demo-123.* and add demo-123-worker.html, which runs the same demo via a Worker. Doc typo fixes. 
FossilOrigin-Name: 2e4a005bd35424caeaa99ace23162cf79e2ebdb159475ffad92b85dc864ad764
 2022-09-19 03:57:31 +00:00
stephan ed85333977 Disable build of wasmfs-using components on aarch64 (ARM), as wasmfs won't build there. 
FossilOrigin-Name: 9a9eeebc2c27b734041089131b4952d7c0440df48ef32f355641aca61d4b30a0
 2022-09-19 00:40:53 +00:00
stephan f386012069 Numerous cleanups in the JS bits. Removed some now-defunct wasm test files. Expose sqlite3.opfs object containing various OPFS-specific utilities. 
FossilOrigin-Name: 26e625d05d9820033b23536f18ad3ddc59ed712ad507d4b0c7fe88abd15d2be8
 2022-09-18 17:32:35 +00:00
stephan 0db3089576 Move the OPFS VFS bits back into api/sqlite3-api-opfs.js. Refactor the OPFS VFS init process to use a Promise-returning function which the client must call, as that eliminates any uncertainty about when the VFS (necessarily activated asynchronously) actually becomes available to the client. Rename x-sync-async.* to test-opfs-vfs.* Milestone: first successful test of OPFS without WASMFS. 
FossilOrigin-Name: b2abf60dbfa6648f671a3932cb65feb28d05a0d5b7f792351d14f9c13d9798c5
 2022-09-18 03:05:55 +00:00
stephan c5313afea7 Move the OPFS VFS bits back into api/sqlite3-api-opfs.js. Refactor the OPFS VFS init process to use a Promise-returning function which the client must call, as that eliminates any uncertainty about when the VFS (necessarily activated asynchronously) actually becomes available to the client. 
FossilOrigin-Name: 1c660970d0f62bcfd6e698a72b050d99972a1e39f45a5ac24194a190f8f78ab3
 2022-09-18 02:35:30 +00:00
stephan 3e2823cbb6 Plug OPFS methods in to their sqlite3_vfs/io_methods counterparts. Add URL args to control debug output and running of sanity-checks in the OPFS init code. 
FossilOrigin-Name: a0e93ed20b2463606a63b03ce8ca41ec1fb22886db5c5c898ace86ba24636f70
 2022-09-18 00:16:12 +00:00
stephan 5f4ad92681 Correct OPFS xRead() and xWrite() impls and add a very basic sanity test for them. 
FossilOrigin-Name: cd06cc670029763955cf60ffcf944b36d41cb005b859d9b9fd0eea1b6741d0e9
 2022-09-17 23:47:20 +00:00
stephan 8200a6d8d7 Implement OPFS xAccess(), albeit with more limited semantics than the VFS API calls for. Add a way for OPFS xDelete() to optionally recursively remove empty dirs left over after deleting a file. 
FossilOrigin-Name: c342b5d745f301104c59851c753287ebbbe95a69a56cb522d376d0f3e352c30f
 2022-09-17 23:29:27 +00:00
stephan e6f8a09532 Generic cleanups in the OPFS sync/async proxy. 
FossilOrigin-Name: f36bddbe54c3acbfaa958042e4d24724f130bdca551401033f9bc63f3da73492
 2022-09-17 21:13:26 +00:00
stephan 0731554629 Add the remaining vfs/io_methods wrappers to the OPFS sync/async proxy, but most are not yet tested. 
FossilOrigin-Name: 44db9132145b3072488ea91db53f6c06be74544beccad5fd07efd22c0f03dc04
 2022-09-17 20:50:12 +00:00
stephan 132a87baa3 Add initial bits of an experimental async-impl-via-synchronous-interface proxy intended to marshal OPFS via sqlite3_vfs API. 
FossilOrigin-Name: 38da059b472415da52f57de7332fbeb8a91e3add1f4be3ff9c1924b52672f77c
 2022-09-17 15:08:22 +00:00
stephan ce260f4e43 Merge kv-vfs branch into fiddle-opfs branch. Adjust speedtest1 --size flags to account for new size limit. 
FossilOrigin-Name: afb79050e635f3c698e51f06c346cbf23b096cfda7d0f1d8e68514ea0c25b7b7
 2022-09-16 20:16:50 +00:00
stephan 81439a07f0 When calculating kvvfs sessionStorage/localStorage size, multiply it by 2 to account for JS using 16-bit characters. 
FossilOrigin-Name: 52d1b185b9f6cee1eb3dec436f47e0f52e4621a127abfad8c27f92fd78147889
 2022-09-16 12:49:45 +00:00
stephan 4c72171b60 Merge kv-vfs branch into fiddle-opfs branch for [21915af560b1|synchronous=off fix]. Remove some duplicate debug output in OPFS test code. 
FossilOrigin-Name: 13899bb98c80525276d2484598b94e4206358f243f06d45c02700024f7e226fd
 2022-09-16 11:45:09 +00:00
stephan 57db2174d5 Add batch-runner-kvvfs.html, a kvvfs-specific build of batch-runner.html. Reduce the speedtest1 --size X value for the batch list generation to 5 so that the kvvfs batch runner can handle it. 
FossilOrigin-Name: d8df25920a047d5cf2093cc6233128c5e6057a9104d0c4397e643645bd646fe1
 2022-09-16 02:30:49 +00:00
stephan 58bcfe075d Remove --nosync flag from speedtest1-kvvfs.html, as that seems to be the trigger for the 'no such table: z1' error. That app now warns if that flag is passed to it via URL args. 
FossilOrigin-Name: ad0677e8abcc077636d1cf1d8485be4943506382581edf832e6b8a2021560040
 2022-09-16 02:28:03 +00:00
stephan 0144a8482a Add return value checks to kvstorageWrite(), necessary for recognizing storage-full errors in the JS binding. speedtest1-kvvfs.html runs successfully on localStorage with --size 1 to 6 and starts failing with storage shortage at --size 7. 
FossilOrigin-Name: 13839759f8f45e4eb0aa6a5052801f3964d5b2dc2e427e5e91ee6692176381eb
 2022-09-16 01:05:19 +00:00
stephan 28ef9bddb3 More work on the synchronous OPFS experimentation. Numerous wasm/js build tweaks. Add speeedtest-wasmfs.html, the wasmfs/opfs counterpart of speedtest1.html. 
FossilOrigin-Name: 00ee49a3a2c148480f614e49a0ee5b35a255758c0a53693f0b464b31e7a4045b
 2022-09-15 06:42:41 +00:00
stephan dae7518ae4 Remove some stale comments. Get scratchpad-opfs-worker2.js running again in prep for reuse in experimenting with the OPFS all-synchronous API. 
FossilOrigin-Name: 4cbbd370186f84e440f496e7f11c641d7dc1723acc46e4a31483392e0eb046e9
 2022-09-15 03:16:49 +00:00
stephan 0117693f62 Split wasmfs-enabled build of sqlite3.js/wasm into sqlite3-wasmfs.js/wasm, as enabling wasmfs breaks all tests/demos which run from a Worker thread. 
FossilOrigin-Name: 08476f3c218d45846e7496bdae0b06e2122466111fdf2aa2aabb1805b15ef982
 2022-09-15 03:09:00 +00:00
stephan 409505c743 Clean up some JS documentation. 
FossilOrigin-Name: 925fdbfc6c1f9f06951346bc0d4fb81ffb5797d5f1123728931973f6cb1a0efa
 2022-09-15 02:27:48 +00:00
stephan 5b91500796 Add/apply various kvvfs-specific utility APIs to the JS layer to assist in testing and analysis. Correct a backwards default arg check for sqlite3ApiBootstrap(). Add exports for sqlite3_db_handle(), sqlite3_file_control(), and the SQLITE_FCNTL_xxx enum. 
FossilOrigin-Name: 0d78961870ee9f22f1ba16d423377d28dcc36e04b1e31ffd57f3e2fd51f8f0f2
 2022-09-13 19:27:03 +00:00
stephan 23d39ce54d Get testing1.js working with a kvvfs build. 
FossilOrigin-Name: 333a45725d1708e0fefa559c33ce1c7eeb425cdb04f594b1f2b48166c1478c79
 2022-09-12 22:27:00 +00:00
stephan 0b769f5b4a Merge kv-vfs branch into fiddle-opfs and add speedtest1-kvvfs.html test app (which currently fails early on). 
FossilOrigin-Name: 8b1608a9a37131121cb3a0cb1e8dbad5e39dc1fc2a341d056f09537f179b2e7a
 2022-09-12 20:18:28 +00:00
stephan a002cc174f Add sqlite3.oo1.DB.clearKvvfsStorage(). Add controls to kvvfs1.js demo to reset and query the db without requiring the dev console. 
FossilOrigin-Name: d845c6c22bd5d3fffc66e0566df346d690dd8bd1fc1688e312161b1a1edcfd79
 2022-09-12 17:59:04 +00:00
stephan 5360f5fcff Merge kv-vfs branch into fiddle-opfs branch to add kvvfs-based wasm build and demo. 
FossilOrigin-Name: a7d8b26acd3c1ae344369e4d70804c0cab45272c0983cfd32d616a0a7b28acb9
 2022-09-12 16:09:50 +00:00
stephan 2811759d5e Resolve missing kvvfs JS test: sqlite3_initialize() must be called by the client in this build. Add basic sanity test of kvvfs using sessionStorage(). 
FossilOrigin-Name: 250a935aeb94d3fadec0d3fe22de85de4e658e2fdb3be3aa9a8bbc8f7b7d8414
 2022-09-12 13:05:54 +00:00
stephan ad5125a624 Initial build of kvvfs in wasm. It loads but cannot find the VFS for as-yet-unknown reasons (sqlite3 shell works fine), and most APIs throw "null function or function signature mismatch" from deep within wasm, presumably as a side effect of the "missing" VFS. 
FossilOrigin-Name: 1a2f24a0bdfc6eaae478916b8f4f9c6b63ead9964534fc2951fb4e995ffe61f1
 2022-09-12 12:39:28 +00:00
stephan 73079dba00 Minor cleanups and documentation in the wasm pieces. 
FossilOrigin-Name: 4e6ce329872eb733ba2f7f7879747c52761ae97790fd8ed169a25a79854cc3d9
 2022-09-11 16:59:40 +00:00
stephan 2a91126d76 Correct the result code from kvstorageRead() for the OOM case to be a negative value. 
FossilOrigin-Name: cdbf09fa1b0c93aeb3222a157de33a4688ae629c2b829ffff0f1f62364c5ae1c
 2022-09-11 05:44:15 +00:00
stephan ade4f4d645 Minor cleanups and docs for the EM_JS() impls of the kvvfs read/write/delete ops. 
FossilOrigin-Name: 53b7572e441be4b4b29b4228d0f95b24457d7faaf6c0b568ed5c970a55e21ffb
 2022-09-11 05:38:39 +00:00
stephan e99f742a32 Add EM_JS() impl for kvstorageRead(). 
FossilOrigin-Name: 06610314fcf644f323c2f7ae11d7f4349b195e66d0ebbee590438dd99d97eb96
 2022-09-11 04:49:29 +00:00
stephan d15822965b Initial prototype impls of write/delete-key ops for the pending kvvfs which use sessionStorage or localStorage for db page storage. read-key op is pending. 
FossilOrigin-Name: a4bd96f5348e607d0fcb627b751f9d86a188173a4b3bfb2c95f92913a78bd31e
 2022-09-11 02:43:08 +00:00
drh 7585f49a0a Move the vfskv.c extension to src/os_kv.c and make it part of the amalgamation, 
activated if and only if SQLITE_OS_KV is true.

FossilOrigin-Name: f6632e69c2ec1a7ddc4e51f3567e3bc082ee94a6dd51fdafdc0c3bf386a32d4c
 2022-09-10 18:20:59 +00:00
drh 25f62508f5 Database must be named either "local" or "session". Each has a distinct 
namespace.

FossilOrigin-Name: c8e41279294ea7c2f041c7d4cbbd85fce47d55f2f56180ca837316f7dfd75237
 2022-09-09 16:16:33 +00:00
drh f7377d5763 Separate sqlite3_io_methods objects for database files and journals. 
FossilOrigin-Name: 74fbf6c2a97729e481d5562344e960d0c237aeb1f07240262d32a24fb1f4b5a3
 2022-09-09 14:22:41 +00:00
drh 126976c9f8 Simple reading and writing now works. 
FossilOrigin-Name: 00845ac9ef2616e90f5f6a20da3dd040fa8bdcfe73f72fe9c06039561150a82d
 2022-09-09 11:41:54 +00:00
stephan f5bf66c875 Remove a reference to a completed TODO. 
FossilOrigin-Name: 757ad69607201a26802a304839b8454b1634712458539d6aa8cf999ab2447e59
 2022-09-09 05:39:36 +00:00