5e8bb0aa13
An alternative messaging strategy for the OPFS VFS proxy which uses only SharedArrayBuffer and Atomics, instead of worker messages, for communication (only the initial one-time handshake during initialization uses worker messages). It runs speedtest1 approx. 15-20% faster but still 20-ish% slower than WASMFS.
...
FossilOrigin-Name: a83ee3082d89439ea3ad5737e63e25bebb0f91895aca006ce5fecf5b93a2651a
2022-09-20 08:27:57 +00:00
138647a552
OPFS VFS: further internal refactoring towards experimenting with a new comms model.
...
FossilOrigin-Name: 5ca412ced24b4e3af5f467e710a597ed440badf7b8335346aade11d3cad3d1a1
2022-09-20 03:31:02 +00:00
c4b87be3e8
OPFS VFS: moved i/o buffer from per-file to the VFS, and related refactoring, in prep for experimentation with a new inter-worker comms model.
...
FossilOrigin-Name: d4d63e4580ad8d497310608175308c03c517e051d7865cb66aa0b10356612d7d
2022-09-20 01:28:47 +00:00
aec046a264
Further metrics and buffer-copy optimizations in the OPFS proxy, but with little effect.
...
FossilOrigin-Name: d1f1fe6f1c60640f7770dfb9245c459a09b8d24ec2ddf664dff77c810bd51f96
2022-09-19 18:22:29 +00:00
f815011a08
Cut the speedtest1 runtime of the OPFS VFS proxy by approximately 3/4ths via xRead/xWrite buffer-copying optimizations. Still slower than the WASMFS impl by approx. 1/5th.
...
FossilOrigin-Name: fb7f287310d74a3e236125ae9c49b859f9263c29ae85161c1bcf9dd0778d8a51
2022-09-19 17:09:09 +00:00
61418d5a67
Remove an obsolete file. Update wasmfs build for today's changes. Fix the case that OPFS.xTruncate() is passed a BigInt instead of a Number.
...
FossilOrigin-Name: 976053925013bf1975f5f9222e28ba648af28e305bb6bdae600eb24d0e136bec
2022-09-19 14:56:13 +00:00
0e0687ccfc
Add sqlite3_web_vfs_list() to JS API. Corrected OPFS VFS's registering itself as the default VFS. speedtest1-worker now uses the xDelete() of both the default VFS and OPFS, to avoid that it starts up with a persistent OPFS test db (the native app calls unlink(), but that unlink call operates on a different virtual filesystem than the OPFS VFS).
...
FossilOrigin-Name: 2ec7e09139a510b9fd29e4c97283b20740a00f369193c6fecbb734f187e81b48
2022-09-19 13:44:23 +00:00
b5ae85eca2
Merge kv-vfs branch into fiddle-opfs. Adjust various JS APIs and apps to deal with the new method of handling kvvfs. Adjust speedtest1 JS build to include sqlite3-api.js so that it can use kvvfs and opfs VFSes. Permit passing of the vfs as a URL parameter to certain demo/test apps. Milestone: speedtest-worker.html?vfs=opfs runs with the standalone OPFS impl.
...
FossilOrigin-Name: ec09f32f7ae2249aaf27388ad2062982afa8bbbb5f88d236d6d9068bf33ad93d
2022-09-19 13:16:35 +00:00
5ef7f04775
speedtest1: add --vfs NAME flag and eliminate code duplication in argc size verification for all flags which take a value.
...
FossilOrigin-Name: 281d09867134e0a057cfadd9bfcbf0e21b8ac9737a278c41973d33a3101be7bc
2022-09-19 11:47:38 +00:00
ba0b157583
Minor doc improvements in demo-123.js.
...
FossilOrigin-Name: 9370ce94780e974da03d5c74cf9a5983eed333e7b3bc54e605c5f30b3fa99287
2022-09-19 10:13:01 +00:00
509f40526e
Generic cleanups in the OPFS VFS proxies.
...
FossilOrigin-Name: 7d5f6adc3b964413fc96ad8d2735312c3e58348024cabdd2099682cbf696eaf7
2022-09-19 09:58:01 +00:00
862281fc47
Correct OPFS VFS xRead() to copy the result buffer if the result code is SQLITE_IOERR_SHORT_READ.
...
FossilOrigin-Name: 56668f9902c6e896b6c63621a444444c6f58ee20f88a5feae97f1699be35892d
2022-09-19 09:25:25 +00:00
8766fd20d1
Replace OPFS VFS xSleep() impl with a more efficient one (no Worker round-trip needed).
...
FossilOrigin-Name: b9773f164878b0a1b7c88cc7a6d1374ea95f64920065e8b2b178a1afffd82fe5
2022-09-19 05:19:04 +00:00
ac51eb7754
Rename demo-oo1.* to demo-123.* and add demo-123-worker.html, which runs the same demo via a Worker. Doc typo fixes.
...
FossilOrigin-Name: 2e4a005bd35424caeaa99ace23162cf79e2ebdb159475ffad92b85dc864ad764
2022-09-19 03:57:31 +00:00
ed85333977
Disable build of wasmfs-using components on aarch64 (ARM), as wasmfs won't build there.
...
FossilOrigin-Name: 9a9eeebc2c27b734041089131b4952d7c0440df48ef32f355641aca61d4b30a0
2022-09-19 00:40:53 +00:00
f386012069
Numerous cleanups in the JS bits. Removed some now-defunct wasm test files. Expose sqlite3.opfs object containing various OPFS-specific utilities.
...
FossilOrigin-Name: 26e625d05d9820033b23536f18ad3ddc59ed712ad507d4b0c7fe88abd15d2be8
2022-09-18 17:32:35 +00:00
0db3089576
Move the OPFS VFS bits back into api/sqlite3-api-opfs.js. Refactor the OPFS VFS init process to use a Promise-returning function which the client must call, as that eliminates any uncertainty about when the VFS (necessarily activated asynchronously) actually becomes available to the client. Rename x-sync-async.* to test-opfs-vfs.* Milestone: first successful test of OPFS without WASMFS.
...
FossilOrigin-Name: b2abf60dbfa6648f671a3932cb65feb28d05a0d5b7f792351d14f9c13d9798c5
2022-09-18 03:05:55 +00:00
c5313afea7
Move the OPFS VFS bits back into api/sqlite3-api-opfs.js. Refactor the OPFS VFS init process to use a Promise-returning function which the client must call, as that eliminates any uncertainty about when the VFS (necessarily activated asynchronously) actually becomes available to the client.
...
FossilOrigin-Name: 1c660970d0f62bcfd6e698a72b050d99972a1e39f45a5ac24194a190f8f78ab3
2022-09-18 02:35:30 +00:00
3e2823cbb6
Plug OPFS methods in to their sqlite3_vfs/io_methods counterparts. Add URL args to control debug output and running of sanity-checks in the OPFS init code.
...
FossilOrigin-Name: a0e93ed20b2463606a63b03ce8ca41ec1fb22886db5c5c898ace86ba24636f70
2022-09-18 00:16:12 +00:00
5f4ad92681
Correct OPFS xRead() and xWrite() impls and add a very basic sanity test for them.
...
FossilOrigin-Name: cd06cc670029763955cf60ffcf944b36d41cb005b859d9b9fd0eea1b6741d0e9
2022-09-17 23:47:20 +00:00
8200a6d8d7
Implement OPFS xAccess(), albeit with more limited semantics than the VFS API calls for. Add a way for OPFS xDelete() to optionally recursively remove empty dirs left over after deleting a file.
...
FossilOrigin-Name: c342b5d745f301104c59851c753287ebbbe95a69a56cb522d376d0f3e352c30f
2022-09-17 23:29:27 +00:00
e6f8a09532
Generic cleanups in the OPFS sync/async proxy.
...
FossilOrigin-Name: f36bddbe54c3acbfaa958042e4d24724f130bdca551401033f9bc63f3da73492
2022-09-17 21:13:26 +00:00
0731554629
Add the remaining vfs/io_methods wrappers to the OPFS sync/async proxy, but most are not yet tested.
...
FossilOrigin-Name: 44db9132145b3072488ea91db53f6c06be74544beccad5fd07efd22c0f03dc04
2022-09-17 20:50:12 +00:00
a50d3b7a5d
Fix an uninitialized variable in the decoder kv-vfs.
...
FossilOrigin-Name: 354726aa6c399053785f29104de15091629ce4bc275b9e2205cb3656a9e81cd7
2022-09-17 18:31:31 +00:00
20a9ed1dc6
Include the kv-vfs as an optional VFS on unix builds if the
...
SQLITE_OS_KV_OPTIONAL compile-time option is present.
FossilOrigin-Name: 852812d1e2ec3c53ad7c6c64662b37d861fefcf1baeee3d58eba88bcb3f6d8df
2022-09-17 18:29:49 +00:00
b9fbc558f7
Merge the latest trunk enhancements into the kv-vfs branch.
...
FossilOrigin-Name: 37c7ca25a6e837e7f83b96a32e2b1a455730e48a48012c458ae2e734a3ccdd1f
2022-09-17 17:26:44 +00:00
132a87baa3
Add initial bits of an experimental async-impl-via-synchronous-interface proxy intended to marshal OPFS via sqlite3_vfs API.
...
FossilOrigin-Name: 38da059b472415da52f57de7332fbeb8a91e3add1f4be3ff9c1924b52672f77c
2022-09-17 15:08:22 +00:00
ce260f4e43
Merge kv-vfs branch into fiddle-opfs branch. Adjust speedtest1 --size flags to account for new size limit.
...
FossilOrigin-Name: afb79050e635f3c698e51f06c346cbf23b096cfda7d0f1d8e68514ea0c25b7b7
2022-09-16 20:16:50 +00:00
756440febc
Corrections to the xRead method for databases in the os_kv.c VFS so that it
...
correctly reads the header, and thus avoids unnecessary cache flushes. This
changes also includes an optimization to the header read logic.
FossilOrigin-Name: ef54961ce69fddb4cfeeff0860288de2858a6f7a5aa396691e8e99933eb9af54
2022-09-16 15:19:04 +00:00
81439a07f0
When calculating kvvfs sessionStorage/localStorage size, multiply it by 2 to account for JS using 16-bit characters.
...
FossilOrigin-Name: 52d1b185b9f6cee1eb3dec436f47e0f52e4621a127abfad8c27f92fd78147889
2022-09-16 12:49:45 +00:00
4c72171b60
Merge kv-vfs branch into fiddle-opfs branch for [21915af560b1|synchronous=off fix]. Remove some duplicate debug output in OPFS test code.
...
FossilOrigin-Name: 13899bb98c80525276d2484598b94e4206358f243f06d45c02700024f7e226fd
2022-09-16 11:45:09 +00:00
84e5076787
Fix os_kv.c so that it uses SQLITE_FCNTL_SYNC and hence
...
works even with PRAGMA synchronous=OFF.
FossilOrigin-Name: 21915af560b111aeeaee751790356151a5f063c2fc703dd4b35b22dc393409fb
2022-09-16 11:37:01 +00:00
57db2174d5
Add batch-runner-kvvfs.html, a kvvfs-specific build of batch-runner.html. Reduce the speedtest1 --size X value for the batch list generation to 5 so that the kvvfs batch runner can handle it.
...
FossilOrigin-Name: d8df25920a047d5cf2093cc6233128c5e6057a9104d0c4397e643645bd646fe1
2022-09-16 02:30:49 +00:00
58bcfe075d
Remove --nosync flag from speedtest1-kvvfs.html, as that seems to be the trigger for the 'no such table: z1' error. That app now warns if that flag is passed to it via URL args.
...
FossilOrigin-Name: ad0677e8abcc077636d1cf1d8485be4943506382581edf832e6b8a2021560040
2022-09-16 02:28:03 +00:00
d22cfa8c80
Pull the src/os_kv.c part of [13839759f8f4] into the kv-vfs branch.
...
FossilOrigin-Name: e334449912d5176e355d02024a07ed867741f71c9d10ce6744ca800414bf3eeb
2022-09-16 01:08:06 +00:00
0144a8482a
Add return value checks to kvstorageWrite(), necessary for recognizing storage-full errors in the JS binding. speedtest1-kvvfs.html runs successfully on localStorage with --size 1 to 6 and starts failing with storage shortage at --size 7.
...
FossilOrigin-Name: 13839759f8f45e4eb0aa6a5052801f3964d5b2dc2e427e5e91ee6692176381eb
2022-09-16 01:05:19 +00:00
28ef9bddb3
More work on the synchronous OPFS experimentation. Numerous wasm/js build tweaks. Add speeedtest-wasmfs.html, the wasmfs/opfs counterpart of speedtest1.html.
...
FossilOrigin-Name: 00ee49a3a2c148480f614e49a0ee5b35a255758c0a53693f0b464b31e7a4045b
2022-09-15 06:42:41 +00:00
dae7518ae4
Remove some stale comments. Get scratchpad-opfs-worker2.js running again in prep for reuse in experimenting with the OPFS all-synchronous API.
...
FossilOrigin-Name: 4cbbd370186f84e440f496e7f11c641d7dc1723acc46e4a31483392e0eb046e9
2022-09-15 03:16:49 +00:00
0117693f62
Split wasmfs-enabled build of sqlite3.js/wasm into sqlite3-wasmfs.js/wasm, as enabling wasmfs breaks all tests/demos which run from a Worker thread.
...
FossilOrigin-Name: 08476f3c218d45846e7496bdae0b06e2122466111fdf2aa2aabb1805b15ef982
2022-09-15 03:09:00 +00:00
409505c743
Clean up some JS documentation.
...
FossilOrigin-Name: 925fdbfc6c1f9f06951346bc0d4fb81ffb5797d5f1123728931973f6cb1a0efa
2022-09-15 02:27:48 +00:00
5b91500796
Add/apply various kvvfs-specific utility APIs to the JS layer to assist in testing and analysis. Correct a backwards default arg check for sqlite3ApiBootstrap(). Add exports for sqlite3_db_handle(), sqlite3_file_control(), and the SQLITE_FCNTL_xxx enum.
...
FossilOrigin-Name: 0d78961870ee9f22f1ba16d423377d28dcc36e04b1e31ffd57f3e2fd51f8f0f2
2022-09-13 19:27:03 +00:00
23d39ce54d
Get testing1.js working with a kvvfs build.
...
FossilOrigin-Name: 333a45725d1708e0fefa559c33ce1c7eeb425cdb04f594b1f2b48166c1478c79
2022-09-12 22:27:00 +00:00
5fd8f27bfb
Remove an extraneous key-copy op in the EM_JS impl of kvstorageDelete().
...
FossilOrigin-Name: 1c5aeee45564c14e7e2e7730f3f52106339ea3148fb5aa786fa767a35db46f51
2022-09-12 20:21:34 +00:00
0b769f5b4a
Merge kv-vfs branch into fiddle-opfs and add speedtest1-kvvfs.html test app (which currently fails early on).
...
FossilOrigin-Name: 8b1608a9a37131121cb3a0cb1e8dbad5e39dc1fc2a341d056f09537f179b2e7a
2022-09-12 20:18:28 +00:00
b126ec18b6
Fix the <sys/time.h> include in os_kv.c.
...
FossilOrigin-Name: 878cc93e779c3857adc711f212520f007256274fcd3f6f3d72c754a937fcd198
2022-09-12 19:33:20 +00:00
d8bb0b5ae6
Add a gettimeofday()-based implementation of xCurrentTimeInt64() to os_kv.c.
...
FossilOrigin-Name: e393ed650ef124143f84e9d787fb996e308dd7af6b8f50df72a6f085b67bf9c3
2022-09-12 19:29:34 +00:00
bb19512438
Fix a debugging/testing edit in the previous check-in.
...
FossilOrigin-Name: 6fc8d34c0ae1f8277544be741f2f0835fad8e475d35bd24573224ccc1699b8bd
2022-09-12 18:10:41 +00:00
a002cc174f
Add sqlite3.oo1.DB.clearKvvfsStorage(). Add controls to kvvfs1.js demo to reset and query the db without requiring the dev console.
...
FossilOrigin-Name: d845c6c22bd5d3fffc66e0566df346d690dd8bd1fc1688e312161b1a1edcfd79
2022-09-12 17:59:04 +00:00
6acd7b0a75
Faster text decoder for kv_os.c.
...
FossilOrigin-Name: 3354a2edb762d70ccc31d4d25f81b70e644d76e01cb1e81d2e97b5414d809d30
2022-09-12 17:44:35 +00:00
5360f5fcff
Merge kv-vfs branch into fiddle-opfs branch to add kvvfs-based wasm build and demo.
...
FossilOrigin-Name: a7d8b26acd3c1ae344369e4d70804c0cab45272c0983cfd32d616a0a7b28acb9
2022-09-12 16:09:50 +00:00
stephan