Revert the native impl of kvvfs's xOpen() (as distinct from the JS impl which the wasm build uses) to the historical db name restrictions of 'local' or 'session', failing with SQLITE_CANTOPEN if passed another name. [ec866b04d088e53b] overhauled support for kvvfs db names in JS but it turns out that the native impl still relies on those names in order to match journals to databases. Correct a related too-lenient assert() in the WASM pieces.

FossilOrigin-Name: fabcad09176cce96cacdd55ef3dde3be5b8062f513ac13a70e3db874358e6be2
Fix a problem that could cause fts5 to return incorrect answers when performing prefix queries on tokens with synonyms. Bug [bugs:/info/2026-06-14T11:11:06Z | 2026-06-14T11:11:06Z].
2026-06-15 12:22:06 +00:00 · 2026-06-15 11:47:52 +00:00 · 2026-06-15 11:03:13 +00:00 · 2026-06-15 10:13:31 +00:00 · 2026-06-15 01:24:06 +00:00 · 2026-06-14 22:01:56 +00:00
265 changed files with 34994 additions and 11183 deletions
@@ -0,0 +1,125 @@
+# AGENTS.md
+
+Guidance for AI coding agents working in this repository.
+
+## Project nature
+
+SQLite is a self-contained, serverless SQL database engine written in C. The
+source is **public domain** — no copyright or license header should ever be
+added to any file. The blessing comment that appears at the top of each source
+file is intentional and should be preserved unchanged.
+
+SQLite does not accept pull requests without prior agreement and/or
+accompanying legal paperwork that places the pull request in the public domain.
+However, the human SQLite developers will review a concise and well-written
+pull request as a proof-of-concept prior to reimplementing the changes
+themselves.
+
+SQLite does not accept agentic code.  However the project
+will accept agentic bug reports that include a reproducible test case.
+Patches or pull requests demonstrating a possible fix, for documentation
+purposes, are welcomed.
+
+SQLite uses the [Fossil](https://fossil-scm.org/) for version control,
+not Git.  The canonical repository is at <https://sqlite.org/src>.
+
+## Build
+
+The configure script uses [autosetup](https://msteveb.github.io/autosetup/),
+not GNU Autoconf.
+
+```bash
+apt install gcc make tcl-dev   # prerequisites (Debian/Ubuntu)
+
+./configure --dev              # debug build
+make sqlite3                   # CLI shell
+make sqlite3d                  # Debugging variant of the CLI shell
+make sqlite3.c                 # amalgamation (single-file distribution form)
+make testfixture               # test runner binary (requires tcl-dev)
+make tclextension-install      # install TCL extension before running tests
+```
+
+## Testing
+
+Tests are TCL scripts run through the `testfixture` enhanced interpreter.
+
+```bash
+# From the build directory:
+./testfixture test/main.test   # single test file
+test/testrunner.tcl            # quick suite
+test/testrunner.tcl full       # full suite
+test/testrunner.tcl fts5%      # pattern match
+
+# Check for failures:
+grep '!' testrunner.log
+```
+
+`make devtest` is the fastest way to run a representative subset. Always run
+at least `devtest` after any change to `src/`.
+
+## Architecture
+
+SQL text → **tokenizer** (`tokenize.c`) → **parser** (`parse.y` / Lemon) →
+**code generator** (`build.c`, `select.c`, `insert.c`, `update.c`, `delete.c`,
+`expr.c`) → **optimizer** (`where*.c`) → **VDBE** (`vdbe.c`) → **B-Tree**
+(`btree.c`) → **Pager** (`pager.c`) → **WAL** (`wal.c`) → **VFS**
+(`os_unix.c`, `os_win.c`).
+
+The master internal header is `src/sqliteInt.h`. Major subsystems have private
+headers: `vdbeInt.h`, `btreeInt.h`, `whereInt.h`. The public API is defined in
+`src/sqlite.h.in` (template) which generates `sqlite3.h`.
+
+## Do not edit generated files
+
+These files are produced by scripts and must not be edited by hand:
+
+| File | Regenerate with |
+|---|---|
+| `sqlite3.h` | `tool/mksqlite3h.tcl` |
+| `parse.c`, `parse.h` | build lemon (`tool/lemon.c`) then run on `src/parse.y` |
+| `opcodes.h` | `tool/mkopcodeh.tcl` (reads `src/vdbe.c`) |
+| `opcodes.c` | `tool/mkopcodec.tcl` (reads `opcodes.h`) |
+| `keywordhash.h` | `tool/mkkeywordhash.c` |
+| `pragma.h` | `tool/mkpragmatab.tcl` |
+| `sqlite3.c` | `tool/mksqlite3c.tcl` (the amalgamation) |
+
+Editing rules:
+- To add a PRAGMA: edit `tool/mkpragmatab.tcl`, then regenerate `pragma.h`.
+- To add a VDBE opcode: add the `case OP_Xxx:` handler in `src/vdbe.c`; the
+  opcode number and name are extracted automatically by `mkopcodeh.tcl`.
+- To change the SQL grammar: edit `src/parse.y`, not `parse.c`.
+
+## Coding conventions
+
+- C89/C99 compatible C only. No C++, no STL, no exceptions, no VLAs.
+- All memory allocation goes through `sqlite3Malloc` / `sqlite3_malloc64`
+  (never raw `malloc`). The `sqlite3MallocZero` variant zero-initializes.
+- Integer widths: use `i64` (`sqlite3_int64`) for 64-bit values, `u32`/`u64`
+  for unsigned. Avoid bare `long` or `int` for values that could exceed 2G.
+- Error propagation: functions return `SQLITE_OK` (0) on success and a
+  `SQLITE_*` error code on failure. Many routines also set `db->mallocFailed`
+  on OOM, allowing deferred error checking.
+- Assert liberally for invariants that must hold in correct code; use
+  `ALWAYS(x)` / `NEVER(x)` for conditions that are logically always
+  true/false but that the compiler cannot prove.
+
+## Extensions (`ext/`)
+
+Extensions are compiled separately from the core and are not part of the
+amalgamation by default (except where explicitly included). Major subsystems:
+
+- `ext/fts5/` — Full-Text Search 5 (current)
+- `ext/rtree/` — R-Tree spatial indexing
+- `ext/session/` — Changesets and sessions
+- `ext/recover/` — Database file recovery
+- `ext/misc/` — Single-file utility extensions (many included in the CLI)
+- `ext/qrf/` — Query Result Formatter utility library
+
+## Useful references
+
+- Architecture: <https://sqlite.org/arch.html>
+- File format: <https://sqlite.org/fileformat2.html>
+- VDBE opcodes: <https://sqlite.org/opcode.html>
+- Query planner: <https://sqlite.org/optoverview.html>
+- Lemon parser generator: <https://sqlite.org/doc/trunk/doc/lemon.html>
+- Compile-time options: <https://sqlite.org/compile.html>
@@ -131,9 +131,15 @@ libsqlite3.DLL.install-rules = @SQLITE_DLL_INSTALL_RULES@
 CFLAGS.fuzzcheck-asan.fsanitize = @CFLAGS_ASAN_FSANITIZE@

 #
-# Intended to either be empty or be set to -g -DSQLITE_DEBUG=1.
+# TARGET_DEBUG is intended to either be empty or be set to -g
+# -DSQLITE_DEBUG=1, plus any other flags relevant for debug-only
+# builds. These get added near the front of $(T.cc) so that they can
+# be overridden by CFLAGS passed directly to make. Historically (prior
+# to 2026-03-30) these were at/near the end of the flags but that made
+# it impossible to pass custom -O* flags to specific binaries without
+# reconfiguring.
 #
-T.cc.TARGET_DEBUG = @TARGET_DEBUG@
+T.cc += @TARGET_DEBUG@

 #
 # $(JIMSH) and $(CFLAGS.jimsh) are documented in main.mk.  $(JIMSH)
@@ -164,7 +170,9 @@ OPT_FEATURE_FLAGS = @OPT_FEATURE_FLAGS@ $(OPTIONS)
 PACKAGE_VERSION = @PACKAGE_VERSION@

 #
-# Filename extensions for binaries and libraries
+# Filename extensions for binaries and libraries. B.* are for the
+# being-built-on platform and T.* are for the build's target platform.
+# i.e. B.* is for build-time tooling and T.* is for deliverables.
 #
 B.exe = @BUILD_EXEEXT@
 T.exe = @TARGET_EXEEXT@
@@ -1719,6 +1719,7 @@ FUZZERSHELL_COMPILE_OPTS =
 FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -I$(TOP)\test -I$(TOP)\ext\recover
 FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_MEMSYS5
 FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_OSS_FUZZ
+FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_DECIMAL_MAX_DIGIT=1000
 FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_BYTECODE_VTAB
 FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_CARRAY
 FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_DBPAGE_VTAB
@@ -1765,7 +1766,18 @@ FUZZCHECK_SRC = $(FUZZCHECK_SRC) $(TOP)\test\fuzzinvariants.c
 FUZZCHECK_SRC = $(FUZZCHECK_SRC) $(TOP)\test\vt02.c
 FUZZCHECK_SRC = $(FUZZCHECK_SRC) $(TOP)\ext\recover\dbdata.c
 FUZZCHECK_SRC = $(FUZZCHECK_SRC) $(TOP)\ext\recover\sqlite3recover.c
+FUZZCHECK_SRC = $(FUZZCHECK_SRC) $(TOP)\ext\misc\base64.c
+FUZZCHECK_SRC = $(FUZZCHECK_SRC) $(TOP)\ext\misc\base85.c
+FUZZCHECK_SRC = $(FUZZCHECK_SRC) $(TOP)\ext\misc\completion.c
+FUZZCHECK_SRC = $(FUZZCHECK_SRC) $(TOP)\ext\misc\decimal.c
+FUZZCHECK_SRC = $(FUZZCHECK_SRC) $(TOP)\ext\misc\diskused.c
+FUZZCHECK_SRC = $(FUZZCHECK_SRC) $(TOP)\ext\misc\ieee754.c
 FUZZCHECK_SRC = $(FUZZCHECK_SRC) $(TOP)\ext\misc\randomjson.c
+FUZZCHECK_SRC = $(FUZZCHECK_SRC) $(TOP)\ext\misc\regexp.c
+FUZZCHECK_SRC = $(FUZZCHECK_SRC) $(TOP)\ext\misc\series.c
+FUZZCHECK_SRC = $(FUZZCHECK_SRC) $(TOP)\ext\misc\shathree.c
+FUZZCHECK_SRC = $(FUZZCHECK_SRC) $(TOP)\ext\misc\sha1.c
+FUZZCHECK_SRC = $(FUZZCHECK_SRC) $(TOP)\ext\misc\stmtrand.c

 OSSSHELL_SRC = $(TOP)\test\ossshell.c $(TOP)\test\ossfuzz.c
 DBFUZZ_COMPILE_OPTS = -DSQLITE_THREADSAFE=0 -DSQLITE_OMIT_LOAD_EXTENSION
@@ -1885,9 +1897,6 @@ RSYNC_OPT = \
 sqlite3_rsync.exe:	$(RSYNC_SRC) $(LIBRESOBJS)
 	$(LTLINK) $(RSYNC_OPT) $(NO_WARN) $(RSYNC_SRC) /link $(LDFLAGS) $(LTLINKOPTS) $(LIBRESOBJS)

-scrub.exe:	$(TOP)\ext\misc\scrub.c $(SQLITE3C) $(SQLITE3H)
-	$(LTLINK) $(NO_WARN) -DSCRUB_STANDALONE=1 $(TOP)\ext\misc\scrub.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS)
-
 srcck1.exe:	$(TOP)\tool\srcck1.c
 	$(BCC) $(NO_WARN) -Fe$@ $(TOP)\tool\srcck1.c

@@ -2336,6 +2345,7 @@ SHELL_DEP = \
    $(TOP)\ext\misc\base85.c \
    $(TOP)\ext\misc\completion.c \
    $(TOP)\ext\misc\decimal.c \
+    $(TOP)\ext\misc\diskused.c \
    $(TOP)\ext\misc\fileio.c \
    $(TOP)\ext\misc\ieee754.c \
    $(TOP)\ext\misc\memtrace.c \
@@ -2540,43 +2550,19 @@ coretestprogs:	testfixture.exe sqlite3.exe

 testprogs:	$(TESTPROGS) srcck1.exe fuzzcheck.exe sessionfuzz.exe

-fulltest:	alltest fuzztest
-
-alltest:	$(TESTPROGS)
-	@set PATH=$(LIBTCLPATH);$(PATH)
-	.\testfixture.exe $(TOP)\test\all.test $(TESTOPTS)
-
-soaktest:	$(TESTPROGS)
-	@set PATH=$(LIBTCLPATH);$(PATH)
-	.\testfixture.exe $(TOP)\test\all.test -soak=1 $(TESTOPTS)
-
-fulltestonly:	$(TESTPROGS) fuzztest
-	@set PATH=$(LIBTCLPATH);$(PATH)
-	.\testfixture.exe $(TOP)\test\full.test
-
-queryplantest:	testfixture.exe shell
-	@set PATH=$(LIBTCLPATH);$(PATH)
-	.\testfixture.exe $(TOP)\test\permutations.test queryplanner $(TESTOPTS)
-
 fuzztest:	fuzzcheck.exe
 	.\fuzzcheck.exe $(FUZZDATA)

-# Legacy testing target for third-party integrators.  The SQLite
-# developers seldom use this target themselves.  Instead
-# they use "nmake /f Makefile.msc devtest" which runs tests on
-# a standard set of options
 #
-test:	$(TESTPROGS) sourcetest fuzztest tcltest
-
-# Minimal testing that runs in less than 3 minutes (on a fast machine)
-#
-quicktest:	testfixture.exe sourcetest
-	@set PATH=$(LIBTCLPATH);$(PATH)
-	.\testfixture.exe $(TOP)\test\extraquick.test $(TESTOPTS)
-
-# This is the common case.  Run many tests that do not take too long,
-# including fuzzcheck, sqlite3_analyzer, and sqldiff tests.
+# Legacy testing targets, no longer used by the developers and
+# now aliased to one of the commonly used testing targets.
 #
+quicktest:	devtest
+test:	devtest
+fulltest:	releasetest
+alltest:	releasetest
+soaktest:	releasetest
+fulltestonly:	releasetest

 # The veryquick.test TCL tests.
 #
@@ -2640,6 +2626,12 @@ smoketest:	$(TESTPROGS)
 	@set PATH=$(LIBTCLPATH);$(PATH)
 	.\testfixture.exe $(TOP)\test\main.test $(TESTOPTS)

+# Measure the performance of floating-point conversions.
+#
+fp-speed-test:	fp-speed-1.exe fp-speed-2.exe
+	fp-speed-1 1000000
+	fp-speed-2 1000000
+
 shelltest:
 	$(TCLSH_CMD) $(TOP)\test\testrunner.tcl release shell

@@ -2732,6 +2724,14 @@ speedtest1.exe:	$(TOP)\test\speedtest1.c $(SQLITE3C) $(SQLITE3H)
 	$(LTLINK) $(NO_WARN) $(ST_COMPILE_OPTS) -DSQLITE_OMIT_LOAD_EXTENSION \
 		$(TOP)\test\speedtest1.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS)

+fp-speed-1.exe:	$(TOP)\test\fp-speed-1.c $(SQLITE3C) $(SQLITE3H)
+	$(LTLINK) $(NO_WARN) $(ST_COMPILE_OPTS) -DSQLITE_OMIT_LOAD_EXTENSION \
+		$(TOP)\test\fp-speed-1.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS)
+
+fp-speed-2.exe:	$(TOP)\test\fp-speed-2.c $(SQLITE3C) $(SQLITE3H)
+	$(LTLINK) $(NO_WARN) $(ST_COMPILE_OPTS) -DSQLITE_OMIT_LOAD_EXTENSION \
+		$(TOP)\test\fp-speed-2.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS)
+
 kvtest.exe:	$(TOP)\test\kvtest.c $(SQLITE3C) $(SQLITE3H)
 	$(LTLINK) $(NO_WARN) $(KV_COMPILE_OPTS) \
 		$(TOP)\test\kvtest.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS)
@@ -2740,6 +2740,9 @@ rbu.exe:	$(TOP)\ext\rbu\rbu.c $(TOP)\ext\rbu\sqlite3rbu.c $(SQLITE3C) $(SQLITE3H
 	$(LTLINK) $(NO_WARN) -DSQLITE_ENABLE_RBU \
 		$(TOP)\ext\rbu\rbu.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS)

+$(AUXTEST).exe:	$(TOP)\test\c\$(AUXTEST).c
+	$(LTLINK) $(NO_WARN) $(TOP)\test\c\$(AUXTEST).c sqlite3.lo /link $(LDFLAGS) $(LTLINKOPTS)
+
 THREADTEST3_SRC = \
  $(TOP)\test\threadtest3.c \
  $(TOP)\test\tt3_checkpoint.c \
@@ -2921,6 +2924,7 @@ clean:
 	del /Q LogEst.exe fts3view.exe rollback-test.exe showdb.exe dbdump.exe 2>NUL
 	del /Q changeset.exe 2>NUL
 	del /Q showjournal.exe showstat4.exe showwal.exe speedtest1.exe 2>NUL
+	del /Q fp-speed-1.exe fp-speed-2.exe 2>NUL
 	del /Q mptester.exe wordcount.exe rbu.exe srcck1.exe 2>NUL
 	del /Q sqlite3.c sqlite3-*.c sqlite3.h 2>NUL
 	del /Q sqlite3rc.h 2>NUL
@@ -2935,4 +2939,5 @@ clean:
 	del /Q fts5.* fts5parse.* 2>NUL
 	del /q src-verify.exe 2>NUL
 	del /q jimsh.exe jimsh0.exe 2>NUL
+	-$(TCLSH_CMD) test/testrunner.tcl clean
 # <</mark>>
@@ -33,10 +33,11 @@ verify its integrity, there are hints on how to do that in the

 ## Contacting The SQLite Developers

-The preferred way to ask questions or make comments about SQLite or to
-report bugs against SQLite is to visit the 
-[SQLite Forum](https://sqlite.org/forum) at <https://sqlite.org/forum/>.
-Anonymous postings are permitted.
+The preferred way to ask questions or make comments about SQLite is to
+visit the [SQLite Forum](https://sqlite.org/forum) at
+<https://sqlite.org/forum/>.  Anonymous postings are permitted. For
+bug reports, please use the [SQLite bugs
+forum](https://sqlite.org/bugs) at <https://sqlite.org/bugs>.

 If you think you have found a bug that has security implications and
 you do not want to report it on the public forum, you can send a private
@@ -80,8 +81,8 @@ Then run commands like this:
        fossil open https://sqlite.org/src

 The initial "fossil open" command will take two or three minutes.  Afterwards,
-you can do fast, bandwidth-efficient updates to the whatever versions
-of SQLite you like.  Some examples:
+you can do fast, bandwidth-efficient updates to whatever versions of SQLite you
+like.  Some examples:

        fossil update trunk             ;# latest trunk check-in
        fossil update release           ;# latest official release
@@ -97,6 +98,8 @@ the build products.  It is recommended, but not required, that the
 build directory be separate from the source directory.  Cd into the
 build directory and then from the build directory run the configure
 script found at the root of the source tree.  Then run "make".
+See the [compile-for-unix.md](doc/compile-for-unix.md) document for
+more detail.

 For example:

@@ -1 +1 @@
-3.52.1
+3.54.0
@@ -461,4 +461,4 @@ all other significant processing.
 [sqlite-config.tcl]: /file/autosetup/sqlite-config.tcl
 [Makefile.in]: /file/Makefile.in
 [main.mk]: /file/main.mk
-[JimTCL]: https://jim.tcl.tk
+[JimTCL]: https://msteveb.github.io/jimtcl/
@@ -2028,7 +2028,7 @@ array set proj__Cache {}
 # Returns a cache key for the given argument:
 #
 #   integer: relative call stack levels to get the scope name of for
-#   use as a key. [proj-scope [expr {1 + $arg + addLevel}]] is
+#   use as a key. [proj-scope [expr {1 + $arg + $addLevel}]] is
 #   then used to generate the key. i.e. the default of 0 uses the
 #   calling scope's name as the key.
 #
@@ -2226,7 +2226,7 @@ proc proj-coalesce {args} {
 #
 # Pragmas:
 #
-# Passing :PRAGMAS to this function may modify how it works. The
+# The prototype list may contain :PRAGMAS to modify how it works. The
 # following pragmas are supported (note the leading ":"):
 #
 #   :all-flags indicates that the whole input list should be scanned,
@@ -743,12 +743,11 @@ proc sqlite-setup-default-cflags {} {
  # compiling binaries for the target system (CC a.k.a. $(T.cc)).
  # Normally they're the same, but they will differ when
  # cross-compiling.
-  #
-  # When cross-compiling we default to not using the -g flag, based on a
-  # /chat discussion prompted by
-  # https://sqlite.org/forum/forumpost/9a67df63eda9925c
  set defaultCFlags {-O2}
  if {!$::sqliteConfig(is-cross-compiling)} {
+    # When cross-compiling we default to not using the -g flag, based
+    # on a /chat discussion prompted by
+    # https://sqlite.org/forum/forumpost/9a67df63eda9925c
    lappend defaultCFlags -g
  }
  define CFLAGS [proj-get-env CFLAGS $defaultCFlags]
@@ -1068,7 +1067,7 @@ proc sqlite-get-readline-dir-list {} {
    }
    *-haiku {
      lappend dirs /boot/system/develop/headers
-      if {[opt-val with-readline-ldflags] in {auto ""}} {
+      if {![opt-bool editline] && [opt-val with-readline-ldflags] in {auto ""}} {
        # If the user did not supply their own --with-readline-ldflags
        # value, hijack that flag to inject options which are known to
        # work on Haiku OS installations.
@@ -1095,16 +1094,30 @@ proc sqlite-get-readline-dir-list {} {
 #   - HAVE_LINENOISE to 0, 1, or 2
 #   - HAVE_EDITLINE to 0 or 1
 #
-# Only one of ^^^ those will be set to non-0.
+#  Those will be set as follows:
+#
+#   HAVE_LINENOISE=0, HAVE_EDITLINE=0, HAVE_READLINE=0: no line editing
+#
+#   HAVE_LINENOISE>0: 1=antirez flavor, 2=msteveb flavor
+#
+#   HAVE_EDITLINE=0, HAVE_READLINE=1: use GNU readline
+#
+#   HAVE_EDITLINE=1, HAVE_READLINE=1: use BSD editline header
+#   <readline/readline.h>.
+#
+#   HAVE_EDITLINE=1, HAVE_READLINE=0: use BSD editline header
+#   <editline/readline.h>.
+#
+# Other defines it sets:
 #
 #   - LDFLAGS_READLINE = linker flags or empty string
 #
 #   - CFLAGS_READLINE = compilation flags for clients or empty string.
 #
-# Note that LDFLAGS_READLINE and CFLAGS_READLINE may refer to
-# linenoise or editline, not necessarily libreadline.  In some cases
-# it will set HAVE_READLINE=1 when it's really using editline, for
-# reasons described in this function's comments.
+# Both LDFLAGS_READLINE and CFLAGS_READLINE may refer to linenoise or
+# editline, not necessarily libreadline.  In some cases it will set
+# HAVE_READLINE=1 when it's really using editline, for reasons
+# described in this function's comments.
 #
 # Returns a string describing which line-editing approach to use, or
 # "none" if no option is available.
@@ -1142,14 +1155,14 @@ proc sqlite-check-line-editing {} {
  # If none of --with-linenoise, --enable-readline, or --enable-editline
  # are provided, but there exists a directory "linenoise" at $HOME or
  # a sibling of the build or source directory, then try to use that linenoise
-  # direcctory.
+  # directory.
  #
  if {"" eq $dirLn
   && ![proj-opt-was-provided readline]
   && ![proj-opt-was-provided editline]
  } {
    set dirlist ../linenoise
-    catch {lappend dirlist [file normalize $::autosetup(srcdir)/../linenoise]}
+    catch {lappend dirlist [file-normalize $::autosetup(srcdir)/../linenoise]}
    catch {lappend dirlist $::env(HOME)/linenoise}
    foreach d $dirlist {
      if {[file exists $d/linenoise.c] && [file exists $d/linenoise.h]} {
@@ -1195,7 +1208,7 @@ proc sqlite-check-line-editing {} {
      define-append CFLAGS_JIMSH -DUSE_LINENOISE [get-define CFLAGS_READLINE]
      user-notice "Adding linenoise support to jimsh."
    } else {
-      msg-result "Using linenoise at [file normalize $dirLn]"
+      msg-result "Using linenoise at [file-normalize $dirLn]"
    }
    return "linenoise ($flavor)"
  } elseif {[opt-bool editline]} {
@@ -1306,14 +1319,18 @@ proc sqlite-check-line-editing {} {

  # If we found a library, configure the build to use it...
  if {"" ne $rlLib} {
-    if {"editline" eq $editLibName && "HAVE_READLINE" eq $editLibDef} {
-      # Alert the user that, despite outward appearances, we won't be
-      # linking to the GPL'd libreadline. Presumably that distinction is
-      # significant for those using --editline.
-      proj-indented-notice {
-        NOTE: the local libedit uses <readline/readline.h> so we
-        will compile with -DHAVE_READLINE=1 but will link with
-        libedit.
+    if {"editline" eq $editLibName} {
+      # sqlite-add-shell-opt -DEL_WIDECHAR=1;
+      if {"HAVE_READLINE" eq $editLibDef} {
+        sqlite-add-shell-opt -DHAVE_EDITLINE=1
+        # Alert the user that, despite outward appearances, we won't be
+        # linking to the GPL'd libreadline. Presumably that distinction is
+        # significant for those using --editline.
+        proj-indented-notice {
+          NOTE: the local libedit uses <readline/readline.h> so we
+          will compile with -DHAVE_READLINE=1 but will link with
+          libedit.
+        }
      }
    }
    set rlLib [join $rlLib]
@@ -1746,8 +1763,12 @@ proc sqlite-handle-env-quirks {} {
  set autoDll 0; # true if --out-implib/--dll-basename should be implied
  set host [get-define host]
  switch -glob -- $host {
-    *apple* -
-    *darwin*    { set instName darwin }
+    *-*-darwin* {
+      set instName darwin
+      # We don't look for *apple* because:
+      # https://sqlite.org/forum/forumpost/7b218c3c9f207646
+      # There's at least one Linux out there which matches *apple*.
+    }
    default {
      set x [sqlite-env-is-unix-on-windows $host]
      if {"" ne $x} {
@@ -2043,7 +2064,7 @@ proc sqlite-check-tcl {} {
      }
    }
    if {![file-isexec $with_tclsh]} {
-      proj-warn "Cannot find a usable tclsh (tried: $tryThese)
+      proj-warn "Cannot find a usable tclsh (tried: $tryThese)"
    }
  }
  define TCLSH_CMD $with_tclsh
@@ -37,18 +37,35 @@ guidance on building for Windows.
  4.  Download the SQLite source tree and unpack it. CD into the
      toplevel directory of the source tree.

-  5.  Run: `./configure --enable-all --with-tclsh=$HOME/local/bin/tclsh9.0`
+  5.  *(Optional):* Download Antirez's "linenoise" command-line editing
+      library to provide command-line editing in the CLI.  You can find
+      a suitable copy of the linenoise sources at
+      <https://sqlite.org/linenoise.tar.gz> or at various other locations
+      on the internet.  If you put the linenoise source tree in
+      a directory named $HOME/linenoise or a directory "linenoise" which
+      is a sibling of the SQLite source tree, then the SQLite ./configure
+      script will automatically find and use that source code to provide
+      command-line editing in the CLI. If you would rather use the readline
+      or editline libraries or a precompiled linenoise library, there are
+      ./configure options to accommodate that choice. The SQLite developers
+      typically use $HOME/linenoise since linenoise is small, has no
+      external dependencies, "just works", and the ./configure script
+      will pick it up and use it automatically. But you do what works
+      best for you.
+      <p>
+      You are not required to have any command-line editing support in
+      order to use SQLite.  But command-line editing does make the
+      interactive experience more enjoyable.
+
+  6.  Run: `./configure --enable-all --with-tclsh=$HOME/local/bin/tclsh9.0`

      You do not need to use --with-tclsh if the tclsh you want to use is the
      first one on your PATH or if you are building without TCL.

-      The SQLite developers typically add the
-      &ensp; `--with-linenoise=$HOME/linenoise` &ensp; option
-      to provide command-line editing.  "$HOME/linenoise" is a directory
-      that contains [linenoise](https://github.com/antirez/linenoise) source
-      code files, `linenoise.c` and `linenoise.h`.
+      Lots of other options to ./configure are available.
+      Run `./configure --help` for further guidance.

-  6.  Run the "`Makefile`" makefile with an appropriate target.
+  7.  Run the "`Makefile`" makefile with an appropriate target.
      Examples:
      <ul>
      <li>  `make sqlite3.c`
@@ -73,5 +90,7 @@ guidance on building for Windows.
      of SQLite.


-  7.  For a debugging build of the CLI, where the ".treetrace" and ".wheretrace"
-      commands work, add the the --with-debug argument to configure.
+  8.  For a debugging build of the CLI, use `./configure --dev`.  A debugging
+      build contains lots of extra debugging code, so it is slow and a lot
+      bigger.  You probably do not want to deploy a debugging build.  But if
+      you are working on the code, a debugging build works much better.
@@ -33,7 +33,7 @@
 The testrunner.tcl program is a Tcl script used to run multiple SQLite
 tests in parallel, thus reducing testing time on multi-core machines.
 The testrunner.tcl supports running tests that based on `testfixture`,
-`sqlite3`, and `fuzzcheck`.
+`sqlite3`, `fuzzcheck`, and C programs found in test/c.

 <a name="runtr"></a>
 ## 1.1 Running testrunner.tcl
@@ -59,11 +59,16 @@ up front.
 The standard Makefiles for SQLite include targets that invoke
 testrunner.tcl.  So the following commands also run testrunner.tcl:

+   *  `make test`
   *  `make devtest`
   *  `make releasetest`
   *  `make sdevtest`
   *  `make testrunner`

+Since SQLite 3.53.0, there is a "<tt>make.bat</tt>" script in the root
+of the source tree that causes the "make" command to work on Windows.
+There is no need to do something different using "nmake" on Windows.
+
 <a name="outputs"></a>
 ## 1.3 Outputs from testrunner.tcl

@@ -606,4 +606,40 @@ ifcapable fts5 {
  }
 }

+#-------------------------------------------------------------------------
+reset_db
+
+set ci {CREATE INDEX i1 ON t1(a COLLATE "binary,sqlite_expert_rem(999,0)");}
+
+do_execsql_test 8.0 {
+  BEGIN TRANSACTION;
+  CREATE TABLE t1(a TEXT, b TEXT);
+  INSERT INTO t1 VALUES('v0','d0');
+  INSERT INTO t1 VALUES('v1','d1');
+  INSERT INTO t1 VALUES('v2','d2');
+  INSERT INTO t1 VALUES('v3','d3');
+  INSERT INTO t1 VALUES('v4','d4');
+  INSERT INTO t1 VALUES('v5','d5');
+  INSERT INTO t1 VALUES('v6','d6');
+  INSERT INTO t1 VALUES('v7','d7');
+  INSERT INTO t1 VALUES('v8','d8');
+  INSERT INTO t1 VALUES('v9','d9');
+  CREATE INDEX i1 ON t1(a);
+  COMMIT;
+  PRAGMA writable_schema = ON;
+  UPDATE sqlite_schema SET sql = $ci WHERE name = 'i1';
+}
+
+db close
+sqlite3 db test.db
+
+do_test 8.1 {
+  set expert [sqlite3_expert_new db]
+  $expert sql { SELECT 1234 }
+  list [catch { $expert analyze } msg] $msg
+} {1 {no such collation sequence: binary,sqlite_expert_rem(999,0)}}
+
+$expert destroy
+
+
 finish_test
@@ -1501,7 +1501,7 @@ static int idxCreateVtabSchema(sqlite3expert *p, char **pzErrmsg){
        /* The statement the vtab will pass to sqlite3_declare_vtab() */
        zInner = idxAppendText(&rc, 0, "CREATE TABLE x(");
        for(i=0; i<pTab->nCol; i++){
-          zInner = idxAppendText(&rc, zInner, "%s%Q COLLATE %s", 
+          zInner = idxAppendText(&rc, zInner, "%s%Q COLLATE %Q",
              (i==0 ? "" : ", "), pTab->aCol[i].zName, pTab->aCol[i].zColl
          );
        }
@@ -1701,7 +1701,7 @@ static int idxPopulateOneStat1(
      return sqlite3_reset(pIndexXInfo);
    }
    zCols = idxAppendText(&rc, zCols, 
-        "%sx.%Q IS sqlite_expert_rem(%d, x.%Q) COLLATE %s", 
+        "%sx.%Q IS sqlite_expert_rem(%d, x.%Q) COLLATE %Q", 
        zComma, zName, nCol, zName, zColl
    );
    zOrder = idxAppendText(&rc, zOrder, "%s%d", zComma, ++nCol);
@@ -302,6 +302,12 @@
  SQLITE_EXTENSION_INIT1
 #endif

+
+/*
+** Assume any b-tree layer with more levels than this is corrupt.
+*/
+#define FTS3_MAX_BTREE_HEIGHT 48
+
 typedef struct Fts3HashWrapper Fts3HashWrapper;
 struct Fts3HashWrapper {
  Fts3Hash hash;                  /* Hash table */
@@ -2018,7 +2024,11 @@ static int fts3SelectLeaf(
  assert( piLeaf || piLeaf2 );

  fts3GetVarint32(zNode, &iHeight);
-  rc = fts3ScanInteriorNode(zTerm, nTerm, zNode, nNode, piLeaf, piLeaf2);
+  if( iHeight>FTS3_MAX_BTREE_HEIGHT ){
+    rc = FTS_CORRUPT_VTAB;
+  }else{
+    rc = fts3ScanInteriorNode(zTerm, nTerm, zNode, nNode, piLeaf, piLeaf2);
+  }
  assert_fts3_nc( !piLeaf2 || !piLeaf || rc!=SQLITE_OK || (*piLeaf<=*piLeaf2) );

  if( rc==SQLITE_OK && iHeight>1 ){
@@ -2063,8 +2073,13 @@ static void fts3PutDeltaVarint(
  sqlite3_int64 iVal              /* Write this value to the list */
 ){
  assert_fts3_nc( iVal-*piPrev > 0 || (*piPrev==0 && iVal==0) );
-  *pp += sqlite3Fts3PutVarint(*pp, iVal-*piPrev);
-  *piPrev = iVal;
+  if( iVal-(*piPrev)>=0 ){
+    /* Refuse to write a negative delta integer. This only happens with a 
+    ** corrupt db (see the assert above) and can cause buffer overwrites
+    ** in some cases.  */
+    *pp += sqlite3Fts3PutVarint(*pp, iVal-*piPrev);
+    *piPrev = iVal;
+  }
 }

 /*
@@ -203,7 +203,16 @@ typedef sqlite3_int64 i64;        /* 8-byte signed integer */
 #define LARGEST_INT64  (0xffffffff|(((i64)0x7fffffff)<<32))
 #define SMALLEST_INT64 (((i64)-1) - LARGEST_INT64)

-#define deliberate_fall_through
+#if !defined(deliberate_fall_through)
+# if defined(__has_attribute)
+#  if __has_attribute(fallthrough)
+#    define deliberate_fall_through __attribute__((fallthrough));
+#  endif
+# endif
+#endif
+#if !defined(deliberate_fall_through)
+# define deliberate_fall_through
+#endif

 /*
 ** Macros needed to provide flexible arrays in a portable way
@@ -325,7 +325,7 @@ static int fts3auxNextMethod(sqlite3_vtab_cursor *pCursor){
            pCsr->aStat[1].nDoc++;
          }
          eState = 2;
-          /* fall through */
+          /* no break */ deliberate_fall_through

        case 2:
          if( v==0 ){       /* 0x00. Next integer will be a docid. */
@@ -341,7 +341,7 @@ static int fts3auxNextMethod(sqlite3_vtab_cursor *pCursor){
        /* State 3. The integer just read is a column number. */
        default: assert( eState==3 );
          iCol = (int)v;
-          if( iCol<1 ){
+          if( iCol<1 || iCol>(pFts3->nColumn+1) ){
            rc = SQLITE_CORRUPT_VTAB;
            break;
          }
@@ -471,6 +471,7 @@ static int getNextNode(
        assert( nKey==4 );
        if( zInput[4]=='/' && zInput[5]>='0' && zInput[5]<='9' ){
          nKey += 1+sqlite3Fts3ReadInt(&zInput[nKey+1], &nNear);
+          if( nNear>=1000000000 ) nNear = 1000000000;
        }
      }

@@ -132,10 +132,9 @@ struct StrBuffer {
 /*
 ** Allocate a two-slot MatchinfoBuffer object.
 */
-static MatchinfoBuffer *fts3MIBufferNew(size_t nElem, const char *zMatchinfo){
+static MatchinfoBuffer *fts3MIBufferNew(i64 nElem, const char *zMatchinfo){
  MatchinfoBuffer *pRet;
-  sqlite3_int64 nByte = sizeof(u32) * (2*(sqlite3_int64)nElem + 1)
-                           + SZ_MATCHINFOBUFFER(1);
+  sqlite3_int64 nByte = sizeof(u32) * (2*(i64)nElem+1) + SZ_MATCHINFOBUFFER(1);
  sqlite3_int64 nStr = strlen(zMatchinfo);

  pRet = sqlite3Fts3MallocZero(nByte + nStr+1);
@@ -887,7 +886,7 @@ static int fts3ExprLHits(
      if( p->flag==FTS3_MATCHINFO_LHITS ){
        p->aMatchinfo[iStart + iCol] = (u32)nHit;
      }else if( nHit ){
-        p->aMatchinfo[iStart + (iCol+1)/32] |= (1 << (iCol&0x1F));
+        p->aMatchinfo[iStart + iCol/32] |= (1U << (iCol&0x1F));
      }
    }
    assert( *pIter==0x00 || *pIter==0x01 );
@@ -1006,8 +1005,8 @@ static int fts3MatchinfoCheck(
  return SQLITE_ERROR;
 }

-static size_t fts3MatchinfoSize(MatchInfo *pInfo, char cArg){
-  size_t nVal;                      /* Number of integers output by cArg */
+static i64 fts3MatchinfoSize(MatchInfo *pInfo, char cArg){
+  i64 nVal;                      /* Number of integers output by cArg */

  switch( cArg ){
    case FTS3_MATCHINFO_NDOC:
@@ -1023,16 +1022,16 @@ static size_t fts3MatchinfoSize(MatchInfo *pInfo, char cArg){
      break;

    case FTS3_MATCHINFO_LHITS:
-      nVal = (size_t)pInfo->nCol * pInfo->nPhrase;
+      nVal = (i64)pInfo->nCol * pInfo->nPhrase;
      break;

    case FTS3_MATCHINFO_LHITS_BM:
-      nVal = (size_t)pInfo->nPhrase * ((pInfo->nCol + 31) / 32);
+      nVal = (i64)pInfo->nPhrase * ((pInfo->nCol + 31) / 32);
      break;

    default:
      assert( cArg==FTS3_MATCHINFO_HITS );
-      nVal = (size_t)pInfo->nCol * pInfo->nPhrase * 3;
+      nVal = (i64)pInfo->nCol * pInfo->nPhrase * 3;
      break;
  }

@@ -1314,7 +1313,7 @@ static int fts3MatchinfoValues(

      case FTS3_MATCHINFO_LHITS_BM:
      case FTS3_MATCHINFO_LHITS: {
-        size_t nZero = fts3MatchinfoSize(pInfo, zArg[i]) * sizeof(u32);
+        i64 nZero = fts3MatchinfoSize(pInfo, zArg[i]) * sizeof(u32);
        memset(pInfo->aMatchinfo, 0, nZero);
        rc = fts3ExprLHitGather(pCsr->pExpr, pInfo);
        break;
@@ -1383,7 +1382,7 @@ static void fts3GetMatchinfo(
  ** initialize those elements that are constant for every row.
  */
  if( pCsr->pMIBuffer==0 ){
-    size_t nMatchinfo = 0;        /* Number of u32 elements in match-info */
+    i64 nMatchinfo = 0;           /* Number of u32 elements in match-info */
    int i;                        /* Used to iterate through zArg */

    /* Determine the number of phrases in the query */
@@ -3129,6 +3129,10 @@ static void fts3ReadEndBlockField(
    for(/* no-op */; zText[i]>='0' && zText[i]<='9'; i++){
      iVal = iVal*10 + (zText[i] - '0');
    }
+
+    /* This if() clause is just to avoid an integer overflow. The record is 
+    ** corrupt in this case.  */
+    if( (i64)iVal==SMALLEST_INT64 ) iMul = 1;
    *pnByte = ((i64)iVal * (i64)iMul);
  }
 }
@@ -426,7 +426,7 @@ static void fts5SnippetFunction(
  int rc = SQLITE_OK;             /* Return code */
  int iCol;                       /* 1st argument to snippet() */
  const char *zEllips;            /* 4th argument to snippet() */
-  int nToken;                     /* 5th argument to snippet() */
+  i64 nToken;                     /* 5th argument to snippet() */
  int nInst = 0;                  /* Number of instance matches this row */
  int i;                          /* Used to iterate through instances */
  int nPhrase;                    /* Number of phrases in query */
@@ -451,7 +451,7 @@ static void fts5SnippetFunction(
  ctx.zClose = fts5ValueToText(apVal[2]);
  ctx.iRangeEnd = -1;
  zEllips = fts5ValueToText(apVal[3]);
-  nToken = sqlite3_value_int(apVal[4]);
+  nToken = (int)(MIN( MAX(sqlite3_value_int64(apVal[4]), 0), 64));

  iBestCol = (iCol>=0 ? iCol : 0);
  nPhrase = pApi->xPhraseCount(pFts);
@@ -807,7 +807,7 @@ static int fts5ExprNearIsMatch(int *pRc, Fts5ExprNearset *pNear){
      i64 iPos = a[i].reader.iPos;
      Fts5PoslistWriter *pWriter = &a[i].writer;
      if( a[i].pOut->n==0 || iPos!=pWriter->iPrev ){
-        sqlite3Fts5PoslistWriterAppend(a[i].pOut, pWriter, iPos);
+        sqlite3Fts5PoslistSafeAppend(a[i].pOut, &pWriter->iPrev, iPos);
      }
    }

@@ -1758,10 +1758,10 @@ static int fts5ParseTokenize(
      memset(pSyn, 0, (size_t)nByte);
      pSyn->pTerm = ((char*)pSyn) + sizeof(Fts5ExprTerm) + sizeof(Fts5Buffer);
      pSyn->nFullTerm = pSyn->nQueryTerm = nToken;
+      memcpy(pSyn->pTerm, pToken, nToken);
      if( pCtx->pConfig->bTokendata ){
        pSyn->nQueryTerm = (int)strlen(pSyn->pTerm);
      }
-      memcpy(pSyn->pTerm, pToken, nToken);
      pSyn->pSynonym = pPhrase->aTerm[pPhrase->nTerm-1].pSynonym;
      pPhrase->aTerm[pPhrase->nTerm-1].pSynonym = pSyn;
    }
@@ -323,7 +323,7 @@ int sqlite3Fts5HashWrite(
    **     + 5 bytes for the new position offset (32-bit max).
    */
    if( (p->nAlloc - p->nData) < (9 + 4 + 1 + 3 + 5) ){
-      sqlite3_int64 nNew = p->nAlloc * 2;
+      sqlite3_int64 nNew = (i64)p->nAlloc * 2;
      Fts5HashEntry *pNew;
      Fts5HashEntry **pp;
      pNew = (Fts5HashEntry*)sqlite3_realloc64(p, nNew);
@@ -922,7 +922,7 @@ static void fts5DataRelease(Fts5Data *pData){
 static Fts5Data *fts5LeafRead(Fts5Index *p, i64 iRowid){
  Fts5Data *pRet = fts5DataRead(p, iRowid);
  if( pRet ){
-    if( pRet->nn<4 || pRet->szLeaf>pRet->nn ){
+    if( pRet->szLeaf<4 || pRet->szLeaf>pRet->nn ){
      FTS5_CORRUPT_ROWID(p, iRowid);
      fts5DataRelease(pRet);
      pRet = 0;
@@ -1695,6 +1695,7 @@ static int fts5DlidxLvlPrev(Fts5DlidxLvl *pLvl){
    pLvl->bEof = 1;
  }else{
    u8 *a = pLvl->pData->p;
+    int nn = pLvl->pData->nn;

    pLvl->iOff = 0;
    fts5DlidxLvlNext(pLvl);
@@ -1703,7 +1704,7 @@ static int fts5DlidxLvlPrev(Fts5DlidxLvl *pLvl){
      int ii = pLvl->iOff;
      u64 delta = 0;

-      while( a[ii]==0 ){
+      while( ii<nn && a[ii]==0 ){
        nZero++;
        ii++;
      }
@@ -2122,7 +2123,7 @@ static void fts5SegIterReverseNewPage(Fts5Index *p, Fts5SegIter *pIter){
  while( p->rc==SQLITE_OK && pIter->iLeafPgno>pIter->iTermLeafPgno ){
    Fts5Data *pNew;
    pIter->iLeafPgno--;
-    pNew = fts5DataRead(p, FTS5_SEGMENT_ROWID(
+    pNew = fts5LeafRead(p, FTS5_SEGMENT_ROWID(
          pIter->pSeg->iSegid, pIter->iLeafPgno
    ));
    if( pNew ){
@@ -2576,6 +2577,10 @@ static void fts5LeafSeek(
    if( nKeep<nMatch ){
      goto search_failed;
    }
+    if( (iOff+nNew)>n ){
+      FTS5_CORRUPT_ITER(p, pIter);
+      return;
+    }

    assert( nKeep>=nMatch );
    if( nKeep==nMatch ){
@@ -3552,8 +3557,7 @@ static void fts5PoslistFilterCallback(

    do {
      while( i<nChunk && pChunk[i]!=0x01 ){
-        while( pChunk[i] & 0x80 ) i++;
-        i++;
+        fts5IndexSkipVarint(pChunk, i);
      }
      if( pCtx->eState ){
        fts5BufferSafeAppendBlob(pCtx->pBuf, &pChunk[iStart], i-iStart);
@@ -3702,7 +3706,7 @@ static void fts5IndexExtractColset(
      /* Advance pointer p until it points to pEnd or an 0x01 byte that is
      ** not part of a varint */
      while( p<pEnd && *p!=0x01 ){
-        while( *p++ & 0x80 );
+        while( p<pEnd && (*p++ & 0x80) );
      }

      if( pColset->aiCol[i]==iCurrent ){
@@ -3799,8 +3803,11 @@ static void fts5IterSetOutputs_Col100(Fts5Iter *pIter, Fts5SegIter *pSeg){

  assert( pIter->pIndex->pConfig->eDetail==FTS5_DETAIL_COLUMNS );
  assert( pIter->pColset );
+  assert( pIter->poslist.nSpace>=pIter->pIndex->pConfig->nCol );

-  if( pSeg->iLeafOffset+pSeg->nPos>pSeg->pLeaf->szLeaf ){
+  if( pSeg->iLeafOffset+pSeg->nPos>pSeg->pLeaf->szLeaf 
+   || pSeg->nPos>pIter->pIndex->pConfig->nCol
+  ){
    fts5IterSetOutputs_Col(pIter, pSeg);
  }else{
    u8 *a = (u8*)&pSeg->pLeaf->p[pSeg->iLeafOffset];
@@ -5294,6 +5301,11 @@ static void fts5DoSecureDelete(
    }else{
      iStart = fts5GetU16(&aPg[0]);
    }
+    if( iStart>nPg ){
+      FTS5_CORRUPT_IDX(p);
+      sqlite3_free(aIdx);
+      return;
+    }

    iSOP = iStart + fts5GetVarint(&aPg[iStart], &iDelta);
    assert_nc( iSOP<=pSeg->iLeafOffset );
@@ -7979,8 +7991,8 @@ static void fts5IndexTombstoneRebuild(
 ){
  const int MINSLOT = 32;
  int nSlotPerPage = MAX(MINSLOT, (p->pConfig->pgsz - 8) / szKey);
-  int nSlot = 0;                  /* Number of slots in each output page */
-  int nOut = 0;
+  i64 nSlot = 0;                  /* Number of slots in each output page */
+  i64 nOut = 0;

  /* Figure out how many output pages (nOut) and how many slots per 
  ** page (nSlot).  There are three possibilities: 
@@ -8005,23 +8017,26 @@ static void fts5IndexTombstoneRebuild(
    nSlot = MINSLOT;
  }else if( pSeg->nPgTombstone==1 ){
    /* Case 2. */
-    int nElem = (int)fts5GetU32(&pData1->p[4]);
+    u32 nElem = fts5GetU32(&pData1->p[4]);
    assert( pData1 && iPg1==0 );
-    nOut = 1;
-    nSlot = MAX(nElem*4, MINSLOT);
-    if( nSlot>nSlotPerPage ) nOut = 0; 
+    if( nElem>((u32)nSlotPerPage/4) ){
+      nOut = 0;
+    }else{
+      nOut = 1;
+      nSlot = MAX((i64)nElem*4, MINSLOT);
+    }
  }
  if( nOut==0 ){
    /* Case 3. */
-    nOut = (pSeg->nPgTombstone * 2 + 1);
+    nOut = ((i64)pSeg->nPgTombstone * 2 + 1);
    nSlot = nSlotPerPage;
  }

  /* Allocate the required array and output pages */
  while( 1 ){
    int res = 0;
-    int ii = 0;
-    int szPage = 0;
+    i64 ii = 0;
+    i64 szPage = 0;
    Fts5Data **apOut = 0;

    /* Allocate space for the new hash table */
@@ -8559,7 +8574,7 @@ static void fts5IndexIntegrityCheckSegment(
        /* Check any rowid-less pages that occur before the current leaf. */
        for(iPg=iPrevLeaf+1; iPg<fts5DlidxIterPgno(pDlidx); iPg++){
          iKey = FTS5_SEGMENT_ROWID(iSegid, iPg);
-          pLeaf = fts5DataRead(p, iKey);
+          pLeaf = fts5LeafRead(p, iKey);
          if( pLeaf ){
            if( fts5LeafFirstRowidOff(pLeaf)!=0 ) FTS5_CORRUPT_ROWID(p, iKey);
            fts5DataRelease(pLeaf);
@@ -8570,7 +8585,7 @@ static void fts5IndexIntegrityCheckSegment(
        /* Check that the leaf page indicated by the iterator really does
        ** contain the rowid suggested by the same. */
        iKey = FTS5_SEGMENT_ROWID(iSegid, iPrevLeaf);
-        pLeaf = fts5DataRead(p, iKey);
+        pLeaf = fts5LeafRead(p, iKey);
        if( pLeaf ){
          i64 iRowid;
          int iRowidOff = fts5LeafFirstRowidOff(pLeaf);
@@ -517,7 +517,7 @@ static void fts5SetEstimatedRows(sqlite3_index_info *pIdxInfo, i64 nRow){
  if( sqlite3_libversion_number()>=3008002 )
 #endif
  {
-    pIdxInfo->estimatedRows = nRow;
+    pIdxInfo->estimatedRows = MAX(1, nRow);
  }
 #endif
 }
@@ -586,19 +586,30 @@ static int fts5UsePatternMatch(
 **  a) If a MATCH operator is present, the cost depends on the other
 **     constraints also present. As follows:
 **
-**       * No other constraints:         cost=1000.0
-**       * One rowid range constraint:   cost=750.0
-**       * Both rowid range constraints: cost=500.0
-**       * An == rowid constraint:       cost=100.0
+**       * No other constraints:         cost=50000.0
+**       * One rowid range constraint:   cost=37500.0
+**       * Both rowid range constraints: cost=30000.0
+**       * An == rowid constraint:       cost=25000.0
 **
 **  b) Otherwise, if there is no MATCH:
 **
-**       * No other constraints:         cost=1000000.0
-**       * One rowid range constraint:   cost=750000.0
-**       * Both rowid range constraints: cost=250000.0
-**       * An == rowid constraint:       cost=10.0
+**       * No other constraints:         cost=3000000.0
+**       * One rowid range constraints:  cost=2250000.0
+**       * Both rowid range constraint:  cost=750000.0
+**       * An == rowid constraint:       cost=25.0
 **
 ** Costs are not modified by the ORDER BY clause.
+**
+** The ratios used in case (a) are based on informal results obtained from 
+** the tool/fts5cost.tcl script. The "MATCH and ==" combination has the
+** cost set quite high because the query may be a prefix query. Unless
+** there is a prefix index, prefix queries with rowid constraints are much 
+** more expensive than non-prefix queries with rowid constraints.
+**
+** The estimated rows returned is set to the cost/40. For simple queries,
+** experimental results show that cost/4 might be about right. But for
+** more complex queries that use multiple terms the number of rows might
+** be far fewer than this. So we compromise and use cost/40.
 */
 static int fts5BestIndexMethod(sqlite3_vtab *pVTab, sqlite3_index_info *pInfo){
  Fts5Table *pTab = (Fts5Table*)pVTab;
@@ -724,21 +735,35 @@ static int fts5BestIndexMethod(sqlite3_vtab *pVTab, sqlite3_index_info *pInfo){

  /* Calculate the estimated cost based on the flags set in idxFlags. */
  if( bSeenEq ){
-    pInfo->estimatedCost = nSeenMatch ? 1000.0 : 25.0;
-    fts5SetUniqueFlag(pInfo);
+    pInfo->estimatedCost = nSeenMatch ? 25000.0 : 25.0;
    fts5SetEstimatedRows(pInfo, 1);
+    fts5SetUniqueFlag(pInfo);
  }else{
-    if( bSeenLt && bSeenGt ){
-      pInfo->estimatedCost = nSeenMatch ? 5000.0 :   750000.0;
-    }else if( bSeenLt || bSeenGt ){
-      pInfo->estimatedCost = nSeenMatch ? 7500.0 :  2250000.0;
+    i64 nEstRows;
+    if( nSeenMatch ){
+      if( bSeenLt && bSeenGt ){
+        pInfo->estimatedCost = 50000.0;
+      }else if( bSeenLt || bSeenGt ){
+        pInfo->estimatedCost = 37500.0;
+      }else{
+        pInfo->estimatedCost = 50000.0;
+      }
+      nEstRows = (i64)(pInfo->estimatedCost / 40.0);
+      for(i=1; i<nSeenMatch; i++){
+        pInfo->estimatedCost *= 2.5;
+        nEstRows = nEstRows / 2;
+      }
    }else{
-      pInfo->estimatedCost = nSeenMatch ? 10000.0 : 3000000.0;
+      if( bSeenLt && bSeenGt ){
+        pInfo->estimatedCost = 750000.0;
+      }else if( bSeenLt || bSeenGt ){
+        pInfo->estimatedCost = 2250000.0;
+      }else{
+        pInfo->estimatedCost = 3000000.0;
+      }
+      nEstRows = (i64)(pInfo->estimatedCost / 4.0);
    }
-    for(i=1; i<nSeenMatch; i++){
-      pInfo->estimatedCost *= 0.4;
-    }
-    fts5SetEstimatedRows(pInfo, (i64)(pInfo->estimatedCost / 4.0));
+    fts5SetEstimatedRows(pInfo, nEstRows);
  }

  pInfo->idxNum = idxFlags;
@@ -586,8 +586,14 @@ static int fts5PorterCreate(
  const char *zBase = "unicode61";
  fts5_tokenizer_v2 *pV2 = 0;

-  if( nArg>0 ){
-    zBase = azArg[0];
+  while( nArg>0 ){
+    if( sqlite3_stricmp(azArg[0],"porter")==0 ){
+      nArg--;
+      azArg++;
+    }else{
+      zBase = azArg[0];
+      break;
+    }
  }

  pRet = (PorterTokenizer*)sqlite3_malloc64(sizeof(PorterTokenizer));
@@ -712,9 +718,9 @@ static int fts5Porter_Ostar(char *zStem, int nStem){
    for(i=0; i<nStem; i++){
      bCons = !fts5PorterIsVowel(zStem[i], bCons);
      assert( bCons==0 || bCons==1 );
-      mask = (mask << 1) + bCons;
+      mask = ((mask << 1) + bCons) & 0x0007;
    }
-    return ((mask & 0x0007)==0x0005);
+    return (mask==0x0005);
  }
 }

@@ -191,6 +191,29 @@ do_execsql_test 5.6 {
  SELECT snippet(p1, 0, '[', NULL, '...', 6) FROM p1('x OR ' || x'DB');
 } {{[x a a a a a...}}

+do_execsql_test 5.7.1 {
+  SELECT snippet(p1, 0, '[', NULL, '...', -2147483647) FROM p1('x OR ' || x'DB')
+} {{[x...}}
+do_execsql_test 5.7.2 {
+  SELECT snippet(p1, 0, '[', NULL, '...', -2147483648) FROM p1('x OR ' || x'DB')
+} {{[x...}}
+do_execsql_test 5.7.3 {
+  SELECT snippet(p1, 0, '[', NULL, '...', -2147483649) FROM p1('x OR ' || x'DB')
+} {{[x...}}
+do_execsql_test 5.7.4 {
+  SELECT snippet(p1, 0, '[', NULL, '...', 0) FROM p1('x OR ' || x'DB')
+} {{[x...}}
+
+do_execsql_test 5.8.1 {
+  SELECT snippet(p1, 0, '[', NULL, '...', 2147483647) FROM p1('x OR ' || x'DB')
+} {{[x a a a a a a a a a a}}
+do_execsql_test 5.8.2 {
+  SELECT snippet(p1, 0, '[', NULL, '...', 2147483648) FROM p1('x OR ' || x'DB')
+} {{[x a a a a a a a a a a}}
+do_execsql_test 5.8.3 {
+  SELECT snippet(p1, 0, '[', NULL, '...', 2147483649) FROM p1('x OR ' || x'DB')
+} {{[x a a a a a a a a a a}}
+
 } ;# foreach_detail_mode 

 reset_db
@@ -0,0 +1,28 @@
+# 2026 June 19
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#*************************************************************************
+# This file implements regression tests for SQLite library.  The
+# focus of this script is testing the FTS5 module.
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+set testprefix fts5bigtok2
+return_if_no_fts5
+
+set big [string repeat a 1080000000]
+do_execsql_test 1.0 {
+  CREATE VIRTUAL TABLE t USING fts5(x);
+}
+
+do_catchsql_test 1.1 { INSERT INTO t VALUES($big); } {1 {out of memory}}
+
+set big {}
+
+finish_test
@@ -15153,9 +15153,9 @@ do_execsql_test 78.1 {
  CREATE VIRTUAL TABLE t3 USING fts5vocab('t1','col');
 }

-do_execsql_test 78.2 {
+do_catchsql_test 78.2 {
  SELECT count(rowid) FROM t3 WHERE term>='nsocse';
-} 2
+} {1 {database disk image is malformed}}

 #-------------------------------------------------------------------------
 reset_db
@@ -1220,7 +1220,7 @@ do_test 8.0 {

 do_catchsql_test 8.1 {
  SELECT rowid FROM t1('a* NOT ý') ;
-} {0 {1 2 3 4 5 6 7}}
+} {1 {fts5: corruption on page 1, segment 1, table "t1"}}

 #-------------------------------------------------------------------------
 reset_db
@@ -1917,6 +1917,222 @@ do_catchsql_test 11.3 {
  REPLACE INTO t1(rowid,b,a,rowid) VALUES(x'44023b9eb002d28b0ee90c',1,2,3);
 } {1 {database disk image is malformed}}

+#-------------------------------------------------------------------------
+reset_db
+do_test 12.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+.open --hexdb
+| size 24576 pagesize 4096 filename p.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 03 00 00 00 06   .....@  ........
+|     32: 00 00 00 00 00 00 00 00 00 00 00 06 00 00 00 04   ................
+|     48: 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00   ................
+|     80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03   ................
+|     96: 00 2e 76 89 0d 00 00 00 06 0d ed 00 0f bb 0f 64   ..v............d
+|    112: 0e f9 0e a2 0e 45 0d ed 00 00 00 00 00 00 00 00   .....E..........
+|   3552: 00 00 00 00 00 00 00 00 00 00 00 00 00 56 06 06   .............V..
+|   3568: 17 1f 1f 01 7d 74 61 62 6c 65 66 74 5f 63 6f 6e   .....tableft_con
+|   3584: 66 69 67 66 74 5f 63 6f 6e 66 69 67 06 43 52 45   figft_config.CRE
+|   3600: 41 54 45 20 54 41 42 4c 45 20 27 66 74 5f 63 6f   ATE TABLE 'ft_co
+|   3616: 6e 66 69 67 27 28 6b 20 50 52 49 4d 41 52 59 20   nfig'(k PRIMARY 
+|   3632: 4b 45 59 2c 20 76 29 20 57 49 54 48 4f 55 54 20   KEY, v) WITHOUT 
+|   3648: 52 4f 57 49 44 5b 05 07 17 21 21 01 81 01 74 61   ROWID[...!!...ta
+|   3664: 62 6c 65 66 74 5f 64 6f 63 73 69 7a 65 66 74 5f   bleft_docsizeft_
+|   3680: 64 6f 63 73 69 7a 65 05 43 52 45 41 54 45 20 54   docsize.CREATE T
+|   3696: 41 42 4c 45 20 27 66 74 5f 64 6f 63 73 69 7a 65   ABLE 'ft_docsize
+|   3712: 27 28 69 64 20 49 4e 54 45 47 45 52 20 50 52 49   '(id INTEGER PRI
+|   3728: 4d 41 52 59 20 4b 45 59 2c 20 73 7a 20 42 4c 4f   MARY KEY, sz BLO
+|   3744: 42 29 55 04 06 17 21 21 01 77 74 61 62 6c 65 66   B)U...!!.wtablef
+|   3760: 74 5f 63 6f 6e 74 65 6e 74 66 74 5f 63 6f 6e 74   t_contentft_cont
+|   3776: 65 6e 74 04 43 52 45 41 54 45 20 54 41 42 4c 45   ent.CREATE TABLE
+|   3792: 20 27 66 74 5f 63 6f 6e 74 65 6e 74 27 28 69 64    'ft_content'(id
+|   3808: 20 49 4e 54 45 47 45 52 20 50 52 49 4d 41 52 59    INTEGER PRIMARY
+|   3824: 20 4b 45 59 2c 20 63 30 29 69 03 07 17 19 19 01    KEY, c0)i......
+|   3840: 81 2d 74 61 62 6c 65 66 74 5f 69 64 78 66 74 5f   .-tableft_idxft_
+|   3856: 69 64 78 03 43 52 45 41 54 45 20 54 41 42 4c 45   idx.CREATE TABLE
+|   3872: 20 27 66 74 5f 69 64 78 27 28 73 65 67 69 64 2c    'ft_idx'(segid,
+|   3888: 20 74 65 72 6d 2c 20 70 67 6e 6f 2c 20 50 52 49    term, pgno, PRI
+|   3904: 4d 41 52 59 20 4b 45 59 28 73 65 67 69 64 2c 20   MARY KEY(segid, 
+|   3920: 74 65 72 6d 29 29 20 57 49 54 48 4f 55 54 20 52   term)) WITHOUT R
+|   3936: 4f 57 49 44 55 02 07 17 1b 1b 01 81 01 74 61 62   OWIDU........tab
+|   3952: 6c 65 66 74 5f 64 61 74 61 66 74 5f 64 61 74 61   left_dataft_data
+|   3968: 02 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 66   .CREATE TABLE 'f
+|   3984: 74 5f 64 61 74 61 27 28 69 64 20 49 4e 54 45 47   t_data'(id INTEG
+|   4000: 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20   ER PRIMARY KEY, 
+|   4016: 62 6c 6f 63 6b 20 42 4c 4f 42 29 43 01 06 17 11   block BLOB)C....
+|   4032: 11 08 75 74 61 62 6c 65 66 74 66 74 43 52 45 41   ..utableftftCREA
+|   4048: 54 45 20 56 49 52 54 55 41 4c 20 54 41 42 4c 45   TE VIRTUAL TABLE
+|   4064: 20 66 74 20 55 53 49 4e 47 20 66 74 73 35 28 61    ft USING fts5(a
+|   4080: 2c 20 64 65 74 61 69 6c 3d 63 6f 6c 75 6d 6e 29   , detail=column)
+| page 2 offset 4096
+|      0: 0d 00 00 00 03 0f 7f 00 0f e8 0f ef 0f 7f 00 00   ................
+|   3952: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 62   ...............b
+|   3968: 84 80 80 80 80 01 04 00 81 48 00 00 00 54 04 30   .........H...T.0
+|   3984: 61 61 61 01 81 02 02 02 02 02 02 02 02 02 02 02   aaa.............
+|   4000: 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02   ................
+|   4016: 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02   ................
+|   4032: 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02   ................
+|   4048: 02 02 02 02 02 02 02 03 6a 6a 6a 01 02 02 04 08   ........jjj.....
+|   4064: 08 08 08 08 08 08 08 08 05 01 03 00 10 01 0a 0f   ................
+|   4080: 0a 03 00 24 00 00 00 00 01 01 01 00 01 01 01 01   ...$............
+| page 3 offset 8192
+|      0: 0a 00 00 00 01 0f fa 00 0f fa 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0c 01 02   ................
+| page 4 offset 12288
+|      0: 0d 00 00 00 01 0f d4 00 0f d4 00 00 00 00 00 00   ................
+|   4048: 00 00 00 00 2a 01 03 00 5b 61 61 61 20 62 62 62   ....*...[aaa bbb
+|   4064: 20 63 63 63 20 64 64 64 20 65 65 65 20 66 66 66    ccc ddd eee fff
+|   4080: 20 67 67 67 20 68 68 68 20 69 69 69 20 6a 6a 6a    ggg hhh iii jjj
+| page 5 offset 16384
+|      0: 0d 00 00 00 01 0f fa 00 0f fa 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 00 00 00 00 00 00 04 01 03 00 0e 0a   ................
+| page 6 offset 20480
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| end p.db
+}]} {}
+
+do_catchsql_test 12.1 {
+  SELECT rowid FROM ft('a:aaa')
+} {0 1}
+
+#-------------------------------------------------------------------------
+reset_db
+do_test 13.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+.open --hexdb
+| size 24576 pagesize 4096 filename vuln_001.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 03 00 00 00 06   .....@  ........
+|     32: 00 00 00 00 00 00 00 00 00 00 00 06 00 00 00 04   ................
+|     48: 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00   ................
+|     80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03   ................
+|     96: 00 2e 76 89 0d 00 00 00 06 0e 09 00 0f ca 0f 77   ..v............w
+|    112: 0f 0f 0e b7 0e 5e 0e 09 00 00 00 00 00 00 00 00   .....^..........
+|   3584: 00 00 00 00 00 00 00 00 00 53 06 06 17 1d 1d 01   .........S......
+|   3600: 7b 74 61 62 6c 65 74 5f 63 6f 6e 66 69 67 74 5f   .tablet_configt_
+|   3616: 63 6f 6e 66 69 67 06 43 52 45 41 54 45 20 54 41   config.CREATE TA
+|   3632: 42 4c 45 20 27 74 5f 63 6f 6e 66 69 67 27 28 6b   BLE 't_config'(k
+|   3648: 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 76 29    PRIMARY KEY, v)
+|   3664: 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 57 05    WITHOUT ROWIDW.
+|   3680: 06 17 1f 1f 01 7f 74 61 62 6c 65 74 5f 64 6f 63   ......tablet_doc
+|   3696: 73 69 7a 65 74 5f 64 6f 63 73 69 7a 65 05 43 52   sizet_docsize.CR
+|   3712: 45 41 54 45 20 54 41 42 4c 45 20 27 74 5f 64 6f   EATE TABLE 't_do
+|   3728: 63 73 69 7a 65 27 28 69 64 20 49 4e 54 45 47 45   csize'(id INTEGE
+|   3744: 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 73   R PRIMARY KEY, s
+|   3760: 7a 20 42 4c 4f 42 29 56 04 06 17 1f 1f 01 7d 74   z BLOB)V.......t
+|   3776: 61 62 6c 65 74 5f 63 6f 6e 74 65 6e 74 74 5f 63   ablet_contentt_c
+|   3792: 6f 6e 74 65 6e 74 04 43 52 45 41 54 45 20 54 41   ontent.CREATE TA
+|   3808: 42 4c 45 20 27 74 5f 63 6f 6e 74 65 6e 74 27 28   BLE 't_content'(
+|   3824: 69 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d 41   id INTEGER PRIMA
+|   3840: 52 59 20 4b 45 59 2c 20 63 30 2c 20 63 31 29 66   RY KEY, c0, c1)f
+|   3856: 03 07 17 17 17 01 81 2b 74 61 62 6c 65 74 5f 69   .......+tablet_i
+|   3872: 64 78 74 5f 69 64 78 03 43 52 45 41 54 45 20 54   dxt_idx.CREATE T
+|   3888: 41 42 4c 45 20 27 74 5f 69 64 78 27 28 73 65 67   ABLE 't_idx'(seg
+|   3904: 69 64 2c 20 74 65 72 6d 2c 20 70 67 6e 6f 2c 20   id, term, pgno, 
+|   3920: 50 52 49 4d 41 52 59 20 4b 45 59 28 73 65 67 69   PRIMARY KEY(segi
+|   3936: 64 2c 20 74 65 72 6d 29 29 20 57 49 54 48 4f 55   d, term)) WITHOU
+|   3952: 54 20 52 4f 57 49 44 51 02 06 17 19 19 01 7f 74   T ROWIDQ.......t
+|   3968: 61 62 6c 65 74 5f 64 61 74 61 74 5f 64 61 74 61   ablet_datat_data
+|   3984: 02 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74   .CREATE TABLE 't
+|   4000: 5f 64 61 74 61 27 28 69 64 20 49 4e 54 45 47 45   _data'(id INTEGE
+|   4016: 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 62   R PRIMARY KEY, b
+|   4032: 6c 6f 63 6b 20 42 4c 4f 42 29 34 01 06 17 0f 0f   lock BLOB)4.....
+|   4048: 08 5b 74 61 62 6c 65 74 74 43 52 45 41 54 45 20   .[tablettCREATE 
+|   4064: 56 49 52 54 55 41 4c 20 54 41 42 4c 45 20 74 20   VIRTUAL TABLE t 
+|   4080: 55 53 49 4e 47 20 66 74 73 35 28 61 2c 20 62 29   USING fts5(a, b)
+| page 2 offset 4096
+|      0: 0d 00 00 00 04 0b da 00 0f e7 0f ef 0f ce 0b da   ................
+|   3024: 00 00 00 00 00 00 00 00 00 00 87 6c 84 80 80 80   ...........l....
+|   3040: 80 02 04 00 8f 5c 00 00 03 e8 80 80 80 80 80 80   ................
+|   3056: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   3072: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   3088: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   3104: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   3120: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   3136: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   3152: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   3168: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   3184: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   3200: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   3216: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   3232: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   3248: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   3264: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   3280: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   3296: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   3312: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   3328: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   3344: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   3360: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   3376: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   3392: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   3408: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   3424: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   3440: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   3456: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   3472: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   3488: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   3504: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   3520: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   3536: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   3552: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   3568: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   3584: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   3600: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   3616: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   3632: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   3648: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   3664: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   3680: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   3696: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   3712: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   3728: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   3744: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   3760: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   3776: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   3792: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   3808: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   3824: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   3840: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   3856: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   3872: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   3888: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   3904: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   3920: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   3936: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   3952: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   3968: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   3984: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   4000: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   4016: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80   ................
+|   4032: 80 80 80 80 80 80 80 80 80 80 80 80 80 80 12 84   ................
+|   4048: 80 80 80 80 01 03 00 2a 00 0b 00 0e 06 30 68 65   .......*.....0he
+|   4064: 6c 6c 6f 01 04 04 04 06 01 03 00 12 01 02 02 0f   llo.............
+|   4080: 0a 03 00 24 00 00 00 00 01 01 01 00 01 01 01 02   ...$............
+| page 3 offset 8192
+|      0: 0a 00 00 00 01 0f fa 00 0f fa 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0c 01 02   ................
+| page 4 offset 12288
+|      0: 0d 00 00 00 01 0f e8 00 0f e8 00 00 00 00 00 00   ................
+|   4064: 00 00 00 00 00 00 00 00 16 01 04 00 23 1b 68 65   ............#.he
+|   4080: 6c 6c 6f 20 77 6f 72 6c 64 66 6f 6f 20 62 61 72   llo worldfoo bar
+| page 5 offset 16384
+|      0: 0d 00 00 00 01 0f f9 00 0f f9 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 00 00 00 00 00 05 01 03 00 10 02 02   ................
+| page 6 offset 20480
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| end vuln_001.db
+}]} {}
+
+do_catchsql_test 13.1 {
+  SELECT * FROM t('a:hello')
+} {0 {{hello world} {foo bar}}}

 sqlite3_fts5_may_be_corrupt 0
 finish_test
@@ -125,4 +125,40 @@ do_catchsql_test 2.3 {
  DELETE FROM t1 WHERE rowid = 1
 } {/.*fts5: corrupt.*/}

+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 3.0 {
+  PRAGMA page_size=4096;
+  PRAGMA journal_mode=DELETE;
+  CREATE VIRTUAL TABLE t USING fts5(x, detail=none);
+
+  WITH s(i) AS (
+    VALUES(1) UNION ALL SELECT i+1 FROM s WHERE i<5000
+  ) 
+  INSERT INTO t(rowid, x) SELECT i, 'vulnerabilitytest' FROM s;
+
+  INSERT INTO t(t) VALUES('optimize');
+  INSERT INTO t(t, rank) VALUES('secure-delete', 1);
+} {delete}
+
+do_test 3.1 {
+  db eval { SELECT rowid AS rowid, block FROM t_data ORDER BY rowid } {
+    if {$rowid>=10 && [string length $block]>=4} {
+      binary scan $block Su first_rowid_off
+      set pgno [expr ($rowid & 0x7FFFFFFF)]
+      if {$pgno>=2 && $first_rowid_off>0} break
+    }
+  }
+
+  set bad [binary format a*a* "\xFF\xFF" [string range $block 2 end]]
+  db eval {
+    UPDATE t_data SET block = $bad WHERE rowid=$rowid
+  }
+} {}
+
+do_catchsql_test 3.2 {
+  DELETE FROM t WHERE rowid=4500;
+} {1 {fts5: corruption in table "t"}}
+
+
 finish_test
@@ -0,0 +1,273 @@
+# 2026 May 11
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#***********************************************************************
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+set testprefix fts5corruptA
+
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
+ifcapable !fts5 {
+  finish_test
+  return
+}
+sqlite3_fts5_may_be_corrupt 1
+
+do_execsql_test 1.0 {
+  CREATE VIRTUAL TABLE t USING fts5(x, detail='full');
+  INSERT INTO t(t, rank) VALUES('pgsz', 32);
+}
+
+set big [string repeat "a " 200]
+do_execsql_test 1.1 {
+  INSERT INTO t(rowid, x) VALUES(1, $big)
+}
+
+do_test 1.2 {
+  db eval { 
+      SELECT min(rowid) AS base_rowid, count(*) AS page_count FROM t_data
+      WHERE rowid>1000
+  } {}
+} {}
+
+do_test 1.3 {
+  for {set ii 0} {$ii < 5} {incr ii} {
+    db eval { 
+      INSERT INTO t_data(rowid, block) 
+      VALUES( $base_rowid + $page_count + $ii, zeroblob(4) );
+    }
+  }
+  db eval { 
+    INSERT INTO t_data(rowid, block) 
+      VALUES( $base_rowid + $page_count + 5, 
+      unhex('00000080' || 'CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC') );
+  }
+  set {} {}
+} {}
+
+db close
+
+do_test 1.4 {
+  set hex [hexio_read test.db 0 [file size test.db]]
+
+  set off [string first "023061018310" $hex]
+  set hex [string replace $hex $off [expr $off+11] 023061018370]
+  hexio_write test.db 0 $hex
+} {6144}
+
+sqlite3 db test.db
+
+do_catchsql_test 1.5 {
+  SELECT rowid FROM t WHERE t MATCH 'a'
+} {1 {fts5: corruption found reading blob 137438953481 from table "t"}}
+
+#-------------------------------------------------------------------------
+reset_db
+
+do_execsql_test 2.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(a);
+  BEGIN;
+    INSERT INTO t1(rowid, a) VALUES(1, 'aaaa');
+    INSERT INTO t1(rowid, a) VALUES(2, 'bbbb');
+    INSERT INTO t1(rowid, a) VALUES(3, 'cccc');
+  COMMIT;
+}
+
+db close
+
+do_test 2.1 {
+  set hex [hexio_read test.db 0 [file size test.db]]
+  set off [string first "010463636363" $hex]
+  set hex [string replace $hex $off [expr $off+11] 017F63636363]
+  hexio_write test.db 0 $hex
+  set {} {}
+} {}
+
+sqlite3 db test.db
+
+do_catchsql_test 2.2 {
+  SELECT rowid FROM t1('cccccccccccccccccccccccccccccccccccccccccccccccc');
+} {1 {fts5: corruption on page 1, segment 1, table "t1"}}
+
+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 3.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(a, b, c);
+  INSERT INTO t1 VALUES(
+      'hello',
+      'alpha bravo charlie delta echo foxtrot golf hotel india juliet kilo lima mike november oscar papa quebec romeo sierra tango uniform victor whiskey xray yankee zulu',
+      'ant bear cat dog elephant fox gorilla hippo iguana jaguar kangaroo lion monkey newt octopus panda quail rabbit snake turtle unicorn vulture walrus xerus yak zebra'
+  );
+
+  UPDATE t1_data SET block = unhex(
+      -- Preserve bytes 0-168: header through hello's nSz field
+      hex(substr(block, 1, 169))
+      -- Inject 448 bytes of 0xFF: corrupted poslist + subsequent term data
+      || replace(hex(zeroblob(448)), '00', 'FF')
+      -- Preserve 16 bytes of page index needed for hello lookup
+      || hex(substr(block, 618, 16))
+      -- Corrupt remaining 37 footer bytes (not needed for hello lookup)
+      || replace(hex(zeroblob(37)), '00', 'FF')
+      )
+  WHERE id = 137438953473;
+}
+
+do_test 3.1 {
+  execsql { SELECT * FROM t1 WHERE t1 MATCH '{a b}: hello' }
+  set {} {}
+} {}
+
+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 4.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(content);
+  INSERT INTO t1(t1, rank) VALUES('pgsz', 32);
+  BEGIN;
+    WITH s(i) AS (
+      SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<5000
+    ) 
+    INSERT INTO t1(content) SELECT 'xyzterm' FROM s;
+  COMMIT;
+  INSERT INTO t1(t1) VALUES('optimize');
+}
+
+do_test 4.1 {
+  db eval {
+    SELECT rowid AS a, hex(block) AS block FROM t1_data 
+    WHERE ((rowid>>36) & 0x01)==1
+      AND ((rowid>>31) & 0x1F)==0
+  } {
+    set n [string length $block]
+    set block [string range $block 0 11]
+    append block [string repeat "00" [expr ($n/2)-7-2]]
+    append block "8080"
+    db eval {
+      UPDATE t1_data SET block = unhex($block) WHERE rowid = $a
+    }
+  }
+} {}
+
+do_catchsql_test 4.2 {
+  SELECT count(*) FROM (
+      SELECT rowid FROM t1 WHERE t1 MATCH 'xyzterm' ORDER BY rowid ASC
+  )
+} {0 5000}
+
+do_catchsql_test 4.3 {
+  SELECT count(*) FROM (
+      SELECT rowid FROM t1 WHERE t1 MATCH 'xyzterm' ORDER BY rowid DESC
+  )
+} {0 4987}
+
+do_test 4.4 {
+  db eval {
+    SELECT rowid AS a, hex(block) AS block FROM t1_data 
+    WHERE ((rowid>>36) & 0x01)==1
+      AND ((rowid>>31) & 0x1F)==0
+  } {
+    set n [string length $block]
+    set block [string range $block 0 $n-5]
+    append block [string repeat "00" 2]
+    db eval {
+      UPDATE t1_data SET block = unhex($block) WHERE rowid = $a
+    }
+  }
+} {}
+
+do_catchsql_test 4.5 {
+  SELECT count(*) FROM (
+      SELECT rowid FROM t1 WHERE t1 MATCH 'xyzterm' ORDER BY rowid ASC
+  )
+} {0 5000}
+
+do_catchsql_test 4.6 {
+  SELECT count(*) FROM (
+      SELECT rowid FROM t1 WHERE t1 MATCH 'xyzterm' ORDER BY rowid DESC
+  )
+} {0 4879}
+
+#-------------------------------------------------------------------------
+reset_db
+
+do_execsql_test 5.0 {
+  CREATE VIRTUAL TABLE t USING fts5(x);
+  INSERT INTO t(t,rank) VALUES('pgsz', 64);
+  WITH s(i) AS (
+    SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<80
+  )
+  INSERT INTO t SELECT 'shared word' || (i%10) || ' shared shared' FROM s;
+  INSERT INTO t(t) VALUES('optimize');
+}
+
+set lLeaf [db eval { SELECT id FROM t_data }]
+db_save_and_close
+
+foreach leaf $lLeaf {
+  db_restore_and_reopen
+  set leaf [expr $leaf]
+  set hex [db one { 
+    SELECT hex(block) FROM t_data WHERE id=$leaf 
+  }]
+
+  # Replace the first 4 bytes of each leaf page with the size of the leaf in
+  # bytes plus 50 as a 2 byte integer, followed by 0x7FFF.
+  #
+  set nn [expr [string length $hex]/2]
+  set first "[format %.4x [expr $nn+50]]7FFF"
+  set hex [string replace $hex 0 7 $first]
+
+  db eval { UPDATE t_data SET block=unhex($hex) WHERE id=$leaf }
+  do_test 5.1.$leaf {
+    catchsql {
+      SELECT rowid FROM t WHERE t MATCH 'shared' ORDER BY rowid DESC;
+    }
+    set {} {}
+  } {}
+
+  do_test 5.1.$leaf.2 {
+    catchsql {
+      PRAGMA integrity_check;
+    }
+    set {} {}
+  } {}
+}
+
+#-------------------------------------------------------------------------
+reset_db
+
+do_execsql_test 6.0 {
+  CREATE VIRTUAL TABLE t USING fts5(x, content='', contentless_delete=1);
+  INSERT INTO t(rowid,x) 
+    VALUES(1,'a b'), (2,'c d'), (3,'e f'), (4,'g h'), (5,'i j');
+  INSERT INTO t(t) VALUES('optimize');
+  DELETE FROM t WHERE rowid=2;
+}
+
+do_test 6.1 {
+  db eval {
+    SELECT rowid AS rid, hex(block) AS blk 
+    FROM t_data WHERE rowid>1_000_000_000_000
+  } {}
+
+  set blk [string replace $blk 8 15 20000000]
+  execsql {
+    UPDATE t_data SET block = unhex($blk) WHERE rowid=$rid
+  }
+} {}
+
+do_execsql_test 6.2 {
+  DELETE FROM t WHERE rowid=3;
+}
+
+
+sqlite3_fts5_may_be_corrupt 0
+finish_test
+
+
@@ -65,5 +65,14 @@ do_eqp_test 1.4 {
  `--SCAN vt VIRTUAL TABLE INDEX 0:=
 }

+do_eqp_test 1.5 {
+  SELECT * FROM vt, t1 
+  WHERE vt.rowid = t1.rowid AND vt MATCH ? AND b = ?
+} {
+  QUERY PLAN
+  |--SCAN vt VIRTUAL TABLE INDEX 0:M1
+  `--SEARCH t1 USING INTEGER PRIMARY KEY (rowid=?)
+}
+

 finish_test
@@ -66,5 +66,18 @@ do_near_test 1.23 "a b c d e f g h i" { NEAR(a+b+c+d i b+c, 4) } 0
 do_near_test 1.24 "a b c d e f g h i" { NEAR(i a+b+c+d b+c, 5) } 1
 do_near_test 1.25 "a b c d e f g h i" { NEAR(i a+b+c+d b+c, 4) } 0

+#-------------------------------------------------------------------------
+# Check that https://sqlite.org/bugs/forumpost/16bb36f7e8 is fixed.
+#
+reset_db
+do_execsql_test 2.0 {
+  CREATE VIRTUAL TABLE t USING fts5(x, tokenize=trigram);
+  INSERT INTO t 
+  VALUES('aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa');
+}
+
+do_execsql_test 2.1 {
+  SELECT count(*) FROM t WHERE t MATCH 'NEAR(aaa aaa, 4)';
+} {1}

 finish_test
@@ -65,5 +65,13 @@ foreach {in out} $test_vocab {
  incr i
 }

+#-------------------------------------------------------------------------
+reset_db
+  
+do_execsql_test 2.0 {
+  CREATE VIRTUAL TABLE tbl USING fts5(t, tokenize='porter unicode61');
+  INSERT INTO tbl VALUES(' andeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee');
+}
+

 finish_test
@@ -418,6 +418,34 @@ do_execsql_test 7.1.2 {
  INSERT INTO t2(t2) VALUES('integrity-check');
 }

+#-------------------------------------------------------------------------
+
+reset_db
+fts5_aux_test_functions db
+
+proc tcl_create {args} { return "tcl_tokenize" }
+sqlite3_fts5_create_tokenizer db tcl tcl_create
+
+proc tcl_tokenize {tflags text} {
+  foreach {w iStart iEnd} [fts5_tokenize_split $text] {
+    sqlite3_fts5_token $w $iStart $iEnd
+    if {$w=="usa"} {
+      sqlite3_fts5_token -colo "united" $iStart $iEnd
+    }
+  }
+}
+
+do_execsql_test 8.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(x, tokenize='tcl', tokendata=1);
+  INSERT INTO t1 VALUES('usa is great');
+  INSERT INTO t1 VALUES('canada is cool');
+  INSERT INTO t1 VALUES('australia is what matters');
+}
+
+do_execsql_test 8.1 {
+  SELECT * FROM t1 WHERE t1 MATCH 'usa*';
+} {{usa is great}}
+
 } ;# foreach_detail_mode

 finish_test
@@ -371,4 +371,14 @@ do_execsql_test 10.10 {
  a 2 2 b 2 2 c 2 2 d 1 1 e 1 1 f 1 1
 }

+# Bug report 2026-05-18T05:19:01Z
+#
+set manyporters {porter Porter}
+for {set i 0} {$i<18} {incr i} {
+  set manyporters "$manyporters $manyporters"
+}
+do_execsql_test 11.1 "
+  CREATE VIRTUAL TABLE t11 USING fts5(a,b,tokenize=\"$manyporters\");
+" {}
+
 finish_test
@@ -0,0 +1,153 @@
+#
+# 2026 March 20
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#--------------------------------------------------------------------------
+#
+# This script extracts the documentation for the API used by fts5 auxiliary 
+# functions from header file fts5.h. It outputs html text on stdout that
+# is included in the documentation on the web.
+# 
+
+
+sqlite3 db fts5cost.db
+
+# Create an IPK table with 1,000,000 entries. Short records.
+#
+set res [list [catch { db eval {SELECT count(*) FROM t1} } msg] $msg]
+if {$res!="0 1000000"} {
+  db eval {
+    PRAGMA mmap_size = 1000000000;  -- 1GB
+    DROP TABLE IF EXISTS t1;
+    CREATE TABLE t1(a INTEGER PRIMARY KEY, b TEXT);
+    WITH s(i) AS (
+        SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<1_000_000
+    )
+    INSERT INTO t1 SELECT i, hex(randomblob(8)) FROM s;
+  }
+}
+
+# Create an FTS5 table with 1,000,000 entries. Each row contains a single
+# column containing a document of 100 terms chosen pseudo-randomly from
+# a vocabularly of 2000.
+set res [list [catch { db eval {SELECT count(*) FROM f1} } msg] $msg]
+if {$res!="0 1000000"} {
+  set nVocab 2000
+  set nTerm   100
+  db eval {
+    BEGIN;
+    DROP TABLE IF EXISTS vocab1;
+    CREATE TABLE vocab1(w);
+  }
+  for {set ii 0} {$ii<$nVocab} {incr ii} {
+    set word [format %06x [expr {int(abs(rand()) * 0xFFFFFF)}]]
+    db eval { INSERT INTO vocab1 VALUES($word) }
+    lappend lVocab $word
+  }
+  db func doc doc
+  proc doc {} {
+    for {set ii 0} {$ii<$::nTerm} {incr ii} {
+      lappend ret [lindex $::lVocab [expr int(abs(rand())*$::nVocab)]]
+    }
+    set ret
+  }
+  db eval {
+    DROP TABLE IF EXISTS f1;
+    CREATE VIRTUAL TABLE f1 USING fts5(x);
+    WITH s(i) AS (
+        SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<1_000_000
+    )
+    INSERT INTO f1(rowid, x) SELECT i, doc() FROM s;
+    COMMIT;
+  }
+} else {
+  set lVocab [db eval { SELECT * FROM vocab1 }]
+  set nVocab [llength $lVocab]
+}
+
+proc rowid_query {n} {
+  set rowid 654
+  for {set ii 0} {$ii<$n} {incr ii} {
+    db eval { SELECT b FROM t1 WHERE a = $rowid }
+    set rowid [expr {($rowid + 7717) % 1000000}]
+  }
+}
+
+proc rowid_query_fts {n} {
+  set rowid 654
+  for {set ii 0} {$ii<$n} {incr ii} {
+    db eval { SELECT * FROM f1 WHERE rowid = $rowid }
+    set rowid [expr {($rowid + 7717) % 1000000}]
+  }
+}
+
+proc match_query_fts {n} {
+  set idx 654
+  for {set ii 0} {$ii<$n} {incr ii} {
+    set match [lrange $::lVocab $idx $idx+1]
+    db eval { SELECT * FROM f1($match) }
+    set idx [expr {($idx + 7717) % $::nVocab}]
+  }
+}
+
+proc prefix_query_fts {n} {
+  set idx 654
+  for {set ii 0} {$ii<$n} {incr ii} {
+    set match "[lindex $::lVocab $idx]*"
+    db eval { SELECT * FROM f1($match) }
+    set idx [expr {($idx + 7717) % $::nVocab}]
+  }
+}
+
+proc match_rowid_query_fts {n} {
+  set idx 654
+  for {set ii 0} {$ii<$n} {incr ii} {
+    set match "[lindex $::lVocab $idx]"
+    db eval { SELECT * FROM f1($match) WHERE rowid=500000 }
+    set idx [expr {($idx + 7717) % $::nVocab}]
+  }
+}
+
+proc prefix_rowid_query_fts {n} {
+  set idx 654
+  for {set ii 0} {$ii<$n} {incr ii} {
+    set match "[lindex $::lVocab $idx]*"
+    db eval { SELECT * FROM f1($match) WHERE rowid=500000 }
+    set idx [expr {($idx + 7717) % $::nVocab}]
+  }
+}
+
+
+proc mytime {cmd div} {
+  set tm [time $cmd]
+  expr {[lindex $tm 0] / $div}
+}
+
+#set us [mytime { match_rowid_query_fts 1000 } 1000]
+#puts "1000 match/rowid queries on fts5 table: ${us} per query"
+
+set us [mytime { prefix_rowid_query_fts 1000 } 1000]
+puts "1000 prefix/rowid queries on fts5 table: ${us} per query"
+
+set us [mytime { match_query_fts 10 } 10]
+puts "10 match queries on fts5 table: ${us} per query"
+
+set us [mytime { prefix_query_fts 10 } 10]
+puts "10 prefix queries on fts5 table: ${us} per query"
+
+set us [mytime { prefix_rowid_query_fts 1000 } 1000]
+puts "1000 prefix/rowid queries on fts5 table: ${us} per query"
+
+set us [mytime { rowid_query 10000 } 10000]
+puts "10000 by-rowid queries on normal table: ${us} per query"
+
+set us [mytime { rowid_query_fts 10000 } 10000]
+puts "10000 by-rowid queries on fts5 table: ${us} per query"
+
+
@@ -105,6 +105,9 @@ static const unsigned char icuUtf8Trans1[] = {
    while( (*zIn & 0xc0)==0x80 ){                          \
      c = (c<<6) + (0x3f & *(zIn++));                      \
    }                                                      \
+    if( c<0x80                                             \
+        || (c&0xFFFFF800)==0xD800                          \
+        || (c&0xFFFFFFFE)==0xFFFE ){  c = 0xFFFD; }        \
  }

 #define SQLITE_ICU_SKIP_UTF8(zIn)                          \
@@ -362,7 +365,7 @@ static void icuCaseFunc16(sqlite3_context *p, int nArg, sqlite3_value **apArg){
  const UChar *zInput;            /* Pointer to input string */
  UChar *zOutput = 0;             /* Pointer to output buffer */
  int nInput;                     /* Size of utf-16 input string in bytes */
-  int nOut;                       /* Size of output buffer in bytes */
+  sqlite3_int64 nOut;             /* Size of output buffer in bytes */
  int cnt;
  int bToUpper;                   /* True for toupper(), false for tolower() */
  UErrorCode status;
@@ -385,7 +388,7 @@ static void icuCaseFunc16(sqlite3_context *p, int nArg, sqlite3_value **apArg){
  }

  for(cnt=0; cnt<2; cnt++){
-    UChar *zNew = sqlite3_realloc(zOutput, nOut);
+    UChar *zNew = sqlite3_realloc64(zOutput, nOut);
    if( zNew==0 ){
      sqlite3_free(zOutput);
      sqlite3_result_error_nomem(p);
@@ -394,9 +397,9 @@ static void icuCaseFunc16(sqlite3_context *p, int nArg, sqlite3_value **apArg){
    zOutput = zNew;
    status = U_ZERO_ERROR;
    if( bToUpper ){
-      nOut = 2*u_strToUpper(zOutput,nOut/2,zInput,nInput/2,zLocale,&status);
+      nOut = 2LL*u_strToUpper(zOutput,nOut/2,zInput,nInput/2,zLocale,&status);
    }else{
-      nOut = 2*u_strToLower(zOutput,nOut/2,zInput,nInput/2,zLocale,&status);
+      nOut = 2LL*u_strToLower(zOutput,nOut/2,zInput,nInput/2,zLocale,&status);
    }

    if( U_SUCCESS(status) ){
@@ -516,6 +519,7 @@ static void icuLoadCollation(
      }
      sqlite3_result_error(p, sqlite3_str_value(pStr), -1);
      sqlite3_free(sqlite3_str_finish(pStr));
+      ucol_close(pUCollator);
      return;
    }
  }
@@ -847,7 +847,7 @@ static int amatchConnect(

  (void)pAux;
  *ppVtab = 0;
-  pNew = sqlite3_malloc( sizeof(*pNew) );
+  pNew = sqlite3_malloc64( sizeof(*pNew) );
  if( pNew==0 ) return SQLITE_NOMEM;
  rc = SQLITE_NOMEM;
  memset(pNew, 0, sizeof(*pNew));
@@ -928,7 +928,7 @@ amatchConnectError:
 static int amatchOpen(sqlite3_vtab *pVTab, sqlite3_vtab_cursor **ppCursor){
  amatch_vtab *p = (amatch_vtab*)pVTab;
  amatch_cursor *pCur;
-  pCur = sqlite3_malloc( sizeof(*pCur) );
+  pCur = sqlite3_malloc64( sizeof(*pCur) );
  if( pCur==0 ) return SQLITE_NOMEM;
  memset(pCur, 0, sizeof(*pCur));
  pCur->pVtab = p;
@@ -1157,7 +1157,7 @@ static int amatchNext(sqlite3_vtab_cursor *cur){
 #endif
    nWord = (int)strlen(pWord->zWord+2);
    if( nWord+20>nBuf ){
-      nBuf = (char)(nWord+100);
+      nBuf = nWord+100;
      zBuf = sqlite3_realloc64(zBuf, nBuf);
      if( zBuf==0 ) return SQLITE_NOMEM;
    }
@@ -232,7 +232,7 @@ static void base64(sqlite3_context *context, int na, sqlite3_value *av[]){
      sqlite3_result_text(context,"",-1,SQLITE_STATIC);
      break;
    }
-    cBuf = sqlite3_malloc(nc);
+    cBuf = sqlite3_malloc64(nc);
    if( !cBuf ) goto memFail;
    nc = (int)(toBase64(bBuf, nb, cBuf) - cBuf);
    sqlite3_result_text(context, cBuf, nc, sqlite3_free);
@@ -254,7 +254,7 @@ static void base64(sqlite3_context *context, int na, sqlite3_value *av[]){
      sqlite3_result_zeroblob(context, 0);
      break;
    }
-    bBuf = sqlite3_malloc(nb);
+    bBuf = sqlite3_malloc64(nb);
    if( !bBuf ) goto memFail;
    nb = (int)(fromBase64(cBuf, nc, bBuf) - bBuf);
    sqlite3_result_blob(context, bBuf, nb, sqlite3_free);
@@ -309,7 +309,7 @@ static void base85(sqlite3_context *context, int na, sqlite3_value *av[]){
      sqlite3_result_text(context,"",-1,SQLITE_STATIC);
      break;
    }
-    cBuf = sqlite3_malloc(nc);
+    cBuf = sqlite3_malloc64(nc);
    if( !cBuf ) goto memFail;
    nc = (int)(toBase85(bBuf, nb, cBuf, "\n") - cBuf);
    sqlite3_result_text(context, cBuf, nc, sqlite3_free);
@@ -331,7 +331,7 @@ static void base85(sqlite3_context *context, int na, sqlite3_value *av[]){
      sqlite3_result_zeroblob(context, 0);
      break;
    }
-    bBuf = sqlite3_malloc(nb);
+    bBuf = sqlite3_malloc64(nb);
    if( !bBuf ) goto memFail;
    nb = (int)(fromBase85(cBuf, nc, bBuf) - bBuf);
    sqlite3_result_blob(context, bBuf, nb, sqlite3_free);
@@ -1,152 +0,0 @@
-/*
-** 2019-03-30
-**
-** The author disclaims copyright to this source code.  In place of
-** a legal notice, here is a blessing:
-**
-**    May you do good and not evil.
-**    May you find forgiveness for yourself and forgive others.
-**    May you share freely, never taking more than you give.
-**
-******************************************************************************
-**
-** An SQL function that uses the incremental BLOB I/O mechanism of SQLite
-** to read or write part of a blob.  This is intended for debugging use
-** in the CLI.
-**
-**      readblob(SCHEMA,TABLE,COLUMN,ROWID,OFFSET,N)
-**
-** Returns N bytes of the blob starting at OFFSET.
-**
-**      writeblob(SCHEMA,TABLE,COLUMN,ROWID,OFFSET,NEWDATA)
-**
-** NEWDATA must be a blob.  The content of NEWDATA overwrites the
-** existing BLOB data at SCHEMA.TABLE.COLUMN for row ROWID beginning
-** at OFFSET bytes into the blob.
-*/
-#include "sqlite3ext.h"
-SQLITE_EXTENSION_INIT1
-#include <assert.h>
-#include <string.h>
-
-static void readblobFunc(
-  sqlite3_context *context,
-  int argc,
-  sqlite3_value **argv
-){
-  sqlite3_blob *pBlob = 0;
-  const char *zSchema;
-  const char *zTable;
-  const char *zColumn;
-  sqlite3_int64 iRowid;
-  int iOfst;
-  unsigned char *aData;
-  int nData;
-  sqlite3 *db;
-  int rc;
-
-  zSchema = (const char*)sqlite3_value_text(argv[0]);
-  zTable = (const char*)sqlite3_value_text(argv[1]);
-  if( zTable==0 ){
-    sqlite3_result_error(context, "bad table name", -1);
-    return;
-  }
-  zColumn = (const char*)sqlite3_value_text(argv[2]);
-  if( zTable==0 ){
-    sqlite3_result_error(context, "bad column name", -1);
-    return;
-  }
-  iRowid = sqlite3_value_int64(argv[3]);
-  iOfst = sqlite3_value_int(argv[4]);
-  nData = sqlite3_value_int(argv[5]);
-  if( nData<=0 ) return;
-  aData = sqlite3_malloc64( nData+1 );
-  if( aData==0 ){
-    sqlite3_result_error_nomem(context);
-    return;
-  }
-  db = sqlite3_context_db_handle(context);
-  rc = sqlite3_blob_open(db, zSchema, zTable, zColumn, iRowid, 0, &pBlob);
-  if( rc ){
-    sqlite3_free(aData);
-    sqlite3_result_error(context, "cannot open BLOB pointer", -1);
-    return;
-  }
-  rc = sqlite3_blob_read(pBlob, aData, nData, iOfst);
-  sqlite3_blob_close(pBlob);
-  if( rc ){
-    sqlite3_free(aData);
-    sqlite3_result_error(context, "BLOB read failed", -1);
-  }else{
-    sqlite3_result_blob(context, aData, nData, sqlite3_free);
-  }
-}    
-
-static void writeblobFunc(
-  sqlite3_context *context,
-  int argc,
-  sqlite3_value **argv
-){
-  sqlite3_blob *pBlob = 0;
-  const char *zSchema;
-  const char *zTable;
-  const char *zColumn;
-  sqlite3_int64 iRowid;
-  int iOfst;
-  unsigned char *aData;
-  int nData;
-  sqlite3 *db;
-  int rc;
-
-  zSchema = (const char*)sqlite3_value_text(argv[0]);
-  zTable = (const char*)sqlite3_value_text(argv[1]);
-  if( zTable==0 ){
-    sqlite3_result_error(context, "bad table name", -1);
-    return;
-  }
-  zColumn = (const char*)sqlite3_value_text(argv[2]);
-  if( zTable==0 ){
-    sqlite3_result_error(context, "bad column name", -1);
-    return;
-  }
-  iRowid = sqlite3_value_int64(argv[3]);
-  iOfst = sqlite3_value_int(argv[4]);
-  if( sqlite3_value_type(argv[5])!=SQLITE_BLOB ){
-    sqlite3_result_error(context, "6th argument must be a BLOB", -1);
-    return;
-  }
-  nData = sqlite3_value_bytes(argv[5]);
-  aData = (unsigned char *)sqlite3_value_blob(argv[5]);
-  db = sqlite3_context_db_handle(context);
-  rc = sqlite3_blob_open(db, zSchema, zTable, zColumn, iRowid, 1, &pBlob);
-  if( rc ){
-    sqlite3_result_error(context, "cannot open BLOB pointer", -1);
-    return;
-  }
-  rc = sqlite3_blob_write(pBlob, aData, nData, iOfst);
-  sqlite3_blob_close(pBlob);
-  if( rc ){
-    sqlite3_result_error(context, "BLOB write failed", -1);
-  }
-}    
-
-
-#ifdef _WIN32
-__declspec(dllexport)
-#endif
-int sqlite3_blobio_init(
-  sqlite3 *db, 
-  char **pzErrMsg, 
-  const sqlite3_api_routines *pApi
-){
-  int rc = SQLITE_OK;
-  SQLITE_EXTENSION_INIT2(pApi);
-  (void)pzErrMsg;  /* Unused parameter */
-  rc = sqlite3_create_function(db, "readblob", 6, SQLITE_UTF8, 0,
-                               readblobFunc, 0, 0);
-  if( rc==SQLITE_OK ){
-    rc = sqlite3_create_function(db, "writeblob", 6, SQLITE_UTF8, 0,
-                               writeblobFunc, 0, 0);
-  }
-  return rc;
-}
@@ -60,6 +60,13 @@
 **
 **      SELECT name FROM sqlite_btreeinfo
 **       WHERE type='table' AND NOT hasRowid;
+**
+** UNUSED, UNTESTED, UNSUPPORTED
+**
+** This extension exists for information and demonstration purposes
+** only.  The developers are not aware of any real-world use of this
+** extension.  The extension has no formal test cases.  The developers
+** do not actively support it.
 */
 #if !defined(SQLITEINT_H)
 #include "sqlite3ext.h"
@@ -286,6 +293,11 @@ static int binfoCompute(sqlite3 *db, int pgno, BinfoCursor *pCsr){
  pCsr->depth = 1;
  while(1){
    sqlite3_bind_int(pStmt, 1, pgno);
+    if( pCsr->depth>25 ){
+      sqlite3_set_errmsg(db, SQLITE_CORRUPT, "btree nested too deep");
+      rc = SQLITE_ERROR;
+      break;
+    }
    rc = sqlite3_step(pStmt);
    if( rc!=SQLITE_ROW ){
      rc = SQLITE_ERROR;
@@ -516,7 +516,7 @@ static int closureConnect(

  (void)pAux;
  *ppVtab = 0;
-  pNew = sqlite3_malloc( sizeof(*pNew) );
+  pNew = sqlite3_malloc64( sizeof(*pNew) );
  if( pNew==0 ) return SQLITE_NOMEM;
  rc = SQLITE_NOMEM;
  memset(pNew, 0, sizeof(*pNew));
@@ -579,7 +579,7 @@ closureConnectError:
 static int closureOpen(sqlite3_vtab *pVTab, sqlite3_vtab_cursor **ppCursor){
  closure_vtab *p = (closure_vtab*)pVTab;
  closure_cursor *pCur;
-  pCur = sqlite3_malloc( sizeof(*pCur) );
+  pCur = sqlite3_malloc64( sizeof(*pCur) );
  if( pCur==0 ) return SQLITE_NOMEM;
  memset(pCur, 0, sizeof(*pCur));
  pCur->pVtab = p;
@@ -638,7 +638,7 @@ static int closureInsertNode(
  sqlite3_int64 id,       /* The node ID */
  int iGeneration         /* The generation number for this node */
 ){
-  closure_avl *pNew = sqlite3_malloc( sizeof(*pNew) );
+  closure_avl *pNew = sqlite3_malloc64( sizeof(*pNew) );
  if( pNew==0 ) return SQLITE_NOMEM;
  memset(pNew, 0, sizeof(*pNew));
  pNew->id = id;
@@ -132,7 +132,7 @@ static int completionConnect(
      "  phase INT HIDDEN"        /* Used for debugging only */
      ")");
  if( rc==SQLITE_OK ){
-    pNew = sqlite3_malloc( sizeof(*pNew) );
+    pNew = sqlite3_malloc64( sizeof(*pNew) );
    *ppVtab = (sqlite3_vtab*)pNew;
    if( pNew==0 ) return SQLITE_NOMEM;
    memset(pNew, 0, sizeof(*pNew));
@@ -154,7 +154,7 @@ static int completionDisconnect(sqlite3_vtab *pVtab){
 */
 static int completionOpen(sqlite3_vtab *p, sqlite3_vtab_cursor **ppCursor){
  completion_cursor *pCur;
-  pCur = sqlite3_malloc( sizeof(*pCur) );
+  pCur = sqlite3_malloc64( sizeof(*pCur) );
  if( pCur==0 ) return SQLITE_NOMEM;
  memset(pCur, 0, sizeof(*pCur));
  pCur->db = ((completion_vtab*)p)->db;
@@ -199,6 +199,7 @@ static int completionNext(sqlite3_vtab_cursor *cur){
  completion_cursor *pCur = (completion_cursor*)cur;
  int eNextPhase = 0;  /* Next phase to try if current phase reaches end */
  int iCol = -1;       /* If >=0, step pCur->pStmt and use the i-th column */
+  int rc;
  pCur->iRowid++;
  while( pCur->ePhase!=COMPLETION_EOF ){
    switch( pCur->ePhase ){
@@ -237,11 +238,14 @@ static int completionNext(sqlite3_vtab_cursor *cur){
            );
            zSep = " UNION ";
          }
-          sqlite3_finalize(pS2);
+          rc = sqlite3_finalize(pS2);
          zSql = sqlite3_str_finish(pStr);
          if( zSql==0 ) return SQLITE_NOMEM;
-          sqlite3_prepare_v2(pCur->db, zSql, -1, &pCur->pStmt, 0);
+          if( rc==SQLITE_OK ){
+            sqlite3_prepare_v2(pCur->db, zSql, -1, &pCur->pStmt, 0);
+          }
          sqlite3_free(zSql);
+          if( rc ) return rc;
        }
        iCol = 0;
        eNextPhase = COMPLETION_COLUMNS;
@@ -265,11 +269,14 @@ static int completionNext(sqlite3_vtab_cursor *cur){
            );
            zSep = " UNION ";
          }
-          sqlite3_finalize(pS2);
+          rc = sqlite3_finalize(pS2);
          zSql = sqlite3_str_finish(pStr);
          if( zSql==0 ) return SQLITE_NOMEM;
-          sqlite3_prepare_v2(pCur->db, zSql, -1, &pCur->pStmt, 0);
+          if( rc==SQLITE_OK ){
+            sqlite3_prepare_v2(pCur->db, zSql, -1, &pCur->pStmt, 0);
+          }
          sqlite3_free(zSql);
+          if( rc ) return rc;
        }
        iCol = 0;
        eNextPhase = COMPLETION_EOF;
@@ -286,9 +293,10 @@ static int completionNext(sqlite3_vtab_cursor *cur){
        pCur->szRow = sqlite3_column_bytes(pCur->pStmt, iCol);
      }else{
        /* When all rows are finished, advance to the next phase */
-        sqlite3_finalize(pCur->pStmt);
+        rc = sqlite3_finalize(pCur->pStmt);
        pCur->pStmt = 0;
        pCur->ePhase = eNextPhase;
+        if( rc ) return rc;
        continue;
      }
    }
@@ -60,6 +60,10 @@ static void compressFunc(
  nIn = sqlite3_value_bytes(argv[0]);
  nOut = 13 + nIn + (nIn+999)/1000;
  pOut = sqlite3_malloc64( nOut+5 );
+  if( pOut==0 ){
+    sqlite3_result_error_nomem(context);
+    return;
+  }
  for(i=4; i>=0; i--){
    x[i] = (nIn >> (7*(4-i)))&0x7f;
  }
@@ -99,6 +103,10 @@ static void uncompressFunc(
    if( (pIn[i]&0x80)!=0 ){ i++; break; }
  }
  pOut = sqlite3_malloc64( nOut+1 );
+  if( pOut==0 ){
+    sqlite3_result_error_nomem(context);
+    return;
+  }
  rc = uncompress(pOut, &nOut, &pIn[i], nIn-i);
  if( rc==Z_OK ){
    sqlite3_result_blob(context, pOut, nOut, sqlite3_free);
@@ -64,6 +64,7 @@ SQLITE_EXTENSION_INIT1

 #ifndef SQLITEINT_H
 typedef sqlite3_int64 i64;
+typedef sqlite3_uint64 u64;
 #endif

 /* Max size of the error message in a CsvReader */
@@ -128,7 +129,7 @@ static int csv_reader_open(
  const char *zData           /*  ... or use this data */
 ){
  if( zFilename ){
-    p->zIn = sqlite3_malloc( CSV_INBUFSZ );
+    p->zIn = sqlite3_malloc64( CSV_INBUFSZ );
    if( p->zIn==0 ){
      csv_errmsg(p, "out of memory");
      return 1;
@@ -221,7 +222,7 @@ static char *csv_read_one_field(CsvReader *p){
  }
  if( c=='"' ){
    int pc, ppc;
-    int startLine = p->nLine;
+    i64 startLine = p->nLine;
    pc = ppc = 0;
    while( 1 ){
      c = csv_getc(p);
@@ -325,7 +326,7 @@ typedef struct CsvCursor {
  sqlite3_vtab_cursor base;       /* Base class.  Must be first */
  CsvReader rdr;                  /* The CsvReader object */
  char **azVal;                   /* Value of the current row */
-  int *aLen;                      /* Length of each entry */
+  i64 *aLen;                      /* Length of each entry */
  sqlite3_int64 iRowid;           /* The current rowid.  Negative for EOF */
 } CsvCursor;

@@ -497,7 +498,7 @@ static int csvtabConnect(
  CsvTable *pNew = 0;        /* The CsvTable object to construct */
  int bHeader = -1;          /* header= flags.  -1 means not seen yet */
  int rc = SQLITE_OK;        /* Result code from this routine */
-  int i, j;                  /* Loop counters */
+  u64 i, j;                  /* Loop counters */
 #ifdef SQLITE_TEST
  int tstFlags = 0;          /* Value for testflags=N parameter */
 #endif
@@ -516,7 +517,7 @@ static int csvtabConnect(
  assert( sizeof(azPValue)==sizeof(azParam) );
  memset(&sRdr, 0, sizeof(sRdr));
  memset(azPValue, 0, sizeof(azPValue));
-  for(i=3; i<argc; i++){
+  for(i=3; i<(u64)argc; i++){
    const char *z = argv[i];
    const char *zValue;
    for(j=0; j<sizeof(azParam)/sizeof(azParam[0]); j++){
@@ -546,6 +547,10 @@ static int csvtabConnect(
      if( nCol<=0 ){
        csv_errmsg(&sRdr, "column= value must be positive");
        goto csvtab_connect_error;
+      }else if( nCol>sqlite3_limit(db,SQLITE_LIMIT_COLUMN,-1) ){
+        csv_errmsg(&sRdr, "column= value too big, max %d",
+                   sqlite3_limit(db, SQLITE_LIMIT_COLUMN,-1));
+        goto csvtab_connect_error;
      }
    }else
    {
@@ -563,7 +568,7 @@ static int csvtabConnect(
  ){
    goto csvtab_connect_error;
  }
-  pNew = sqlite3_malloc( sizeof(*pNew) );
+  pNew = sqlite3_malloc64( sizeof(*pNew) );
  *ppVtab = (sqlite3_vtab*)pNew;
  if( pNew==0 ) goto csvtab_connect_oom;
  memset(pNew, 0, sizeof(*pNew));
@@ -708,13 +713,14 @@ static int csvtabClose(sqlite3_vtab_cursor *cur){
 static int csvtabOpen(sqlite3_vtab *p, sqlite3_vtab_cursor **ppCursor){
  CsvTable *pTab = (CsvTable*)p;
  CsvCursor *pCur;
-  size_t nByte;
-  nByte = sizeof(*pCur) + (sizeof(char*)+sizeof(int))*pTab->nCol;
+  sqlite3_int64 nByte = pTab->nCol;
+  assert( pTab->nCol<32768 );
+  nByte = sizeof(*pCur) + (sizeof(char*)+sizeof(i64))*nByte;
  pCur = sqlite3_malloc64( nByte );
  if( pCur==0 ) return SQLITE_NOMEM;
  memset(pCur, 0, nByte);
  pCur->azVal = (char**)&pCur[1];
-  pCur->aLen = (int*)&pCur->azVal[pTab->nCol];
+  pCur->aLen = (i64*)&pCur->azVal[pTab->nCol];
  *ppCursor = &pCur->base;
  if( csv_reader_open(&pCur->rdr, pTab->zFilename, pTab->zData) ){
    csv_xfer_error(pTab, &pCur->rdr);
@@ -31,6 +31,10 @@ SQLITE_EXTENSION_INIT1
 #define IsSpace(X)  isspace((unsigned char)X)
 #endif

+#ifndef SQLITE_DECIMAL_MAX_DIGIT
+# define SQLITE_DECIMAL_MAX_DIGIT 10000000
+#endif
+
 /* A decimal object */
 typedef struct Decimal Decimal;
 struct Decimal {
@@ -70,7 +74,7 @@ static Decimal *decimalNewFromText(const char *zIn, int n){
  int iExp = 0;

  if( zIn==0 ) goto new_from_text_failed;
-  p = sqlite3_malloc( sizeof(*p) );
+  p = sqlite3_malloc64( sizeof(*p) );
  if( p==0 ) goto new_from_text_failed;
  p->sign = 0;
  p->oom = 0;
@@ -164,9 +168,7 @@ static Decimal *decimalNewFromText(const char *zIn, int n){
    for(i=0; i<p->nDigit && p->a[i]==0; i++){}
    if( i>=p->nDigit ) p->sign = 0;
  }
-#if SQLITE_DECIMAL_MAX_DIGIT+0>10
  if( p->nDigit>SQLITE_DECIMAL_MAX_DIGIT ) goto new_from_text_failed;
-#endif
  return p;

 new_from_text_failed:
@@ -259,7 +261,7 @@ static void decimal_result(sqlite3_context *pCtx, Decimal *p){
    sqlite3_result_null(pCtx);
    return;
  }
-  z = sqlite3_malloc64( (sqlite3_int64)p->nDigit+4 );
+  z = sqlite3_malloc64( (sqlite3_int64)p->nDigit+8 );
  if( z==0 ){
    sqlite3_result_error_nomem(pCtx);
    return;
@@ -297,27 +299,37 @@ static void decimal_result(sqlite3_context *pCtx, Decimal *p){
  sqlite3_result_text(pCtx, z, i, sqlite3_free);
 }

+/* Forward declaration */
+static void decimal_expand(Decimal *p, int nDigit, int nFrac);
+
 /*
 ** Round a decimal value to N significant digits.  N must be positive.
 */
 static void decimal_round(Decimal *p, int N){
  int i;
-  int nZero;
+  int nZero;  /* Number of leading zeros */
  if( N<1 ) return;
  if( p==0 ) return;
+  if( p->nDigit<=N ) return;
  for(nZero=0; nZero<p->nDigit && p->a[nZero]==0; nZero++){}
  N += nZero;
  if( p->nDigit<=N ) return;
-  if( p->a[N]>4 ){
+  if( p->a[N]>=5 ){
+    /* If all leading digits are 9, increase the number of digits
+    ** by adding a new 0 to the front */
+    for(i=0; i<N && p->a[i]==9; i++){}
+    if( i==N ){
+      decimal_expand(p, p->nDigit+1, 0);
+      if( p->oom ) return;
+    }
+
+    /* Do the rounding */
    p->a[N-1]++;
    for(i=N-1; i>0 && p->a[i]>9; i--){
      p->a[i] = 0;
      p->a[i-1]++;
    }
-    if( p->a[0]>9 ){
-      p->a[0] = 1;
-      p->nFrac--;
-    }
+    assert( p->a[0]<=9 );
  }
  memset(&p->a[N], 0, p->nDigit - N);
 }
@@ -462,18 +474,18 @@ cmp_done:
 static void decimal_expand(Decimal *p, int nDigit, int nFrac){
  int nAddSig;
  int nAddFrac;
+  signed char *a;
  if( p==0 ) return;
  nAddFrac = nFrac - p->nFrac;
  nAddSig = (nDigit - p->nDigit) - nAddFrac;
  if( nAddFrac==0 && nAddSig==0 ) return;
-#if SQLITE_DECIMAL_MAX_DIGIT+0>10
  if( nDigit+1>SQLITE_DECIMAL_MAX_DIGIT ){ p->oom = 1; return; }
-#endif
-  p->a = sqlite3_realloc64(p->a, nDigit+1);
-  if( p->a==0 ){
+  a = sqlite3_realloc64(p->a, nDigit+1);
+  if( a==0 ){
    p->oom = 1;
    return;
  }
+  p->a = a;
  if( nAddSig ){
    memmove(p->a+nAddSig, p->a, p->nDigit);
    memset(p->a, 0, nAddSig);
@@ -578,9 +590,7 @@ static void decimalMul(Decimal *pA, Decimal *pB){
  sumDigit = pA->nDigit;
  sumDigit += pB->nDigit;
  sumDigit += 2;
-#if SQLITE_DECIMAL_MAX_DIGIT+0>10
  if( sumDigit>SQLITE_DECIMAL_MAX_DIGIT ){ pA->oom = 1; return; }
-#endif
  acc = sqlite3_malloc64( sumDigit );
  if( acc==0 ){
    pA->oom = 1;
@@ -813,7 +823,7 @@ static void decimalSumStep(
  if( p==0 ) return;
  if( !p->isInit ){
    p->isInit = 1;
-    p->a = sqlite3_malloc(2);
+    p->a = sqlite3_malloc64(2);
    if( p->a==0 ){
      p->oom = 1;
    }else{
@@ -0,0 +1,847 @@
+/*
+** 2026-04-13
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+******************************************************************************
+**
+** This extension implements an SQL function:
+**
+**     diskused(X)
+**
+** Where X is the schema name (typically 'main').  The output is text
+** that describes how much filesystem space the various tables and indexes
+** of the database consume.
+**
+** This function is a replacement for the (now deprecated)
+** "sqlite3_analyzer" utility program.  This function is built
+** into the CLI and is used to implement the ".diskused" command there.
+*/
+#include "sqlite3ext.h"
+SQLITE_EXTENSION_INIT1
+#include <assert.h>
+#include <string.h>
+#include <ctype.h>
+#include <math.h>
+
+/*
+** State information for the analysis
+*/
+typedef struct DiskUsed DiskUsed;
+struct DiskUsed {
+  sqlite3 *db;               /* Database connection */
+  sqlite3_context *context;  /* SQL function context */
+  sqlite3_str *pOut;         /* Write output here */
+  char *zSU;                 /* Name of the temp.space_used table */
+  const char *zSchema;       /* Schema to be analyzed */
+};
+
+/*
+** Free all resources that the DiskUsed object references and
+** reset the DiskUsed object.
+**
+** Call this routine multiple times on the same DiskUsed object
+** is a harmless no-op, as long as the memory for the object itself
+** has not been freed.
+*/
+static void diskusedReset(DiskUsed *p){
+  if( p->zSU ){
+    char *zSql = sqlite3_mprintf("DROP TABLE temp.%s;", p->zSU);
+    if( zSql ){
+      sqlite3_exec(p->db, zSql, 0, 0, 0);
+      sqlite3_free(zSql);
+    }
+  }
+  sqlite3_str_free(p->pOut);
+  sqlite3_free(p->zSU);
+  memset(p, 0, sizeof(*p));
+}
+
+/*
+** Report an error using formatted text.  If zFormat==NULL then report
+** an OOM error.
+*/
+static void diskusedError(DiskUsed *p, const char *zFormat, ...){
+  char *zErr;
+  if( zFormat ){
+    va_list ap;
+    va_start(ap, zFormat);
+    zErr = sqlite3_vmprintf(zFormat, ap);
+    va_end(ap);
+  }else{
+    zErr = 0;
+  }
+  if( zErr==0 ){
+    sqlite3_result_error_nomem(p->context);
+  }else{
+    sqlite3_result_error(p->context, zErr, -1);
+    sqlite3_free(zErr);
+  }
+  diskusedReset(p);
+}
+
+/*
+** Prepare and return an SQL statement.
+*/
+static sqlite3_stmt *diskusedVPrep(DiskUsed *p, const char *zFmt, va_list ap){
+  char *zSql;
+  int rc;
+  sqlite3_stmt *pStmt = 0;
+  zSql = sqlite3_vmprintf(zFmt, ap);
+  if( zSql==0 ){ diskusedError(p,0); return 0; }
+  rc = sqlite3_prepare_v2(p->db, zSql, -1, &pStmt, 0);
+  if( rc ){
+    diskusedError(p, "SQL parse error: %s\nOriginal SQL: %s",
+                                 sqlite3_errmsg(p->db), zSql);
+    sqlite3_finalize(pStmt);
+    diskusedReset(p);
+    pStmt = 0;
+  }
+  sqlite3_free(zSql);
+  return pStmt;
+}
+static sqlite3_stmt *diskusedPrepare(DiskUsed *p, const char *zFormat, ...){
+  va_list ap;
+  sqlite3_stmt *pStmt = 0;
+  va_start(ap, zFormat);
+  pStmt = diskusedVPrep(p,zFormat,ap);
+  va_end(ap);
+  return pStmt;
+}
+
+/*
+** If rc is something other than SQLITE_DONE or SQLITE_OK, then report
+** an error and return true.
+**
+** If rc is SQLITE_DONE or SQLITE_OK, then return false.
+**
+** The prepared statement is closed in either case.
+*/
+static int diskusedStmtFinish(DiskUsed *p, int rc, sqlite3_stmt *pStmt){
+  if( rc==SQLITE_DONE ){
+    rc = SQLITE_OK;
+  }
+  if( rc!=SQLITE_OK || (rc = sqlite3_reset(pStmt))!=SQLITE_OK ){
+    diskusedError(p, "SQL run-time error: %s\nOriginal SQL: %s",
+                  sqlite3_errmsg(p->db), sqlite3_sql(pStmt));
+    diskusedReset(p);
+  }
+  sqlite3_finalize(pStmt);
+  return rc;
+}
+
+/*
+** Run SQL.  Return the number of errors. 
+*/
+static int diskusedSql(DiskUsed *p, const char *zFormat, ...){
+  va_list ap;
+  int rc;
+  sqlite3_stmt *pStmt = 0;
+  va_start(ap, zFormat);
+  pStmt = diskusedVPrep(p,zFormat,ap);
+  va_end(ap);
+  if( pStmt==0 ) return 1;
+  while( (rc = sqlite3_step(pStmt))==SQLITE_ROW ){}
+  if( rc==SQLITE_DONE ){
+    rc = SQLITE_OK;
+  }else{
+    diskusedError(p, "SQL run-time error: %s\nOriginal SQL: %s",
+                  sqlite3_errmsg(p->db), sqlite3_sql(pStmt));
+    diskusedReset(p);
+  }
+  sqlite3_finalize(pStmt);
+  return rc;
+}
+
+/*
+** Run an SQL query that returns an integer.  Write that integer
+** into *piRes.  Return the number of errors. 
+*/
+static int diskusedSqlInt(
+  DiskUsed *p,
+  sqlite3_int64 *piRes,
+  const char *zFormat, ...
+){
+  va_list ap;
+  int rc;
+  sqlite3_stmt *pStmt = 0;
+  va_start(ap, zFormat);
+  pStmt = diskusedVPrep(p,zFormat,ap);
+  va_end(ap);
+  if( pStmt==0 ) return 1;
+  rc = sqlite3_step(pStmt);
+  if( rc==SQLITE_ROW ){
+    *piRes = sqlite3_column_int64(pStmt, 0);
+    rc = SQLITE_OK;
+  }else if( rc==SQLITE_DONE ){
+    rc = SQLITE_OK;
+  }else{
+    if( p->db ){
+      /* p->db is NULL if there was some prior error */
+      diskusedError(p, "SQL run-time error: %s\nOriginal SQL: %s",
+                    sqlite3_errmsg(p->db), sqlite3_sql(pStmt));
+    }
+    diskusedReset(p);
+  }
+  sqlite3_finalize(pStmt);
+  return rc;
+}
+
+/*
+** Add to the output a title line that contains the text determined
+** by the format string.  If the output is initially empty, begin
+** the title line with "/" so that it forms the beginning of a C-style
+** comment.  Otherwise begin with a new-line.  Always finish with a
+** newline.
+*/
+static void diskusedTitle(DiskUsed *p, const char *zFormat, ...){
+  char *zFirst;
+  char *zTitle;
+  size_t nTitle;
+  va_list ap;
+  va_start(ap, zFormat);
+  zTitle = sqlite3_vmprintf(zFormat, ap);
+  va_end(ap);
+  if( zTitle==0 ){
+    diskusedError(p, 0);
+    return;
+  }
+  zFirst = sqlite3_str_length(p->pOut)==0 ? "/" : "\n*";
+  nTitle = strlen(zTitle);
+  if( nTitle>=75 ){
+    sqlite3_str_appendf(p->pOut, "%s** %z\n\n", zFirst, zTitle);
+  }else{
+    int nExtra = 74 - (int)nTitle;
+    sqlite3_str_appendf(p->pOut, "%s** %z %.*c\n\n", zFirst, zTitle,
+                        nExtra, '*');
+  }
+}
+
+/*
+** Add an output line that begins with the zDesc text extended out to
+** 50 columns with "." characters, and followed by whatever text is
+** described by zFormat.
+*/
+static void diskusedLine(
+  DiskUsed *p,             /* DiskUsed context */
+  const char *zDesc,       /* Description */
+  const char *zFormat,     /* Argument to the description */
+  ...
+){
+  char *zTxt;
+  size_t nDesc;
+  va_list ap;
+  va_start(ap, zFormat);
+  zTxt = sqlite3_vmprintf(zFormat, ap);
+  va_end(ap);
+  if( zTxt==0 ){
+    diskusedError(p, 0);
+    return;
+  }
+  nDesc = zDesc ? strlen(zDesc) : 0;
+  if( nDesc>=50 ){
+    sqlite3_str_appendf(p->pOut, "%s %z", zDesc, zTxt);
+  }else{
+    int nExtra = 50 - (int)nDesc;
+    sqlite3_str_appendf(p->pOut, "%s%.*c %z", zDesc, nExtra, '.', zTxt);
+  }
+}
+
+/*
+** Write a percentage into the output.  The number written should show
+** two or three significant digits, with the decimal point being the fourth
+** character.  
+*/
+static void diskusedPercent(DiskUsed *p, double r){
+  char zNum[100];
+  char *zDP;
+  int nLeadingDigit;
+  int sz;
+  sqlite3_snprintf(sizeof(zNum)-5, zNum, r>=10.0 ? "%.3g" :"%.2g", r);
+  sz = (int)strlen(zNum);
+  zDP = strchr(zNum, '.');
+  if( zDP==0 ){
+    memcpy(zNum+sz,".0",3);
+    nLeadingDigit = sz;
+    sz += 2;
+  }else{
+    nLeadingDigit = (int)(zDP - zNum);
+  }
+  if( nLeadingDigit<3 ){
+    sqlite3_str_appendchar(p->pOut, 3-nLeadingDigit, ' ');
+  }
+  sqlite3_str_append(p->pOut, zNum, sz);
+  sqlite3_str_append(p->pOut, "%\n", 2);
+}
+
+/*
+** Create a subreport on a subset of tables and/or indexes.
+**
+** The title if the subreport is given by zTitle.  zWhere is
+** a boolean expression that can go in the WHERE clause to select
+** the relevant rows of the s.zSU table.
+*/
+static int diskusedSubreport(
+  DiskUsed *p,                  /* DiskUsed context */
+  char *zTitle,                 /* Title for this subreport */
+  char *zWhere,                 /* WHERE clause for this subreport */
+  sqlite3_int64 pgsz,           /* Database page size */
+  sqlite3_int64 nPage           /* Number of pages in entire database */
+){
+  sqlite3_stmt *pStmt;          /* Statement to query p->zSU */
+  sqlite3_int64 nentry;         /* Number of btree entires */
+  sqlite3_int64 payload;        /* Payload in bytes */
+  sqlite3_int64 ovfl_payload;   /* overflow payload in bytes */
+  sqlite3_int64 mx_payload;     /* largest individual payload */
+  sqlite3_int64 ovfl_cnt;       /* Number entries using overflow */
+  sqlite3_int64 leaf_pages;     /* Leaf pages */
+  sqlite3_int64 int_pages;      /* internal pages */
+  sqlite3_int64 ovfl_pages;     /* overflow pages */
+  sqlite3_int64 leaf_unused;    /* unused bytes on leaf pages */
+  sqlite3_int64 int_unused;     /* unused bytes on internal pages */
+  sqlite3_int64 ovfl_unused;    /* unused bytes on overflow pages */
+  sqlite3_int64 int_cell;       /* B-tree entries on internal pages */
+  sqlite3_int64 depth;          /* btree depth */
+  sqlite3_int64 cnt;            /* Number of s.zSU entries that match */
+  sqlite3_int64 storage;        /* Total bytes */
+  sqlite3_int64 total_pages;    /* Total page count */
+  sqlite3_int64 total_unused;   /* Total unused bytes */
+  sqlite3_int64 total_meta;     /* Total metadata */
+  int rc;
+
+  if( zTitle==0 || zWhere==0 ){
+    diskusedError(p, 0);
+    return SQLITE_NOMEM;
+  }
+  pStmt = diskusedPrepare(p,
+    "SELECT\n"
+    "  sum(if(is_without_rowid OR is_index,nentry,leaf_entries)),\n" /* 0 */
+    "  sum(payload),\n"            /* 1 */
+    "  sum(ovfl_payload),\n"       /* 2 */
+    "  max(mx_payload),\n"         /* 3 */
+    "  sum(ovfl_cnt),\n"           /* 4 */
+    "  sum(leaf_pages),\n"         /* 5 */
+    "  sum(int_pages),\n"          /* 6 */
+    "  sum(ovfl_pages),\n"         /* 7 */
+    "  sum(leaf_unused),\n"        /* 8 */
+    "  sum(int_unused),\n"         /* 9 */
+    "  sum(ovfl_unused),\n"        /* 10 */
+    "  max(depth),\n"              /* 11 */
+    "  count(*),\n"                /* 12 */
+    "  sum(int_entries)\n"         /* 13 */
+    " FROM temp.%s WHERE %s",
+    p->zSU, zWhere);
+  if( pStmt==0 ) return 1;
+  rc = sqlite3_step(pStmt);
+  if( rc==SQLITE_ROW ){
+    diskusedTitle(p, "%s", zTitle);
+
+    nentry = sqlite3_column_int64(pStmt, 0);
+    payload = sqlite3_column_int64(pStmt, 1);
+    ovfl_payload = sqlite3_column_int64(pStmt, 2);
+    mx_payload = sqlite3_column_int64(pStmt, 3);
+    ovfl_cnt = sqlite3_column_int64(pStmt, 4);
+    leaf_pages = sqlite3_column_int64(pStmt, 5);
+    int_pages = sqlite3_column_int64(pStmt, 6);
+    ovfl_pages = sqlite3_column_int64(pStmt, 7);
+    leaf_unused = sqlite3_column_int64(pStmt, 8);
+    int_unused = sqlite3_column_int64(pStmt, 9);
+    ovfl_unused = sqlite3_column_int64(pStmt, 10);
+    depth = sqlite3_column_int64(pStmt, 11);
+    cnt = sqlite3_column_int64(pStmt, 12);
+    int_cell = sqlite3_column_int64(pStmt, 13);
+    rc = SQLITE_DONE;
+
+    total_pages = leaf_pages + int_pages + ovfl_pages;
+    diskusedLine(p, "Percentage of total database", "%.3g%%\n",
+                 (total_pages*100.0)/(double)nPage);
+    diskusedLine(p, "Number of entries", "%lld\n", nentry);
+    storage = total_pages*pgsz;
+    diskusedLine(p, "Bytes of storage consumed", "%lld\n", storage);
+    diskusedLine(p, "Bytes of payload", "%-11lld ", payload);
+    diskusedPercent(p, payload*100.0/(double)storage);
+    if( ovfl_cnt>0 ){
+      diskusedLine(p, "Bytes of payload in overflow","%-11lld ",ovfl_payload);
+      diskusedPercent(p, ovfl_payload*100.0/(double)payload);
+    }
+    total_unused = leaf_unused + int_unused + ovfl_unused;
+    total_meta = storage - payload - total_unused;
+    diskusedLine(p, "Bytes of metadata","%-11lld ", total_meta);
+    diskusedPercent(p, total_meta*100.0/(double)storage);
+    if( cnt==1 ){
+      diskusedLine(p, "B-tree depth", "%lld\n", depth);
+      if( int_cell>1 ){
+        diskusedLine(p, "Average fanout", "%.1f\n",
+                     (double)(int_cell+int_pages)/(double)int_pages);
+      }
+    }
+    if( nentry>0 ){
+      diskusedLine(p, "Average payload per entry", "%.1f\n",
+                   (double)payload/(double)nentry);
+      diskusedLine(p, "Average unused bytes per entry", "%.1f\n",
+                   (double)total_unused/(double)nentry);
+      diskusedLine(p, "Average metadata per entry", "%.1f\n",
+                   (double)total_meta/(double)nentry);
+    }
+    diskusedLine(p, "Maximum single-entry payload", "%lld\n", mx_payload);
+    if( nentry>0 ){
+      diskusedLine(p, "Entries that use overflow", "%-11lld ", ovfl_cnt);
+      diskusedPercent(p, ovfl_cnt*100.0/(double)nentry);
+    }
+    if( int_pages>0 ){
+      diskusedLine(p, "Index pages used", "%lld\n", int_pages);
+    }
+    diskusedLine(p, "Primary pages used", "%lld\n", leaf_pages);
+    if( ovfl_cnt ){
+      diskusedLine(p, "Overflow pages used", "%lld\n", ovfl_pages);
+    }
+    diskusedLine(p, "Total pages used", "%lld\n", total_pages);
+    if( int_pages>0 ){
+      diskusedLine(p, "Unused bytes on index pages", "%lld\n", int_unused);
+    }
+    diskusedLine(p, "Unused bytes on primary pages", "%lld\n", leaf_unused);
+    if( ovfl_cnt ){
+      diskusedLine(p, "Unused bytes on overflow pages", "%lld\n", ovfl_unused);
+    }
+    diskusedLine(p, "Unused bytes on all pages", "%-11lld ", total_unused);
+    diskusedPercent(p, total_unused*100.0/(double)storage);
+  }
+  return diskusedStmtFinish(p, rc, pStmt);
+}
+
+/*
+** SQL Function:   diskused(SCHEMA)
+**
+** Analyze the database schema named in the argument.  Return text
+** containing the space utilization stats.
+*/
+static void diskusedFunc(
+  sqlite3_context *context,
+  int argc,
+  sqlite3_value **argv
+){
+  int rc;
+  sqlite3_stmt *pStmt;
+  int n;
+  sqlite3_int64 ii;
+  sqlite3_int64 pgsz;
+  sqlite3_int64 nPage;
+  sqlite3_int64 nPageInUse;
+  sqlite3_int64 nFreeList;
+  sqlite3_int64 nIndex;
+  sqlite3_int64 nWORowid;
+  DiskUsed s;
+  sqlite3_uint64 r[2];
+
+  (void)argc;
+  memset(&s, 0, sizeof(s));
+  s.db = sqlite3_context_db_handle(context);
+  s.context = context;
+  s.pOut = sqlite3_str_new(0);
+  if( sqlite3_str_errcode(s.pOut) ){
+    diskusedError(&s, 0);
+    return;
+  }
+  s.zSchema = (const char*)sqlite3_value_text(argv[0]);
+  if( s.zSchema==0 ){
+    s.zSchema = "main";
+  }else if( sqlite3_strlike("temp",s.zSchema,0)==0 ){
+    diskusedReset(&s);
+    sqlite3_result_text(context, "cannot analyze \"temp\"",-1,SQLITE_STATIC);
+    return;
+  }
+  ii = 0;
+  rc = diskusedSqlInt(&s,&ii,"SELECT 1 FROM pragma_database_list"
+                             " WHERE name=%Q COLLATE nocase",s.zSchema);
+  if( rc || ii==0 ){
+    diskusedReset(&s);
+    sqlite3_result_text(context,"no such database",-1,SQLITE_STATIC);
+    return;
+  }
+  sqlite3_randomness(sizeof(r), &r);
+  s.zSU = sqlite3_mprintf("diskused%016llx%016llx", r[0], r[1]);
+  if( s.zSU==0 ){ diskusedError(&s, 0); return; }
+
+  /* The s.zSU table contains the data used for the analysis.
+  ** The table name contains 128-bits of randomness to avoid
+  ** collisions with preexisting tables in temp.
+  */
+  rc = diskusedSql(&s,
+    "CREATE TABLE temp.%s(\n"
+    "   name text,                -- A table or index\n"
+    "   tblname text,             -- Table that owns name\n"
+    "   is_index boolean,         -- TRUE if it is an index\n"
+    "   is_without_rowid boolean, -- TRUE if WITHOUT ROWID table\n"
+    "   nentry int,               -- Number of entries in the BTree\n"
+    "   leaf_entries int,         -- Number of leaf entries\n"
+    "   depth int,                -- Depth of the b-tree\n"
+    "   payload int,              -- Total data stored in this table/index\n"
+    "   ovfl_payload int,         -- Total data stored on overflow pages\n"
+    "   ovfl_cnt int,             -- Number of entries that use overflow\n"
+    "   mx_payload int,           -- Maximum payload size\n"
+    "   int_pages int,            -- Interior pages used\n"
+    "   leaf_pages int,           -- Leaf pages used\n"
+    "   ovfl_pages int,           -- Overflow pages used\n"
+    "   int_unused int,           -- Unused bytes on interior pages\n"
+    "   leaf_unused int,          -- Unused bytes on primary pages\n"
+    "   ovfl_unused int,          -- Unused bytes on overflow pages\n"
+    "   int_entries int           -- Btree cells on internal pages\n"
+    ");",
+    s.zSU
+  );
+  if( rc ) return;
+
+  /* Populate the s.zSU table
+  */
+  rc = diskusedSql(&s,
+    "WITH\n"
+    "  allidx(idxname) AS (\n"
+    "    SELECT name FROM \"%w\".sqlite_schema WHERE type='index'\n"
+    "  ),\n"
+    "  allobj(allname,tblname,isidx,isworowid) AS (\n"
+    "    SELECT 'sqlite_schema',\n"
+    "           'sqlite_schema',\n"
+    "           0,\n"
+    "           0\n"
+    "    UNION ALL\n"
+    "    SELECT name,\n"
+    "           tbl_name,\n"
+    "           type='index',\n"
+    "           EXISTS(SELECT 1\n"
+    "                    FROM pragma_index_list(sqlite_schema.name,%Q)\n"
+    "                   WHERE pragma_index_list.origin='pk'\n"
+    "                     AND pragma_index_list.name NOT IN allidx)\n"
+    "      FROM \"%w\".sqlite_schema\n"
+    "  )\n"
+    "INSERT INTO temp.%s\n"
+    "  SELECT\n"
+    "    allname,\n"
+    "    tblname,\n"
+    "    isidx,\n"
+    "    isworowid,\n"
+    "    sum(ncell),\n"
+    "    sum((pagetype='leaf')*ncell),\n"
+    "    max((length(if(path GLOB '*+*','',path))+3)/4),\n"
+    "    sum(payload),\n"
+    "    sum((pagetype='overflow')*payload),\n"
+    "    sum(path GLOB '*+000000'),\n"
+    "    max(mx_payload),\n"
+    "    sum(pagetype='internal'),\n"
+    "    sum(pagetype='leaf'),\n"
+    "    sum(pagetype='overflow'),\n"
+    "    sum((pagetype='internal')*unused),\n"
+    "    sum((pagetype='leaf')*unused),\n"
+    "    sum((pagetype='overflow')*unused),\n"
+    "    sum(if(pagetype='internal',ncell))\n"
+    "  FROM allobj CROSS JOIN dbstat(%Q) \n"
+    "  WHERE dbstat.name=allobj.allname\n"
+    "  GROUP BY allname;\n",
+    s.zSchema,   /* %w.sqlite_schema -- in allidx */
+    s.zSchema,   /* pragma_index_list(...,%Q) */
+    s.zSchema,   /* %w.sqlite_schema */
+    s.zSU,       /* INTO temp.%s */
+    s.zSchema    /* JOIN dbstat(%Q) */
+  );
+  if( rc ) return;
+
+  nPage = 0;
+  rc = diskusedSqlInt(&s, &nPage, "PRAGMA \"%w\".page_count", s.zSchema);
+  if( rc ) return;
+  if( nPage<=0 ){
+    /* Very brief reply for an empty database */
+    diskusedReset(&s);
+    sqlite3_result_text(context, "empty database", -1, SQLITE_STATIC);
+    return;
+  }
+
+  /* Begin generating the report */
+  diskusedTitle(&s, "Database storage utilization report");
+  pgsz = 0;
+  rc = diskusedSqlInt(&s, &pgsz, "PRAGMA \"%w\".page_size", s.zSchema);
+  if( rc ) return;
+  diskusedLine(&s, "Page size in bytes","%lld\n",pgsz);
+  diskusedLine(&s, "Pages in the database", "%lld\n", nPage);
+
+  nPageInUse = 0;
+  rc = diskusedSqlInt(&s, &nPageInUse, 
+       "SELECT sum(leaf_pages+int_pages+ovfl_pages) FROM temp.%s", s.zSU);
+  if( rc ) return;
+  diskusedLine(&s, "Pages that store data", "%-11lld ", nPageInUse);
+  diskusedPercent(&s, (nPageInUse*100.0)/(double)nPage);
+
+  nFreeList = 0;
+  rc = diskusedSqlInt(&s, &nFreeList, "PRAGMA \"%w\".freelist_count",s.zSchema);
+  if( rc ) return;
+  diskusedLine(&s, "Pages on the freelist", "%-11lld ", nFreeList);
+  diskusedPercent(&s, (nFreeList*100.0)/(double)nPage);
+
+  ii = 0;
+  rc = diskusedSqlInt(&s, &ii, "PRAGMA \"%w\".auto_vacuum", s.zSchema);
+  if( rc ) return;
+  if( ii==0 || nPage<=1 ){
+    ii = 0;
+  }else{
+    double rPtrsPerPage = pgsz/5;
+    double rAvPage = (nPage-1.0)/(rPtrsPerPage+1.0);
+    ii = (sqlite3_int64)ceil(rAvPage);
+  }
+  diskusedLine(&s, "Pages of auto-vacuum overhead", "%-11lld ", ii);
+  diskusedPercent(&s, (ii*100.0)/(double)nPage);
+
+  ii = 0;
+  rc = diskusedSqlInt(&s, &ii, 
+       "SELECT count(*)+1 FROM \"%w\".sqlite_schema WHERE type='table'",
+       s.zSchema);
+  if( rc ) return;
+  diskusedLine(&s, "Number of tables", "%lld\n", ii);
+  nWORowid = 0;
+  rc = diskusedSqlInt(&s, &nWORowid,
+       "SELECT count(*) FROM \"%w\".pragma_table_list WHERE wr",
+       s.zSchema);
+  if( rc ) return;
+  if( nWORowid>0 ){
+    diskusedLine(&s, "Number of WITHOUT ROWID tables", "%lld\n", nWORowid);
+    diskusedLine(&s, "Number of rowid tables", "%lld\n", ii - nWORowid);
+  }
+  nIndex = 0;
+  rc = diskusedSqlInt(&s, &nIndex, 
+       "SELECT count(*) FROM \"%w\".sqlite_schema WHERE type='index'",
+       s.zSchema);
+  if( rc ) return;
+  diskusedLine(&s, "Number of indexes", "%lld\n", nIndex);
+  ii = 0;
+  rc = diskusedSqlInt(&s, &ii, 
+       "SELECT count(*) FROM \"%w\".sqlite_schema"
+       " WHERE name GLOB 'sqlite_autoindex_*' AND type='index'",
+       s.zSchema);
+  if( rc ) return;
+  diskusedLine(&s, "Number of defined indexes", "%lld\n", nIndex - ii);
+  diskusedLine(&s, "Number of implied indexes", "%lld\n", ii);
+  diskusedLine(&s, "Size of the database in bytes", "%lld\n", pgsz*nPage);
+  ii = 0;
+  rc = diskusedSqlInt(&s, &ii, 
+       "SELECT sum(payload) FROM temp.%s"
+       " WHERE NOT is_index AND name NOT LIKE 'sqlite_schema'",
+       s.zSU);
+  if( rc ) return;
+  diskusedLine(&s, "Bytes of payload", "%-11lld ", ii);
+  diskusedPercent(&s, ii*100.0/(double)(pgsz*nPage));
+
+  diskusedTitle(&s, "Page counts for all tables with their indexes");
+  pStmt = diskusedPrepare(&s,
+    "SELECT upper(tblname),\n"
+    "       sum(int_pages+leaf_pages+ovfl_pages)\n"
+    "  FROM temp.%s\n"
+    " WHERE tblname IS NOT NULL\n"
+    " GROUP BY 1\n"
+    " ORDER BY 2 DESC, 1;",
+    s.zSU);
+  if( pStmt==0 ) return;
+  while( (rc = sqlite3_step(pStmt))==SQLITE_ROW ){
+    sqlite3_int64 nn = sqlite3_column_int64(pStmt,1);
+    diskusedLine(&s, (const char*)sqlite3_column_text(pStmt,0), "%-11lld ", nn);
+    diskusedPercent(&s, (nn*100.0)/(double)nPage);
+  }
+  if( diskusedStmtFinish(&s, rc, pStmt) ) return;
+
+  diskusedTitle(&s, "Page counts for all tables and indexes separately");
+  pStmt = diskusedPrepare(&s,
+    "SELECT upper(name),\n"
+    "       sum(int_pages+leaf_pages+ovfl_pages)\n"
+    "  FROM temp.%s\n"
+    " WHERE name IS NOT NULL\n"
+    " GROUP BY 1\n"
+    " ORDER BY 2 DESC, 1;",
+    s.zSU);
+  if( pStmt==0 ) return;
+  while( (rc = sqlite3_step(pStmt))==SQLITE_ROW ){
+    sqlite3_int64 nn = sqlite3_column_int64(pStmt,1);
+    diskusedLine(&s, (const char*)sqlite3_column_text(pStmt,0), "%-11lld ", nn);
+    diskusedPercent(&s, (nn*100.0)/(double)nPage);
+  }
+  if( diskusedStmtFinish(&s, rc, pStmt) ) return;
+
+  rc = diskusedSubreport(&s, "All tables and indexes", "1", pgsz, nPage);
+  if( rc ) return;
+  rc = diskusedSubreport(&s, "All tables", "NOT is_index", pgsz, nPage);
+  if( rc ) return;
+  if( nWORowid>0 ){
+    rc = diskusedSubreport(&s, "All WITHOUT ROWID tables", "is_without_rowid",
+                           pgsz, nPage);
+    if( rc ) return;
+    rc = diskusedSubreport(&s, "All rowid tables",
+                           "NOT is_without_rowid AND NOT is_index",
+                           pgsz, nPage);
+    if( rc ) return;
+  }
+  rc = diskusedSubreport(&s, "All indexes", "is_index", pgsz, nPage);
+  if( rc ) return;
+
+  pStmt = diskusedPrepare(&s,
+    "SELECT upper(tblname), tblname, sum(is_index) FROM temp.%s"
+    " GROUP BY 1 ORDER BY 1",
+    s.zSU);
+  if( pStmt==0 ) return;
+  while( (rc = sqlite3_step(pStmt))==SQLITE_ROW ){
+    const char *zUpper = (const char*)sqlite3_column_text(pStmt, 0);
+    const char *zName = (const char*)sqlite3_column_text(pStmt, 1);
+    int nSubIndex = sqlite3_column_int(pStmt, 2);
+    if( nSubIndex==0 ){
+      char *zTitle = sqlite3_mprintf("Table %s", zUpper);
+      char *zWhere = sqlite3_mprintf("name=%Q", zName);
+      rc = diskusedSubreport(&s, zTitle, zWhere, pgsz, nPage);
+      sqlite3_free(zTitle);
+      sqlite3_free(zWhere);
+      if( rc ) break;
+    }else{
+      sqlite3_stmt *pS2;
+      char *zTitle = sqlite3_mprintf("Table %s and all its indexes", zUpper);
+      char *zWhere = sqlite3_mprintf("tblname=%Q", zName);
+      rc = diskusedSubreport(&s, zTitle, zWhere, pgsz, nPage);
+      sqlite3_free(zTitle);
+      sqlite3_free(zWhere);
+      if( rc ) break;
+      zTitle = sqlite3_mprintf("Table %s w/o any indexes", zUpper);
+      zWhere = sqlite3_mprintf("name=%Q", zName);
+      rc = diskusedSubreport(&s, zTitle, zWhere, pgsz, nPage);
+      sqlite3_free(zTitle);
+      sqlite3_free(zWhere);
+      if( rc ) break;
+      if( nSubIndex>1 ){
+        zTitle = sqlite3_mprintf("All indexes of table %s", zUpper);
+        zWhere = sqlite3_mprintf("tblname=%Q AND is_index", zName);
+        rc = diskusedSubreport(&s, zTitle, zWhere, pgsz, nPage);
+        sqlite3_free(zTitle);
+        sqlite3_free(zWhere);
+        if( rc ) break;
+      }
+      pS2 = diskusedPrepare(&s,
+              "SELECT name, upper(name) FROM temp.%s"
+              " WHERE is_index AND tblname=%Q",
+              s.zSU, zName);
+      if( pS2==0 ){
+        rc = SQLITE_NOMEM;
+        break;
+      }
+      while( (rc = sqlite3_step(pS2))==SQLITE_ROW ){
+        const char *zU = (const char*)sqlite3_column_text(pS2, 1);
+        const char *zN = (const char*)sqlite3_column_text(pS2, 0);
+        zTitle = sqlite3_mprintf("Index %s", zU);
+        zWhere = sqlite3_mprintf("name=%Q", zN);
+        rc = diskusedSubreport(&s, zTitle, zWhere, pgsz, nPage);
+        sqlite3_free(zTitle);
+        sqlite3_free(zWhere);
+        if( rc ) break;
+      }
+      rc = diskusedStmtFinish(&s, rc, pS2);
+      if( rc ) break;
+    }
+  }
+  if( diskusedStmtFinish(&s, rc, pStmt) ) return;
+
+  /* Append SQL statements that will recreate the raw data used for
+  ** the analysis.
+  */
+  diskusedTitle(&s, "Raw data used to generate this report");
+  sqlite3_str_appendf(s.pOut,
+    "The following SQL will create a table named \"space_used\" which\n"
+    "contains most of the information used to generate the report above.\n"
+    "*/\n"
+  );
+  sqlite3_str_appendf(s.pOut,
+    "BEGIN;\n"
+    "CREATE TABLE space_used(\n"
+    "   name text,                -- A table or index\n"                /* 0 */
+    "   tblname text,             -- Table that owns name\n"            /* 1 */
+    "   is_index boolean,         -- TRUE if it is an index\n"          /* 2 */
+    "   is_without_rowid boolean, -- TRUE if WITHOUT ROWID table\n"     /* 3 */
+    "   nentry int,               -- Number of entries in the BTree\n"  /* 4 */
+    "   leaf_entries int,         -- Number of leaf entries\n"          /* 5 */
+    "   depth int,                -- Depth of the b-tree\n"             /* 6 */
+    "   payload int,              -- Total data in this table/index\n"  /* 7 */
+    "   ovfl_payload int,         -- Total data on overflow pages\n"    /* 8 */
+    "   ovfl_cnt int,             -- Entries that use overflow\n"       /* 9 */
+    "   mx_payload int,           -- Maximum payload size\n"            /* 10 */
+    "   int_pages int,            -- Interior pages used\n"             /* 11 */
+    "   leaf_pages int,           -- Leaf pages used\n"                 /* 12 */
+    "   ovfl_pages int,           -- Overflow pages used\n"             /* 13 */
+    "   int_unused int,           -- Unused bytes on interior pages\n"  /* 14 */
+    "   leaf_unused int,          -- Unused bytes on primary pages\n"   /* 15 */
+    "   ovfl_unused int,          -- Unused bytes on overflow pages\n"  /* 16 */
+    "   int_entries int           -- B-tree entries on internal pages\n"/* 17 */
+    ");\n"
+    "INSERT INTO space_used VALUES\n"
+  );
+  pStmt = diskusedPrepare(&s,
+     "SELECT quote(name), quote(tblname),\n"                        /* 0..1 */
+     "       is_index, is_without_rowid, nentry, leaf_entries,\n"   /* 2..5 */
+     "       depth, payload, ovfl_payload, ovfl_cnt, mx_payload,\n" /* 6..10 */
+     "       int_pages, leaf_pages, ovfl_pages, int_unused,\n"      /* 11..14 */
+     "       leaf_unused, ovfl_unused, int_entries\n"               /* 15..17 */
+     "  FROM temp.%s;",
+     s.zSU);
+  if( pStmt==0 ) return;
+  n = 0;
+  while( (rc = sqlite3_step(pStmt))==SQLITE_ROW ){
+    if( n++ ) sqlite3_str_appendf(s.pOut,",\n");
+    sqlite3_str_appendf(s.pOut,
+      " (%s,%s,%lld,%lld,%lld,%lld,%lld,%lld,%lld,"
+      "%lld,%lld,%lld,%lld,%lld,%lld,%lld,%lld,%lld)",
+      sqlite3_column_text(pStmt, 0),
+      sqlite3_column_text(pStmt, 1),
+      sqlite3_column_int64(pStmt, 2),
+      sqlite3_column_int64(pStmt, 3),
+      sqlite3_column_int64(pStmt, 4),
+      sqlite3_column_int64(pStmt, 5),
+      sqlite3_column_int64(pStmt, 6),
+      sqlite3_column_int64(pStmt, 7),
+      sqlite3_column_int64(pStmt, 8),
+      sqlite3_column_int64(pStmt, 9),
+      sqlite3_column_int64(pStmt, 10),
+      sqlite3_column_int64(pStmt, 11),
+      sqlite3_column_int64(pStmt, 12),
+      sqlite3_column_int64(pStmt, 13),
+      sqlite3_column_int64(pStmt, 14),
+      sqlite3_column_int64(pStmt, 15),
+      sqlite3_column_int64(pStmt, 16),
+      sqlite3_column_int64(pStmt, 17));
+  }
+  if( rc!=SQLITE_DONE ){
+    diskusedError(&s, "SQL run-time error: %s\nSQL: %s",
+                  sqlite3_errmsg(s.db), sqlite3_sql(pStmt));
+    sqlite3_finalize(pStmt);
+    return;
+  }
+  sqlite3_str_appendf(s.pOut,";\nCOMMIT;");
+  sqlite3_finalize(pStmt);
+
+  if( sqlite3_str_length(s.pOut) ){
+    sqlite3_result_text(context, sqlite3_str_finish(s.pOut), -1,
+                        sqlite3_free);
+    s.pOut = 0;
+  }
+  diskusedReset(&s);
+}
+
+
+#ifdef _WIN32
+__declspec(dllexport)
+#endif
+int sqlite3_diskused_init(
+  sqlite3 *db, 
+  char **pzErrMsg, 
+  const sqlite3_api_routines *pApi
+){
+  int rc = SQLITE_OK;
+  SQLITE_EXTENSION_INIT2(pApi);
+  (void)pzErrMsg;  /* Unused parameter */
+  rc = sqlite3_create_function(db, "diskused", 1,
+                   SQLITE_UTF8|SQLITE_INNOCUOUS,
+                   0, diskusedFunc, 0, 0);
+  return rc;
+}
@@ -92,7 +92,7 @@ static int explainConnect(
  rc = sqlite3_declare_vtab(db,
     "CREATE TABLE x(addr,opcode,p1,p2,p3,p4,p5,comment,sql HIDDEN)");
  if( rc==SQLITE_OK ){
-    pNew = sqlite3_malloc( sizeof(*pNew) );
+    pNew = sqlite3_malloc64( sizeof(*pNew) );
    *ppVtab = (sqlite3_vtab*)pNew;
    if( pNew==0 ) return SQLITE_NOMEM;
    memset(pNew, 0, sizeof(*pNew));
@@ -114,7 +114,7 @@ static int explainDisconnect(sqlite3_vtab *pVtab){
 */
 static int explainOpen(sqlite3_vtab *p, sqlite3_vtab_cursor **ppCursor){
  explain_cursor *pCur;
-  pCur = sqlite3_malloc( sizeof(*pCur) );
+  pCur = sqlite3_malloc64( sizeof(*pCur) );
  if( pCur==0 ) return SQLITE_NOMEM;
  memset(pCur, 0, sizeof(*pCur));
  pCur->db = ((explain_vtab*)p)->db;
@@ -73,11 +73,6 @@
 **   $path is a relative path, then $path is interpreted relative to $dir. 
 **   And the paths returned in the "name" column of the table are also 
 **   relative to directory $dir.
-**
-** Notes on building this extension for Windows:
-**   Unless linked statically with the SQLite library, a preprocessor
-**   symbol, FILEIO_WIN32_DLL, must be #define'd to create a stand-alone
-**   DLL form of this extension for WIN32. See its use below for details.
 */
 #include "sqlite3ext.h"
 SQLITE_EXTENSION_INIT1
@@ -102,8 +97,6 @@ SQLITE_EXTENSION_INIT1
 #  define STRUCT_STAT struct _stat
 #  define chmod(path,mode) fileio_chmod(path,mode)
 #  define mkdir(path,mode) fileio_mkdir(path)
-   extern LPWSTR sqlite3_win32_utf8_to_unicode(const char*);
-   extern char *sqlite3_win32_unicode_to_utf8(LPCWSTR);
 #endif
 #include <time.h>
 #include <errno.h>
@@ -130,13 +123,69 @@ SQLITE_EXTENSION_INIT1
 #define FSDIR_COLUMN_PATH     5     /* Path to top of search */
 #define FSDIR_COLUMN_DIR      6     /* Path is relative to this directory */

+#ifdef _WIN32
+/*
+** Convert a UTF-8 string to Microsoft Unicode.
+**
+** Space to hold the returned string is obtained from sqlite3_malloc().
+*/
+static wchar_t *winUtf8To16(const char *zText){
+  int nChar;
+  wchar_t *zWideText;
+
+  nChar = MultiByteToWideChar(CP_UTF8, 0, zText, -1, NULL, 0);
+  if( nChar==0 ){
+    return 0;
+  }
+  zWideText = sqlite3_malloc64(nChar*sizeof(WCHAR) );
+  if( zWideText==0 ){
+    return 0;
+  }
+  nChar = MultiByteToWideChar(CP_UTF8, 0, zText, -1, zWideText,
+                                nChar);
+  if( nChar==0 ){
+    sqlite3_free(zWideText);
+    zWideText = 0;
+  }
+  return zWideText;
+}
+#endif /* _WIN32 */
+
+#ifdef _WIN32
+/*
+** Convert a Microsoft Unicode string to UTF-8.
+**
+** Space to hold the returned string is obtained from sqlite3_malloc().
+*/
+static char *winUtf16To8(wchar_t *zWideText){
+  int nByte;
+  char *zText;
+
+  nByte = WideCharToMultiByte(CP_UTF8, 0, zWideText, -1, 0, 0, 0, 0);
+  if( nByte == 0 ){
+    return 0;
+  }
+  zText = sqlite3_malloc64( nByte );
+  if( zText==0 ){
+    return 0;
+  }
+  nByte = WideCharToMultiByte(CP_UTF8, 0, zWideText, -1, zText, nByte,
+                                0, 0);
+  if( nByte == 0 ){
+    sqlite3_free(zText);
+    zText = 0;
+  }
+  return zText;
+}
+#endif /* _WIN32 */
+
 /*
 ** UTF8 chmod() function for Windows
 */
 #if defined(_WIN32) || defined(WIN32)
 static int fileio_chmod(const char *zPath, int pmode){
  int rc;
-  wchar_t *b1 = sqlite3_win32_utf8_to_unicode(zPath);
+  wchar_t *b1 = winUtf8To16(zPath);
  if( b1==0 ) return -1;
  rc = _wchmod(b1, pmode);
  sqlite3_free(b1);
@@ -150,7 +199,7 @@ static int fileio_chmod(const char *zPath, int pmode){
 #if defined(_WIN32) || defined(WIN32)
 static int fileio_mkdir(const char *zPath){
  int rc;
-  wchar_t *b1 = sqlite3_win32_utf8_to_unicode(zPath);
+  wchar_t *b1 = winUtf8To16(zPath);
  if( b1==0 ) return -1;
  rc = _wmkdir(b1);
  sqlite3_free(b1);
@@ -277,7 +326,7 @@ static int fileStat(
 ){
 #if defined(_WIN32)
  int rc;
-  wchar_t *b1 = sqlite3_win32_utf8_to_unicode(zPath);
+  wchar_t *b1 = winUtf8To16(zPath);
  if( b1==0 ) return 1;
  rc = _wstat(b1, pStatBuf);
  if( rc==0 ){
@@ -430,14 +479,13 @@ static int writeFile(
    LONGLONG intervals;
    HANDLE hFile;
    LPWSTR zUnicodeName;
-    extern LPWSTR sqlite3_win32_utf8_to_unicode(const char*);

    GetSystemTime(&currentTime);
    SystemTimeToFileTime(&currentTime, &lastAccess);
    intervals = (mtime*10000000) + 116444736000000000;
    lastWrite.dwLowDateTime = (DWORD)intervals;
    lastWrite.dwHighDateTime = intervals >> 32;
-    zUnicodeName = sqlite3_win32_utf8_to_unicode(zFile);
+    zUnicodeName = winUtf8To16(zFile);
    if( zUnicodeName==0 ){
      return 1;
    }
@@ -618,7 +666,7 @@ static int fsdirConnect(
  (void)pzErr;
  rc = sqlite3_declare_vtab(db, "CREATE TABLE x" FSDIR_SCHEMA);
  if( rc==SQLITE_OK ){
-    pNew = (fsdir_tab*)sqlite3_malloc( sizeof(*pNew) );
+    pNew = (fsdir_tab*)sqlite3_malloc64( sizeof(*pNew) );
    if( pNew==0 ) return SQLITE_NOMEM;
    memset(pNew, 0, sizeof(*pNew));
    sqlite3_vtab_config(db, SQLITE_VTAB_DIRECTONLY);
@@ -641,7 +689,7 @@ static int fsdirDisconnect(sqlite3_vtab *pVtab){
 static int fsdirOpen(sqlite3_vtab *p, sqlite3_vtab_cursor **ppCursor){
  fsdir_cursor *pCur;
  (void)p;
-  pCur = sqlite3_malloc( sizeof(*pCur) );
+  pCur = sqlite3_malloc64( sizeof(*pCur) );
  if( pCur==0 ) return SQLITE_NOMEM;
  memset(pCur, 0, sizeof(*pCur));
  pCur->iLvl = -1;
@@ -1095,12 +1143,12 @@ static char *portable_realpath(const char *zPath){

  if( zPath==0 ) return 0;

-  zPath16 = sqlite3_win32_utf8_to_unicode(zPath);
+  zPath16 = winUtf8To16(zPath);
  if( zPath16==0 ) return 0;
  z = _wfullpath(NULL, zPath16, 0);
  sqlite3_free(zPath16);
  if( z ){
-    zOut = sqlite3_win32_unicode_to_utf8(z);
+    zOut = winUtf16To8(z);
    free(z);
  }
  return zOut;
@@ -1117,6 +1165,9 @@ static char *portable_realpath(const char *zPath){
 ** The file or directory X is not required to exist.  The answer is formed
 ** by calling system realpath() on the prefix of X that does exist and
 ** appending the tail of X that does not (yet) exist.
+**
+** FIXME:  This routine sometimes returns NULL rather than raising
+** an SQLITE_NOMEM error if an OOM is encountered.
 */
 static void realpathFunc(
  sqlite3_context *context,
@@ -1139,6 +1190,7 @@ static void realpathFunc(
  if( zPath==0 ) return;
  if( zPath[0]==0 ) zPath = ".";
  zCopy = sqlite3_mprintf("%s",zPath);
+  if( zCopy==0 ) return;
  len = strlen(zCopy);
  while( len>1 && (zCopy[len-1]=='/' || (isWin && zCopy[len-1]=='\\')) ){
    len--;
@@ -29,6 +29,17 @@
 ** RBU does not use this extension directly.  Rather, this extension is
 ** provided as a convenience to developers who want to analyze RBU files
 ** that contain deltas.
+**
+** Typical build commands assuming
+** 
+**    DIR=ext/misc
+*     NAME=fossildelta
+**
+** First run "make sqlite3ext.h" then:
+**
+**   linux: gcc -shared -I. -fPIC -o $NAME.so $DIR/$NAME.c
+**   OS-X:  gcc -dynamiclib -fPIC -I. -o $NAME.dylib $DIR/$NAME.c
+**   Win11: cl -I. $DIR/$NAME.c -link -dll -out:$NAME.dll
 */
 #include <string.h>
 #include <assert.h>
@@ -43,6 +54,7 @@ SQLITE_EXTENSION_INIT1
 */
 typedef unsigned int u32;
 typedef sqlite3_uint64 u64;
+typedef sqlite3_int64 i64;

 /*
 ** Must be a 16-bit value
@@ -167,16 +179,26 @@ static unsigned int deltaGetInt(const char **pz, int *pLen){
    25, 26, 27, 28, 29, 30, 31, 32,   33, 34, 35, -1, -1, -1, -1, 36,
    -1, 37, 38, 39, 40, 41, 42, 43,   44, 45, 46, 47, 48, 49, 50, 51,
    52, 53, 54, 55, 56, 57, 58, 59,   60, 61, 62, -1, -1, -1, 63, -1,
+
+    -1, -1, -1, -1, -1, -1, -1, -1,   -1, -1, -1, -1, -1, -1, -1, -1,
+    -1, -1, -1, -1, -1, -1, -1, -1,   -1, -1, -1, -1, -1, -1, -1, -1,
+    -1, -1, -1, -1, -1, -1, -1, -1,   -1, -1, -1, -1, -1, -1, -1, -1,
+    -1, -1, -1, -1, -1, -1, -1, -1,   -1, -1, -1, -1, -1, -1, -1, -1,
+    -1, -1, -1, -1, -1, -1, -1, -1,   -1, -1, -1, -1, -1, -1, -1, -1,
+    -1, -1, -1, -1, -1, -1, -1, -1,   -1, -1, -1, -1, -1, -1, -1, -1,
+    -1, -1, -1, -1, -1, -1, -1, -1,   -1, -1, -1, -1, -1, -1, -1, -1,
+    -1, -1, -1, -1, -1, -1, -1, -1,   -1, -1, -1, -1, -1, -1, -1, -1,
  };
  unsigned int v = 0;
  int c;
  unsigned char *z = (unsigned char*)*pz;
-  unsigned char *zStart = z;
-  while( (c = zValue[0x7f&*(z++)])>=0 ){
-     v = (v<<6) + c;
+  unsigned char *zEnd = z + (*pLen);
+  while( z<zEnd && (c = zValue[*z])>=0 ){
+    v = (v<<6) + c;
+    z++;
  }
-  z--;
-  *pLen -= z - zStart;
+
+  *pLen -= (int)(z - (unsigned char*)*pz);
  *pz = (char*)z;
  return v;
 }
@@ -266,7 +288,8 @@ static unsigned int checksum(const char *zIn, size_t N){
 ** The delta string will be NUL-terminated, but it might also contain
 ** embedded NUL characters if either the zSrc or zOut files are
 ** binary.  This function returns the length of the delta string
-** in bytes, excluding the final NUL terminator character.
+** in bytes, excluding the final NUL terminator character.  It returns
+** 0 if an OOM occurs.
 **
 ** Output Format:
 **
@@ -358,6 +381,7 @@ static int delta_create(
  */
  nHash = lenSrc/NHASH;
  collide = sqlite3_malloc64( (sqlite3_int64)nHash*2*sizeof(int) );
+  if( collide==0 ) return 0;
  memset(collide, -1, nHash*2*sizeof(int));
  landmark = &collide[nHash];
  for(i=0; i<lenSrc-NHASH; i+=NHASH){
@@ -508,7 +532,7 @@ static int delta_create(
 static int delta_output_size(const char *zDelta, int lenDelta){
  int size;
  size = deltaGetInt(&zDelta, &lenDelta);
-  if( *zDelta!='\n' ){
+  if( lenDelta<=0 || *zDelta!='\n' ){
    /* ERROR: size integer not terminated by "\n" */
    return -1;
  }
@@ -550,7 +574,7 @@ static int delta_apply(
 #endif

  limit = deltaGetInt(&zDelta, &lenDelta);
-  if( *zDelta!='\n' ){
+  if( lenDelta<=0 || *zDelta!='\n' ){
    /* ERROR: size integer not terminated by "\n" */
    return -1;
  }
@@ -558,6 +582,7 @@ static int delta_apply(
  while( *zDelta && lenDelta>0 ){
    unsigned int cnt, ofst;
    cnt = deltaGetInt(&zDelta, &lenDelta);
+    if( lenDelta<=0 ) return -1;
    switch( zDelta[0] ){
      case '@': {
        zDelta++; lenDelta--;
@@ -769,11 +794,11 @@ struct deltaparsevtab_vtab {
 struct deltaparsevtab_cursor {
  sqlite3_vtab_cursor base;  /* Base class - must be first */
  char *aDelta;              /* The delta being parsed */
-  int nDelta;                /* Number of bytes in the delta */
-  int iCursor;               /* Current cursor location */
+  i64 iCursor;               /* Current cursor location */
+  i64 iNext;                 /* Next cursor value */
+  i64 nDelta;                /* Number of bytes in the delta */
  int eOp;                   /* Name of current operator */
  unsigned int a1, a2;       /* Arguments to current operator */
-  int iNext;                 /* Next cursor value */
 };

 /* Operator names:
@@ -843,7 +868,7 @@ static int deltaparsevtabDisconnect(sqlite3_vtab *pVtab){
 */
 static int deltaparsevtabOpen(sqlite3_vtab *p, sqlite3_vtab_cursor **ppCursor){
  deltaparsevtab_cursor *pCur;
-  pCur = sqlite3_malloc( sizeof(*pCur) );
+  pCur = sqlite3_malloc64( sizeof(*pCur) );
  if( pCur==0 ) return SQLITE_NOMEM;
  memset(pCur, 0, sizeof(*pCur));
  *ppCursor = &pCur->base;
@@ -877,7 +902,7 @@ static int deltaparsevtabNext(sqlite3_vtab_cursor *cur){
  }
  z = pCur->aDelta + pCur->iCursor;
  pCur->a1 = deltaGetInt(&z, &i);
-  switch( z[0] ){
+  switch( i>0 ? z[0] : 0 ){
    case '@': {
      z++;
      if( pCur->iNext>=pCur->nDelta ){
@@ -887,14 +912,14 @@ static int deltaparsevtabNext(sqlite3_vtab_cursor *cur){
      }
      pCur->a2 = deltaGetInt(&z, &i);
      pCur->eOp = DELTAPARSE_OP_COPY;
-      pCur->iNext = (int)(&z[1] - pCur->aDelta);
+      pCur->iNext = (i64)(&z[1] - pCur->aDelta);
      break;
    }
    case ':': {
      z++;
      pCur->a2 = (unsigned int)(z - pCur->aDelta);
      pCur->eOp = DELTAPARSE_OP_INSERT;
-      pCur->iNext = (int)(&z[pCur->a1] - pCur->aDelta);
+      pCur->iNext = (i64)(&z[pCur->a1] - pCur->aDelta);
      break;
    }
    case ';': {
@@ -938,7 +963,7 @@ static int deltaparsevtabColumn(
      if( pCur->eOp==DELTAPARSE_OP_COPY ){
        sqlite3_result_int(ctx, pCur->a2);
      }else if( pCur->eOp==DELTAPARSE_OP_INSERT ){
-        if( pCur->a2 + pCur->a1 > pCur->nDelta ){
+        if( (i64)pCur->a2 + (i64)pCur->a1 > pCur->nDelta ){
          sqlite3_result_zeroblob(ctx, pCur->a1);
        }else{
          sqlite3_result_blob(ctx, pCur->aDelta+pCur->a2, pCur->a1,
@@ -1007,14 +1032,14 @@ static int deltaparsevtabFilter(
  a = pCur->aDelta;
  pCur->eOp = DELTAPARSE_OP_SIZE;
  pCur->a1 = deltaGetInt(&a, &i);
-  if( a[0]!='\n' ){
+  if( i<=0 || a[0]!='\n' ){
    pCur->eOp = DELTAPARSE_OP_ERROR;
    pCur->a1 = pCur->a2 = 0;
    pCur->iNext = pCur->nDelta;
    return SQLITE_OK;
  }
  a++;
-  pCur->iNext = (unsigned int)(a - pCur->aDelta);
+  pCur->iNext = (i64)(a - pCur->aDelta);
  return SQLITE_OK;
 }

@@ -556,7 +556,7 @@ static int fuzzerConnect(
 static int fuzzerOpen(sqlite3_vtab *pVTab, sqlite3_vtab_cursor **ppCursor){
  fuzzer_vtab *p = (fuzzer_vtab*)pVTab;
  fuzzer_cursor *pCur;
-  pCur = sqlite3_malloc( sizeof(*pCur) );
+  pCur = sqlite3_malloc64( sizeof(*pCur) );
  if( pCur==0 ) return SQLITE_NOMEM;
  memset(pCur, 0, sizeof(*pCur));
  pCur->pVtab = p;
@@ -622,7 +622,7 @@ static int fuzzerRender(

  n = pStem->nBasis + pRule->nTo - pRule->nFrom;
  if( (*pnBuf)<n+1 ){
-    (*pzBuf) = sqlite3_realloc((*pzBuf), n+100);
+    (*pzBuf) = sqlite3_realloc64((*pzBuf), n+100);
    if( (*pzBuf)==0 ) return SQLITE_NOMEM;
    (*pnBuf) = n+100;
  }
@@ -85,7 +85,7 @@ static int memstatConnect(

  rc = sqlite3_declare_vtab(db,"CREATE TABLE x(name,schema,value,hiwtr)");
  if( rc==SQLITE_OK ){
-    pNew = sqlite3_malloc( sizeof(*pNew) );
+    pNew = sqlite3_malloc64( sizeof(*pNew) );
    *ppVtab = (sqlite3_vtab*)pNew;
    if( pNew==0 ) return SQLITE_NOMEM;
    memset(pNew, 0, sizeof(*pNew));
@@ -107,7 +107,7 @@ static int memstatDisconnect(sqlite3_vtab *pVtab){
 */
 static int memstatOpen(sqlite3_vtab *p, sqlite3_vtab_cursor **ppCursor){
  memstat_cursor *pCur;
-  pCur = sqlite3_malloc( sizeof(*pCur) );
+  pCur = sqlite3_malloc64( sizeof(*pCur) );
  if( pCur==0 ) return SQLITE_NOMEM;
  memset(pCur, 0, sizeof(*pCur));
  pCur->db = ((memstat_vtab*)p)->db;
@@ -340,45 +340,45 @@ static void percentSort(double *a, unsigned int n){
  int iGt;  /* Entries at or after a[iGt] are greater than rPivot */
  int i;         /* Loop counter */
  double rPivot; /* The pivot value */
-  
-  assert( n>=2 );
-  if( a[0]>a[n-1] ){
-    SWAP_DOUBLE(a[0],a[n-1])
-  }
-  if( n==2 ) return;
-  iGt = n-1;
-  i = n/2;
-  if( a[0]>a[i] ){
-    SWAP_DOUBLE(a[0],a[i])
-  }else if( a[i]>a[iGt] ){
-    SWAP_DOUBLE(a[i],a[iGt])
-  }
-  if( n==3 ) return;
-  rPivot = a[i];
-  iLt = i = 1;
-  do{
-    if( a[i]<rPivot ){
-      if( i>iLt ) SWAP_DOUBLE(a[i],a[iLt])
-      iLt++;
-      i++;
-    }else if( a[i]>rPivot ){
-      do{
-        iGt--;
-      }while( iGt>i && a[iGt]>rPivot );
-      SWAP_DOUBLE(a[i],a[iGt])
-    }else{
-      i++;
+
+  while( n>=2 ){
+    if( a[0]>a[n-1] ){
+      SWAP_DOUBLE(a[0],a[n-1])
+    }
+    if( n==2 ) return;
+    iGt = n-1;
+    i = n/2;
+    if( a[0]>a[i] ){
+      SWAP_DOUBLE(a[0],a[i])
+    }else if( a[i]>a[iGt] ){
+      SWAP_DOUBLE(a[i],a[iGt])
+    }
+    if( n==3 ) return;
+    rPivot = a[i];
+    iLt = i = 1;
+    do{
+      if( a[i]<rPivot ){
+        if( i>iLt ) SWAP_DOUBLE(a[i],a[iLt])
+        iLt++;
+        i++;
+      }else if( a[i]>rPivot ){
+        do{
+          iGt--;
+        }while( iGt>i && a[iGt]>rPivot );
+        SWAP_DOUBLE(a[i],a[iGt])
+      }else{
+        i++;
+      }
+    }while( i<iGt );
+    if( iLt>n/2 ){
+      if( n-iGt>=2 ) percentSort(a+iGt, n-iGt);
+      n = iLt;
+    }else{
+      if( iLt>=2 ) percentSort(a, iLt);
+      a += iGt;
+      n -= iGt;
    }
-  }while( i<iGt );
-  if( iLt>=2 ) percentSort(a, iLt);
-  if( n-iGt>=2 ) percentSort(a+iGt, n-iGt);
-    
-/* Uncomment for testing */
-#if 0
-  for(i=0; i<n-1; i++){
-    assert( a[i]<=a[i+1] );
  }
-#endif
 }


@@ -75,7 +75,7 @@ static int prefixesConnect(
           "CREATE TABLE prefixes(prefix TEXT, original_string TEXT HIDDEN)"
       );
  if( rc==SQLITE_OK ){
-    pNew = sqlite3_malloc( sizeof(*pNew) );
+    pNew = sqlite3_malloc64( sizeof(*pNew) );
    *ppVtab = (sqlite3_vtab*)pNew;
    if( pNew==0 ) return SQLITE_NOMEM;
    memset(pNew, 0, sizeof(*pNew));
@@ -98,7 +98,7 @@ static int prefixesDisconnect(sqlite3_vtab *pVtab){
 */
 static int prefixesOpen(sqlite3_vtab *p, sqlite3_vtab_cursor **ppCursor){
  prefixes_cursor *pCur;
-  pCur = sqlite3_malloc( sizeof(*pCur) );
+  pCur = sqlite3_malloc64( sizeof(*pCur) );
  if( pCur==0 ) return SQLITE_NOMEM;
  memset(pCur, 0, sizeof(*pCur));
  *ppCursor = &pCur->base;
@@ -289,10 +289,14 @@ static void prefixLengthFunc(
  int nL = sqlite3_value_bytes(apVal[0]);
  int nR = sqlite3_value_bytes(apVal[1]);
  int i;
+  if( zL==0 || zR==0 ){
+    sqlite3_result_int(ctx, 0);
+    return;
+  }

  nByte = (nL > nR ? nL : nR);
  for(i=0; i<nByte; i++){
-    if( zL[i]!=zR[i] ) break;
+    if( zL[i]==0 || zL[i]!=zR[i] ) break;
    if( (zL[i] & 0xC0)!=0x80 ) nRet++;
  }

@@ -152,7 +152,7 @@ static int qpvtabConnect(
 #define QPVTAB_FLAGS  11
 #define QPVTAB_NONE   12
  if( rc==SQLITE_OK ){
-    pNew = sqlite3_malloc( sizeof(*pNew) );
+    pNew = sqlite3_malloc64( sizeof(*pNew) );
    *ppVtab = (sqlite3_vtab*)pNew;
    if( pNew==0 ) return SQLITE_NOMEM;
    memset(pNew, 0, sizeof(*pNew));
@@ -174,7 +174,7 @@ static int qpvtabDisconnect(sqlite3_vtab *pVtab){
 */
 static int qpvtabOpen(sqlite3_vtab *p, sqlite3_vtab_cursor **ppCursor){
  qpvtab_cursor *pCur;
-  pCur = sqlite3_malloc( sizeof(*pCur) );
+  pCur = sqlite3_malloc64( sizeof(*pCur) );
  if( pCur==0 ) return SQLITE_NOMEM;
  memset(pCur, 0, sizeof(*pCur));
  *ppCursor = &pCur->base;
@@ -363,7 +363,7 @@ static int qpvtabBestIndex(
    }
    sqlite3_str_appendf(pStr,"aConstraint,%d,%s,%d,%d,",
       i,
-       azColname[iCol],
+       iCol>=0 ? azColname[iCol] : "rowid",
       op,
       pIdxInfo->aConstraint[i].usable);
    pVal = 0;
@@ -676,7 +676,7 @@ static const char *re_compile(
  int i, j;

  *ppRe = 0;
-  pRe = sqlite3_malloc( sizeof(*pRe) );
+  pRe = sqlite3_malloc64( sizeof(*pRe) );
  if( pRe==0 ){
    return "out of memory";
  }
@@ -828,7 +828,6 @@ static void re_bytecode_func(
  int i;
  int n;
  char *z;
-  (void)argc;
  static const char *ReOpName[] = {
    "EOF",
    "MATCH",
@@ -851,6 +850,7 @@ static void re_bytecode_func(
    "ATSTART",
  };

+  (void)argc;
  zPattern = (const char*)sqlite3_value_text(argv[0]);
  if( zPattern==0 ) return;
  zErr = re_compile(&pRe, zPattern, re_maxnfa(re_maxlen(context)),
@@ -865,7 +865,6 @@ static void re_bytecode_func(
    return;
  }
  pStr = sqlite3_str_new(0);
-  if( pStr==0 ) goto re_bytecode_func_err;
  if( pRe->nInit>0 ){
    sqlite3_str_appendf(pStr, "INIT     ");
    for(i=0; i<pRe->nInit; i++){
@@ -877,15 +876,15 @@ static void re_bytecode_func(
    sqlite3_str_appendf(pStr, "%-8s %4d\n",
         ReOpName[(unsigned char)pRe->aOp[i]], pRe->aArg[i]);
  }
+  if( sqlite3_str_errcode(pStr)==SQLITE_NOMEM ){
+    sqlite3_str_finish(pStr);
+    re_free(pRe);
+    sqlite3_result_error_nomem(context);
+    return;
+  }
  n = sqlite3_str_length(pStr);
  z = sqlite3_str_finish(pStr);
-  if( n==0 ){
-    sqlite3_free(z);
-  }else{
-    sqlite3_result_text(context, z, n-1, sqlite3_free);
-  }
-
-re_bytecode_func_err:
+  sqlite3_result_text(context, z, n-1, sqlite3_free);
  re_free(pRe);
 }

@@ -1,610 +0,0 @@
-/*
-** 2016-05-05
-**
-** The author disclaims copyright to this source code.  In place of
-** a legal notice, here is a blessing:
-**
-**    May you do good and not evil.
-**    May you find forgiveness for yourself and forgive others.
-**    May you share freely, never taking more than you give.
-**
-******************************************************************************
-**
-** This file implements a utility function (and a utility program) that
-** makes a copy of an SQLite database while simultaneously zeroing out all
-** deleted content.
-**
-** Normally (when PRAGMA secure_delete=OFF, which is the default) when SQLite
-** deletes content, it does not overwrite the deleted content but rather marks
-** the region of the file that held that content as being reusable.  This can
-** cause deleted content to recoverable from the database file.  This stale
-** content is removed by the VACUUM command, but VACUUM can be expensive for
-** large databases.  When in PRAGMA secure_delete=ON mode, the deleted content
-** is zeroed, but secure_delete=ON has overhead as well.
-**
-** This utility attempts to make a copy of a complete SQLite database where
-** all of the deleted content is zeroed out in the copy, and it attempts to
-** do so while being faster than running VACUUM.
-**
-** Usage:
-**
-**   int sqlite3_scrub_backup(
-**       const char *zSourceFile,   // Source database filename
-**       const char *zDestFile,     // Destination database filename
-**       char **pzErrMsg            // Write error message here
-**   );
-**
-** Simply call the API above specifying the filename of the source database
-** and the name of the backup copy.  The source database must already exist
-** and can be in active use. (A read lock is held during the backup.)  The
-** destination file should not previously exist.  If the pzErrMsg parameter
-** is non-NULL and if an error occurs, then an error message might be written
-** into memory obtained from sqlite3_malloc() and *pzErrMsg made to point to
-** that error message.  But if the error is an OOM, the error might not be
-** reported.  The routine always returns non-zero if there is an error.
-**
-** If compiled with -DSCRUB_STANDALONE then a main() procedure is added and
-** this file becomes a standalone program that can be run as follows:
-**
-**      ./sqlite3scrub SOURCE DEST
-*/
-#include "sqlite3.h"
-#include <assert.h>
-#include <stdio.h>
-#include <stdlib.h>
-#include <stdarg.h>
-#include <string.h>
-
-typedef struct ScrubState ScrubState;
-typedef unsigned char u8;
-typedef unsigned short u16;
-typedef unsigned int u32;
-
-
-/* State information for a scrub-and-backup operation */
-struct ScrubState {
-  const char *zSrcFile;    /* Name of the source file */
-  const char *zDestFile;   /* Name of the destination file */
-  int rcErr;               /* Error code */
-  char *zErr;              /* Error message text */
-  sqlite3 *dbSrc;          /* Source database connection */
-  sqlite3_file *pSrc;      /* Source file handle */
-  sqlite3 *dbDest;         /* Destination database connection */
-  sqlite3_file *pDest;     /* Destination file handle */
-  u32 szPage;              /* Page size */
-  u32 szUsable;            /* Usable bytes on each page */
-  u32 nPage;               /* Number of pages */
-  u32 iLastPage;           /* Page number of last page written so far*/
-  u8 *page1;               /* Content of page 1 */
-};
-
-/* Store an error message */
-static void scrubBackupErr(ScrubState *p, const char *zFormat, ...){
-  va_list ap;
-  sqlite3_free(p->zErr);
-  va_start(ap, zFormat);
-  p->zErr = sqlite3_vmprintf(zFormat, ap);
-  va_end(ap);
-  if( p->rcErr==0 ) p->rcErr = SQLITE_ERROR;
-}
-
-/* Allocate memory to hold a single page of content */
-static u8 *scrubBackupAllocPage(ScrubState *p){
-  u8 *pPage;
-  if( p->rcErr ) return 0;
-  pPage = sqlite3_malloc( p->szPage );
-  if( pPage==0 ) p->rcErr = SQLITE_NOMEM;
-  return pPage;
-}
-
-/* Read a page from the source database into memory.  Use the memory
-** provided by pBuf if not NULL or allocate a new page if pBuf==NULL.
-*/
-static u8 *scrubBackupRead(ScrubState *p, int pgno, u8 *pBuf){
-  int rc;
-  sqlite3_int64 iOff;
-  u8 *pOut = pBuf;
-  if( p->rcErr ) return 0;
-  if( pOut==0 ){
-    pOut = scrubBackupAllocPage(p);
-    if( pOut==0 ) return 0;
-  }
-  iOff = (pgno-1)*(sqlite3_int64)p->szPage;
-  rc = p->pSrc->pMethods->xRead(p->pSrc, pOut, p->szPage, iOff);
-  if( rc!=SQLITE_OK ){
-    if( pBuf==0 ) sqlite3_free(pOut);
-    pOut = 0;
-    scrubBackupErr(p, "read failed for page %d", pgno);
-    p->rcErr = SQLITE_IOERR;
-  }
-  return pOut;  
-}
-
-/* Write a page to the destination database */
-static void scrubBackupWrite(ScrubState *p, int pgno, const u8 *pData){
-  int rc;
-  sqlite3_int64 iOff;
-  if( p->rcErr ) return;
-  iOff = (pgno-1)*(sqlite3_int64)p->szPage;
-  rc = p->pDest->pMethods->xWrite(p->pDest, pData, p->szPage, iOff);
-  if( rc!=SQLITE_OK ){
-    scrubBackupErr(p, "write failed for page %d", pgno);
-    p->rcErr = SQLITE_IOERR;
-  }
-  if( (u32)pgno>p->iLastPage ) p->iLastPage = pgno;
-}
-
-/* Prepare a statement against the "db" database. */
-static sqlite3_stmt *scrubBackupPrepare(
-  ScrubState *p,      /* Backup context */
-  sqlite3 *db,        /* Database to prepare against */
-  const char *zSql    /* SQL statement */
-){
-  sqlite3_stmt *pStmt;
-  if( p->rcErr ) return 0;
-  p->rcErr = sqlite3_prepare_v2(db, zSql, -1, &pStmt, 0);
-  if( p->rcErr ){
-    scrubBackupErr(p, "SQL error \"%s\" on \"%s\"",
-                   sqlite3_errmsg(db), zSql);
-    sqlite3_finalize(pStmt);
-    return 0;
-  }
-  return pStmt;
-}
-
-
-/* Open the source database file */
-static void scrubBackupOpenSrc(ScrubState *p){
-  sqlite3_stmt *pStmt;
-  int rc;
-  /* Open the source database file */
-  p->rcErr = sqlite3_open_v2(p->zSrcFile, &p->dbSrc,
-                 SQLITE_OPEN_READWRITE |
-                 SQLITE_OPEN_URI | SQLITE_OPEN_PRIVATECACHE, 0);
-  if( p->rcErr ){
-    scrubBackupErr(p, "cannot open source database: %s",
-                      sqlite3_errmsg(p->dbSrc));
-    return;
-  }
-  p->rcErr = sqlite3_exec(p->dbSrc, "SELECT 1 FROM sqlite_schema; BEGIN;",
-                          0, 0, 0);
-  if( p->rcErr ){
-    scrubBackupErr(p,
-       "cannot start a read transaction on the source database: %s",
-       sqlite3_errmsg(p->dbSrc));
-    return;
-  }
-  rc = sqlite3_wal_checkpoint_v2(p->dbSrc, "main", SQLITE_CHECKPOINT_FULL,
-                                 0, 0);
-  if( rc ){
-    scrubBackupErr(p, "cannot checkpoint the source database");
-    return;
-  }
-  pStmt = scrubBackupPrepare(p, p->dbSrc, "PRAGMA page_size");
-  if( pStmt==0 ) return;
-  rc = sqlite3_step(pStmt);
-  if( rc==SQLITE_ROW ){
-    p->szPage = sqlite3_column_int(pStmt, 0);
-  }else{
-    scrubBackupErr(p, "unable to determine the page size");
-  }
-  sqlite3_finalize(pStmt);
-  if( p->rcErr ) return;
-  pStmt = scrubBackupPrepare(p, p->dbSrc, "PRAGMA page_count");
-  if( pStmt==0 ) return;
-  rc = sqlite3_step(pStmt);
-  if( rc==SQLITE_ROW ){
-    p->nPage = sqlite3_column_int(pStmt, 0);
-  }else{
-    scrubBackupErr(p, "unable to determine the size of the source database");
-  }
-  sqlite3_finalize(pStmt);
-  sqlite3_file_control(p->dbSrc, "main", SQLITE_FCNTL_FILE_POINTER, &p->pSrc);
-  if( p->pSrc==0 || p->pSrc->pMethods==0 ){
-    scrubBackupErr(p, "cannot get the source file handle");
-    p->rcErr = SQLITE_ERROR;
-  }
-}
-
-/* Create and open the destination file */
-static void scrubBackupOpenDest(ScrubState *p){
-  sqlite3_stmt *pStmt;
-  int rc;
-  char *zSql;
-  if( p->rcErr ) return;
-  p->rcErr = sqlite3_open_v2(p->zDestFile, &p->dbDest,
-                 SQLITE_OPEN_READWRITE | SQLITE_OPEN_CREATE |
-                 SQLITE_OPEN_URI | SQLITE_OPEN_PRIVATECACHE, 0);
-  if( p->rcErr ){
-    scrubBackupErr(p, "cannot open destination database: %s",
-                      sqlite3_errmsg(p->dbDest));
-    return;
-  }
-  zSql = sqlite3_mprintf("PRAGMA page_size(%u);", p->szPage);
-  if( zSql==0 ){
-    p->rcErr = SQLITE_NOMEM;
-    return;
-  }
-  p->rcErr = sqlite3_exec(p->dbDest, zSql, 0, 0, 0);
-  sqlite3_free(zSql);
-  if( p->rcErr ){
-    scrubBackupErr(p,
-       "cannot set the page size on the destination database: %s",
-       sqlite3_errmsg(p->dbDest));
-    return;
-  }
-  sqlite3_exec(p->dbDest, "PRAGMA journal_mode=OFF;", 0, 0, 0);
-  p->rcErr = sqlite3_exec(p->dbDest, "BEGIN EXCLUSIVE;", 0, 0, 0);
-  if( p->rcErr ){
-    scrubBackupErr(p,
-       "cannot start a write transaction on the destination database: %s",
-       sqlite3_errmsg(p->dbDest));
-    return;
-  }
-  pStmt = scrubBackupPrepare(p, p->dbDest, "PRAGMA page_count;");
-  if( pStmt==0 ) return;
-  rc = sqlite3_step(pStmt);
-  if( rc!=SQLITE_ROW ){
-    scrubBackupErr(p, "cannot measure the size of the destination");
-  }else if( sqlite3_column_int(pStmt, 0)>1 ){
-    scrubBackupErr(p, "destination database is not empty - holds %d pages",
-                   sqlite3_column_int(pStmt, 0));
-  }
-  sqlite3_finalize(pStmt);
-  sqlite3_file_control(p->dbDest, "main", SQLITE_FCNTL_FILE_POINTER, &p->pDest);
-  if( p->pDest==0 || p->pDest->pMethods==0 ){
-    scrubBackupErr(p, "cannot get the destination file handle");
-    p->rcErr = SQLITE_ERROR;
-  }
-}
-
-/* Read a 32-bit big-endian integer */
-static u32 scrubBackupInt32(const u8 *a){
-  u32 v = a[3];
-  v += ((u32)a[2])<<8;
-  v += ((u32)a[1])<<16;
-  v += ((u32)a[0])<<24;
-  return v;
-}
-
-/* Read a 16-bit big-endian integer */
-static u32 scrubBackupInt16(const u8 *a){
-  return (a[0]<<8) + a[1];
-}
-
-/*
-** Read a varint.  Put the value in *pVal and return the number of bytes.
-*/
-static int scrubBackupVarint(const u8 *z, sqlite3_int64 *pVal){
-  sqlite3_int64 v = 0;
-  int i;
-  for(i=0; i<8; i++){
-    v = (v<<7) + (z[i]&0x7f);
-    if( (z[i]&0x80)==0 ){ *pVal = v; return i+1; }
-  }
-  v = (v<<8) + (z[i]&0xff);
-  *pVal = v;
-  return 9;
-}
-
-/*
-** Return the number of bytes in a varint.
-*/
-static int scrubBackupVarintSize(const u8 *z){
-  int i;
-  for(i=0; i<8; i++){
-    if( (z[i]&0x80)==0 ){ return i+1; }
-  }
-  return 9;
-}
-
-/*
-** Copy the freelist trunk page given, and all its descendents,
-** zeroing out as much as possible in the process.
-*/
-static void scrubBackupFreelist(ScrubState *p, int pgno, u32 nFree){
-  u8 *a, *aBuf;
-  u32 n, mx;
-
-  if( p->rcErr ) return;
-  aBuf = scrubBackupAllocPage(p);
-  if( aBuf==0 ) return;
- 
-  while( pgno && nFree){
-    a = scrubBackupRead(p, pgno, aBuf);
-    if( a==0 ) break;
-    n = scrubBackupInt32(&a[4]);
-    mx = p->szUsable/4 - 2;
-    if( n<mx ){
-      memset(&a[n*4+8], 0, 4*(mx-n));
-    }
-    scrubBackupWrite(p, pgno, a);
-    pgno = scrubBackupInt32(a);
-#if 0
-    /* There is really no point in copying the freelist leaf pages.
-    ** Simply leave them uninitialized in the destination database.  The
-    ** OS filesystem should zero those pages for us automatically.
-    */
-    for(i=0; i<n && nFree; i++){
-      u32 iLeaf = scrubBackupInt32(&a[i*4+8]);
-      if( aZero==0 ){
-        aZero = scrubBackupAllocPage(p);
-        if( aZero==0 ){ pgno = 0; break; }
-        memset(aZero, 0, p->szPage);
-      }
-      scrubBackupWrite(p, iLeaf, aZero);
-      nFree--;
-    }
-#endif
-  }
-  sqlite3_free(aBuf);
-}
-
-/*
-** Copy an overflow chain from source to destination.  Zero out any
-** unused tail at the end of the overflow chain.
-*/
-static void scrubBackupOverflow(ScrubState *p, int pgno, u32 nByte){
-  u8 *a, *aBuf;
-
-  aBuf = scrubBackupAllocPage(p);
-  if( aBuf==0 ) return;
-  while( nByte>0 && pgno!=0 ){
-    a = scrubBackupRead(p, pgno, aBuf);
-    if( a==0 ) break;
-    if( nByte >= (p->szUsable)-4 ){
-      nByte -= (p->szUsable) - 4;
-    }else{
-      u32 x = (p->szUsable - 4) - nByte;
-      u32 i = p->szUsable - x;
-      memset(&a[i], 0, x);
-      nByte = 0;
-    }
-    scrubBackupWrite(p, pgno, a);
-    pgno = scrubBackupInt32(a);
-  }
-  sqlite3_free(aBuf);      
-}
-   
-
-/*
-** Copy B-Tree page pgno, and all of its children, from source to destination.
-** Zero out deleted content during the copy.
-*/
-static void scrubBackupBtree(ScrubState *p, int pgno, int iDepth){
-  u8 *a;
-  u32 i, n, pc;
-  u32 nCell;
-  u32 nPrefix;
-  u32 szHdr;
-  u32 iChild;
-  u8 *aTop;
-  u8 *aCell;
-  u32 x, y;
-  int ln = 0;
-
-  
-  if( p->rcErr ) return;
-  if( iDepth>50 ){
-    scrubBackupErr(p, "corrupt: b-tree too deep at page %d", pgno);
-    return;
-  }
-  if( pgno==1 ){
-    a = p->page1;
-  }else{
-    a = scrubBackupRead(p, pgno, 0);
-    if( a==0 ) return;
-  }
-  nPrefix = pgno==1 ? 100 : 0;
-  aTop = &a[nPrefix];
-  szHdr = 8 + 4*(aTop[0]==0x02 || aTop[0]==0x05);
-  aCell = aTop + szHdr;
-  nCell = scrubBackupInt16(&aTop[3]);
-
-  /* Zero out the gap between the cell index and the start of the
-  ** cell content area */
-  x = scrubBackupInt16(&aTop[5]);  /* First byte of cell content area */
-  if( x>p->szUsable ){ ln=__LINE__; goto btree_corrupt; }
-  y = szHdr + nPrefix + nCell*2;
-  if( y>x ){ ln=__LINE__; goto btree_corrupt; }
-  if( y<x ) memset(a+y, 0, x-y);  /* Zero the gap */
-
-  /* Zero out all the free blocks */  
-  pc = scrubBackupInt16(&aTop[1]);
-  if( pc>0 && pc<x ){ ln=__LINE__; goto btree_corrupt; }
-  while( pc ){
-    if( pc>(p->szUsable)-4 ){ ln=__LINE__; goto btree_corrupt; }
-    n = scrubBackupInt16(&a[pc+2]);
-    if( pc+n>(p->szUsable) ){ ln=__LINE__; goto btree_corrupt; }
-    if( n>4 ) memset(&a[pc+4], 0, n-4);
-    x = scrubBackupInt16(&a[pc]);
-    if( x<pc+4 && x>0 ){ ln=__LINE__; goto btree_corrupt; }
-    pc = x;
-  }
-
-  /* Write this one page */
-  scrubBackupWrite(p, pgno, a);
-
-  /* Walk the tree and process child pages */
-  for(i=0; i<nCell; i++){
-    u32 X, M, K, nLocal;
-    sqlite3_int64 P;
-    pc = scrubBackupInt16(&aCell[i*2]);
-    if( pc <= szHdr ){ ln=__LINE__; goto btree_corrupt; }
-    if( pc > p->szUsable-3 ){ ln=__LINE__; goto btree_corrupt; }
-    if( aTop[0]==0x05 || aTop[0]==0x02 ){
-      if( pc+4 > p->szUsable ){ ln=__LINE__; goto btree_corrupt; }
-      iChild = scrubBackupInt32(&a[pc]);
-      pc += 4;
-      scrubBackupBtree(p, iChild, iDepth+1);
-      if( aTop[0]==0x05 ) continue;
-    }
-    pc += scrubBackupVarint(&a[pc], &P);
-    if( pc >= p->szUsable ){ ln=__LINE__; goto btree_corrupt; }
-    if( aTop[0]==0x0d ){
-      X = p->szUsable - 35;
-    }else{
-      X = ((p->szUsable - 12)*64/255) - 23;
-    }
-    if( P<=X ){
-      /* All content is local.  No overflow */
-      continue;
-    }
-    M = ((p->szUsable - 12)*32/255)-23;
-    K = M + ((P-M)%(p->szUsable-4));
-    if( aTop[0]==0x0d ){
-      pc += scrubBackupVarintSize(&a[pc]);
-      if( pc > (p->szUsable-4) ){ ln=__LINE__; goto btree_corrupt; }
-    }
-    nLocal = K<=X ? K : M;
-    if( pc+nLocal > p->szUsable-4 ){ ln=__LINE__; goto btree_corrupt; }
-    iChild = scrubBackupInt32(&a[pc+nLocal]);
-    scrubBackupOverflow(p, iChild, (u32)(P-nLocal));
-  }
-
-  /* Walk the right-most tree */
-  if( aTop[0]==0x05 || aTop[0]==0x02 ){
-    iChild = scrubBackupInt32(&aTop[8]);
-    scrubBackupBtree(p, iChild, iDepth+1);
-  }
-
-  /* All done */
-  if( pgno>1 ) sqlite3_free(a);
-  return;
-
-btree_corrupt:
-  scrubBackupErr(p, "corruption on page %d of source database (errid=%d)",
-                 pgno, ln);
-  if( pgno>1 ) sqlite3_free(a);  
-}
-
-/*
-** Copy all ptrmap pages from source to destination.
-** This routine is only called if the source database is in autovacuum
-** or incremental vacuum mode.
-*/
-static void scrubBackupPtrmap(ScrubState *p){
-  u32 pgno = 2;
-  u32 J = p->szUsable/5;
-  u32 iLock = (1073742335/p->szPage)+1;
-  u8 *a, *pBuf;
-  if( p->rcErr ) return;
-  pBuf = scrubBackupAllocPage(p);
-  if( pBuf==0 ) return;
-  while( pgno<=p->nPage ){
-    a = scrubBackupRead(p, pgno, pBuf);
-    if( a==0 ) break;
-    scrubBackupWrite(p, pgno, a);
-    pgno += J+1;
-    if( pgno==iLock ) pgno++;
-  }
-  sqlite3_free(pBuf);
-}
-
-int sqlite3_scrub_backup(
-  const char *zSrcFile,    /* Source file */
-  const char *zDestFile,   /* Destination file */
-  char **pzErr             /* Write error here if non-NULL */
-){
-  ScrubState s;
-  u32 n, i;
-  sqlite3_stmt *pStmt;
-
-  memset(&s, 0, sizeof(s));
-  s.zSrcFile = zSrcFile;
-  s.zDestFile = zDestFile;
-
-  /* Open both source and destination databases */
-  scrubBackupOpenSrc(&s);
-  scrubBackupOpenDest(&s);
-
-  /* Read in page 1 */
-  s.page1 = scrubBackupRead(&s, 1, 0);
-  if( s.page1==0 ) goto scrub_abort;
-  s.szUsable = s.szPage - s.page1[20];
-
-  /* Copy the freelist */    
-  n = scrubBackupInt32(&s.page1[36]);
-  i = scrubBackupInt32(&s.page1[32]);
-  if( n ) scrubBackupFreelist(&s, i, n);
-
-  /* Copy ptrmap pages */
-  n = scrubBackupInt32(&s.page1[52]);
-  if( n ) scrubBackupPtrmap(&s);
-
-  /* Copy all of the btrees */
-  scrubBackupBtree(&s, 1, 0);
-  pStmt = scrubBackupPrepare(&s, s.dbSrc,
-       "SELECT rootpage FROM sqlite_schema WHERE coalesce(rootpage,0)>0");
-  if( pStmt==0 ) goto scrub_abort;
-  while( sqlite3_step(pStmt)==SQLITE_ROW ){
-    i = (u32)sqlite3_column_int(pStmt, 0);
-    scrubBackupBtree(&s, i, 0);
-  }
-  sqlite3_finalize(pStmt);
-
-  /* If the last page of the input db file is a free-list leaf, then the
-  ** backup file on disk is still smaller than the size indicated within 
-  ** the database header. In this case, write a page of zeroes to the 
-  ** last page of the backup database so that SQLite does not mistakenly
-  ** think the db is corrupt.  */
-  if( s.iLastPage<s.nPage ){
-    u8 *aZero = scrubBackupAllocPage(&s);
-    if( aZero ){
-      memset(aZero, 0, s.szPage);
-      scrubBackupWrite(&s, s.nPage, aZero);
-      sqlite3_free(aZero);
-    }
-  }
-
-scrub_abort:    
-  /* Close the destination database without closing the transaction. If we
-  ** commit, page zero will be overwritten. */
-  sqlite3_close(s.dbDest);
-
-  /* But do close out the read-transaction on the source database */
-  sqlite3_exec(s.dbSrc, "COMMIT;", 0, 0, 0);
-  sqlite3_close(s.dbSrc);
-  sqlite3_free(s.page1);
-  if( pzErr ){
-    *pzErr = s.zErr;
-  }else{
-    sqlite3_free(s.zErr);
-  }
-  return s.rcErr;
-}   
-
-#ifdef SCRUB_STANDALONE
-/* Error and warning log */
-static void errorLogCallback(void *pNotUsed, int iErr, const char *zMsg){
-  const char *zType;
-  switch( iErr&0xff ){
-    case SQLITE_WARNING: zType = "WARNING";  break;
-    case SQLITE_NOTICE:  zType = "NOTICE";   break;
-    default:             zType = "ERROR";    break;
-  }
-  fprintf(stderr, "%s: %s\n", zType, zMsg);
-}
-
-/* The main() routine when this utility is run as a stand-alone program */
-int main(int argc, char **argv){
-  char *zErr = 0;
-  int rc;
-  if( argc!=3 ){
-    fprintf(stderr,"Usage: %s SOURCE DESTINATION\n", argv[0]);
-    exit(1);
-  }
-  sqlite3_config(SQLITE_CONFIG_LOG, errorLogCallback, 0);
-  rc = sqlite3_scrub_backup(argv[1], argv[2], &zErr);
-  if( rc==SQLITE_NOMEM ){
-    fprintf(stderr, "%s: out of memory\n", argv[0]);
-    exit(1);
-  }
-  if( zErr ){
-    fprintf(stderr, "%s: %s\n", argv[0], zErr);
-    sqlite3_free(zErr);
-    exit(1);
-  }
-  return 0;
-}
-#endif
@@ -239,7 +239,7 @@ static int seriesConnect(
  rc = sqlite3_declare_vtab(db,
     "CREATE TABLE x(value,start hidden,stop hidden,step hidden)");
  if( rc==SQLITE_OK ){
-    pNew = *ppVtab = sqlite3_malloc( sizeof(*pNew) );
+    pNew = *ppVtab = sqlite3_malloc64( sizeof(*pNew) );
    if( pNew==0 ) return SQLITE_NOMEM;
    memset(pNew, 0, sizeof(*pNew));
    sqlite3_vtab_config(db, SQLITE_VTAB_INNOCUOUS);
@@ -261,7 +261,7 @@ static int seriesDisconnect(sqlite3_vtab *pVtab){
 static int seriesOpen(sqlite3_vtab *pUnused, sqlite3_vtab_cursor **ppCursor){
  series_cursor *pCur;
  (void)pUnused;
-  pCur = sqlite3_malloc( sizeof(*pCur) );
+  pCur = sqlite3_malloc64( sizeof(*pCur) );
  if( pCur==0 ) return SQLITE_NOMEM;
  memset(pCur, 0, sizeof(*pCur));
  *ppCursor = &pCur->base;
@@ -536,7 +536,7 @@ static int seriesFilter(
          if( r<(double)SMALLEST_INT64 ){
            iMin = SMALLEST_INT64;
          }else if( (idxNum & 0x0200)!=0 && r==seriesCeil(r) ){
-            iMin = (sqlite3_int64)seriesCeil(r+1.0);
+            iMin = (sqlite3_int64)seriesCeil(r)+1;
          }else{
            iMin = (sqlite3_int64)seriesCeil(r);
          }
@@ -557,7 +557,7 @@ static int seriesFilter(
          if( r>(double)LARGEST_INT64 ){
            iMax = LARGEST_INT64;
          }else if( (idxNum & 0x2000)!=0 && r==seriesFloor(r) ){
-            iMax = (sqlite3_int64)(r-1.0);
+            iMax = ((sqlite3_int64)r)-1;
          }else{
            iMax = (sqlite3_int64)seriesFloor(r);
          }
@@ -31,6 +31,10 @@ SQLITE_EXTENSION_INIT1
 /******************************************************************************
 ** The Hash Engine
 */
+#if defined(__GNUC__) && __GNUC__>=11
+# pragma GCC diagnostic push
+# pragma GCC diagnostic ignored "-Wstringop-overread"
+#endif
 /* Context for the SHA1 hash */
 typedef struct SHA1Context SHA1Context;
 struct SHA1Context {
@@ -226,6 +230,9 @@ static void hash_finish(
    zOut[i*2]= 0;
  }
 }
+#if defined(__GNUC__) && __GNUC__>=11
+# pragma GCC diagnostic pop
+#endif
 /* End of the hashing logic
 *****************************************************************************/

@@ -32,6 +32,7 @@ SQLITE_EXTENSION_INIT1
 # define NEVER(X)   0
  typedef unsigned char u8;
  typedef unsigned short u16;
+  typedef sqlite3_int64 i64;
 #endif
 #include <ctype.h>

@@ -192,7 +193,7 @@ static const unsigned char className[] = ".ABCDHLRMY9 ?";
 ** Return NULL if memory allocation fails.  
 */
 static unsigned char *phoneticHash(const unsigned char *zIn, int nIn){
-  unsigned char *zOut = sqlite3_malloc64( nIn + 1 );
+  unsigned char *zOut = sqlite3_malloc64( (i64)nIn + 1 );
  int i;
  int nOut = 0;
  char cPrev = 0x77;
@@ -352,6 +353,7 @@ static int substituteCost(char cPrev, char cFrom, char cTo){
 **    -1  One of the inputs is NULL
 **    -2  Non-ASCII characters on input
 **    -3  Unable to allocate memory
+**    -4  Inputs too large
 **
 ** If pnMatch is not NULL, then *pnMatch is set to the number of bytes
 ** of zB that matched the pattern in zA. If zA does not end with a '*',
@@ -390,9 +392,11 @@ static int editdist1(const char *zA, const char *zB, int *pnMatch){
  for(nA=0; zA[nA]; nA++){
    if( zA[nA]&0x80 ) return -2;
  }
+  if( nA>=100000 ) return -4;
  for(nB=0; zB[nB]; nB++){
    if( zB[nB]&0x80 ) return -2;
  }
+  if( nB>=100000 ) return -4;

  /* Special processing if either string is empty */
  if( nA==0 ){
@@ -419,7 +423,7 @@ static int editdist1(const char *zA, const char *zB, int *pnMatch){
  if( nB<(sizeof(mStack)*4)/(sizeof(mStack[0])*5) ){
    m = mStack;
  }else{
-    m = toFree = sqlite3_malloc64( (nB+1)*5*sizeof(m[0])/4 );
+    m = toFree = sqlite3_malloc64( ((i64)nB+1)*5LL*sizeof(m[0])/4 );
    if( m==0 ) return -3;
  }
  cx = (char*)&m[nB+1];
@@ -527,6 +531,8 @@ static void editdistSqlFunc(
  if( res<0 ){
    if( res==(-3) ){
      sqlite3_result_error_nomem(context);
+    }else if( res==(-4) ){
+      sqlite3_result_error_toobig(context);
    }else if( res==(-2) ){
      sqlite3_result_error(context, "non-ASCII input to editdist()", -1);
    }else{
@@ -767,7 +773,7 @@ static int editDist3ConfigLoad(
    if( iCost>=10000 ) continue;  /* Costs above 10K are considered infinite */
    if( pLang==0 || iLang!=iLangPrev ){
      EditDist3Lang *pNew;
-      pNew = sqlite3_realloc64(p->a, (p->nLang+1)*sizeof(p->a[0]));
+      pNew = sqlite3_realloc64(p->a, ((i64)p->nLang+1)*sizeof(p->a[0]));
      if( pNew==0 ){ rc = SQLITE_NOMEM; break; }
      p->a = pNew;
      pLang = &p->a[p->nLang];
@@ -901,7 +907,7 @@ static EditDist3FromString *editDist3FromStringNew(

  if( z==0 ) return 0;
  if( n<0 ) n = (int)strlen(z);
-  pStr = sqlite3_malloc64( sizeof(*pStr) + sizeof(pStr->a[0])*n + n + 1 );
+  pStr = sqlite3_malloc64( sizeof(*pStr) + sizeof(pStr->a[0])*n + (i64)n + 1 );
  if( pStr==0 ) return 0;
  pStr->a = (EditDist3From*)&pStr[1];
  memset(pStr->a, 0, sizeof(pStr->a[0])*n);
@@ -927,13 +933,13 @@ static EditDist3FromString *editDist3FromStringNew(
      if( matchFrom(p, z+i, n-i)==0 ) continue;
      if( p->nTo==0 ){
        apNew = sqlite3_realloc64(pFrom->apDel,
-                                sizeof(*apNew)*(pFrom->nDel+1));
+                                sizeof(*apNew)*((i64)pFrom->nDel+1));
        if( apNew==0 ) break;
        pFrom->apDel = apNew;
        apNew[pFrom->nDel++] = p;
      }else{
        apNew = sqlite3_realloc64(pFrom->apSubst,
-                                sizeof(*apNew)*(pFrom->nSubst+1));
+                                sizeof(*apNew)*((i64)pFrom->nSubst+1));
        if( apNew==0 ) break;
        pFrom->apSubst = apNew;
        apNew[pFrom->nSubst++] = p;
@@ -1008,6 +1014,8 @@ static int editDist3Core(
  unsigned int stackSpace[SQLITE_SPELLFIX_STACKALLOC_SZ/sizeof(unsigned int)];

  /* allocate the Wagner matrix and the aTo[] array for the TO string */
+  if( n2>10000 ) return -2;
+  if( f.n>10000 ) return -2;
  n = (f.n+1)*(n2+1);
  n = (n+1)&~1;
  nByte = n*sizeof(m[0]) + sizeof(a2[0])*n2;
@@ -1198,6 +1206,8 @@ static void editDist3SqlFunc(
    editDist3FromStringDelete(pFrom);
    if( dist==(-1) ){
      sqlite3_result_error_nomem(context);
+    }else if( dist==(-2) ){
+      sqlite3_result_error_toobig(context);
    }else{
      sqlite3_result_int(context, dist);
    }
@@ -1712,9 +1722,9 @@ static const Transliteration *spellfixFindTranslit(int c, int *pxTop){
 */
 static unsigned char *transliterate(const unsigned char *zIn, int nIn){
 #ifdef SQLITE_SPELLFIX_5BYTE_MAPPINGS
-  unsigned char *zOut = sqlite3_malloc64( nIn*5 + 1 );
+  unsigned char *zOut = sqlite3_malloc64( (i64)nIn*5 + 1 );
 #else
-  unsigned char *zOut = sqlite3_malloc64( nIn*4 + 1 );
+  unsigned char *zOut = sqlite3_malloc64( (i64)nIn*4 + 1 );
 #endif
  int c, sz, nOut;
  if( zOut==0 ) return 0;
@@ -2057,7 +2067,7 @@ static int spellfix1Init(
  int i;

  nDbName = (int)strlen(zDbName);
-  pNew = sqlite3_malloc64( sizeof(*pNew) + nDbName + 1);
+  pNew = sqlite3_malloc64( sizeof(*pNew) + (i64)nDbName + 1);
  if( pNew==0 ){
    rc = SQLITE_NOMEM;
  }else{
@@ -2461,7 +2471,7 @@ static void spellfix1RunQuery(MatchQuery *p, const char *zQuery, int nQuery){
      iDist = editdist1(p->zPattern, zK1, 0);
    }
    if( iDist<0 ){
-      p->rc = SQLITE_NOMEM;
+      p->rc = iDist==(-4) ? SQLITE_TOOBIG : SQLITE_NOMEM;
      break;
    }
    pCur->nSearch++;
@@ -2773,7 +2783,7 @@ static int spellfix1Column(
          if( !zTranslit ) return SQLITE_NOMEM;
          res = editdist1(pCur->zPattern, zTranslit, &iMatchlen);
          sqlite3_free(zTranslit);
-          if( res<0 ) return SQLITE_NOMEM;
+          if( res<0 ) return res==(-4) ? SQLITE_TOOBIG : SQLITE_NOMEM;
          iMatchlen = translen_to_charlen(zWord, nWord, iMatchlen);
        }else{
          iMatchlen = utf8Charlen(zWord, nWord);
@@ -46,7 +46,7 @@ static void sqlarCompressFunc(
    uLongf nOut = compressBound(nData);
    Bytef *pOut;

-    pOut = (Bytef*)sqlite3_malloc(nOut);
+    pOut = (Bytef*)sqlite3_malloc64(nOut);
    if( pOut==0 ){
      sqlite3_result_error_nomem(context);
      return;
@@ -84,14 +84,14 @@ static void sqlarUncompressFunc(
  sqlite3_int64 sz;

  assert( argc==2 );
-  sz = sqlite3_value_int(argv[1]);
+  sz = sqlite3_value_int64(argv[1]);

  if( sz<=0 || sz==(nData = sqlite3_value_bytes(argv[0])) ){
    sqlite3_result_value(context, argv[0]);
  }else{
    uLongf szf = sz;
    const Bytef *pData= sqlite3_value_blob(argv[0]);
-    Bytef *pOut = sqlite3_malloc(sz);
+    Bytef *pOut = sqlite3_malloc64(sz);
    if( pOut==0 ){
      sqlite3_result_error_nomem(context);
    }else if( Z_OK!=uncompress(pOut, &szf, pData, nData) ){
@@ -101,8 +101,8 @@ FILE *sqlite3_fopen(const char *zFilename, const char *zMode){

  sz1 = (int)strlen(zFilename);
  sz2 = (int)strlen(zMode);
-  b1 = sqlite3_malloc( (sz1+1)*sizeof(b1[0]) );
-  b2 = sqlite3_malloc( (sz2+1)*sizeof(b1[0]) );
+  b1 = sqlite3_malloc64( (sz1+1)*sizeof(b1[0]) );
+  b2 = sqlite3_malloc64( (sz2+1)*sizeof(b1[0]) );
  if( b1 && b2 ){
    sz1 = MultiByteToWideChar(CP_UTF8, 0, zFilename, sz1, b1, sz1);
    b1[sz1] = 0;
@@ -127,8 +127,8 @@ FILE *sqlite3_popen(const char *zCommand, const char *zMode){

  sz1 = (int)strlen(zCommand);
  sz2 = (int)strlen(zMode);
-  b1 = sqlite3_malloc( (sz1+1)*sizeof(b1[0]) );
-  b2 = sqlite3_malloc( (sz2+1)*sizeof(b1[0]) );
+  b1 = sqlite3_malloc64( (sz1+1)*sizeof(b1[0]) );
+  b2 = sqlite3_malloc64( (sz2+1)*sizeof(b1[0]) );
  if( b1 && b2 ){
    sz1 = MultiByteToWideChar(CP_UTF8, 0, zCommand, sz1, b1, sz1);
    b1[sz1] = 0;
@@ -151,7 +151,7 @@ char *sqlite3_fgets(char *buf, int sz, FILE *in){
    ** that into UTF-8.  Otherwise, non-ASCII characters all get translated
    ** into '?'.
    */
-    wchar_t *b1 = sqlite3_malloc( sz*sizeof(wchar_t) );
+    wchar_t *b1 = sqlite3_malloc64( sz*sizeof(wchar_t) );
    if( b1==0 ) return 0;
 #ifdef SQLITE_USE_W32_FOR_CONSOLE_IO
    DWORD nRead = 0;
@@ -226,7 +226,7 @@ int sqlite3_fputs(const char *z, FILE *out){
    ** to the console on Windows. 
    */
    int sz = (int)strlen(z);
-    wchar_t *b1 = sqlite3_malloc( (sz+1)*sizeof(wchar_t) );
+    wchar_t *b1 = sqlite3_malloc64( (sz+1)*sizeof(wchar_t) );
    if( b1==0 ) return 0;
    sz = MultiByteToWideChar(CP_UTF8, 0, z, sz, b1, sz);
    b1[sz] = 0;
@@ -52,7 +52,7 @@ static void stmtrandFunc(
  p = (Stmtrand*)sqlite3_get_auxdata(context, STMTRAND_KEY);
  if( p==0 ){
    unsigned int seed;
-    p = sqlite3_malloc( sizeof(*p) );
+    p = sqlite3_malloc64( sizeof(*p) );
    if( p==0 ){
      sqlite3_result_error_nomem(context);
      return;
@@ -101,7 +101,7 @@ static int templatevtabConnect(
 #define TEMPLATEVTAB_A  0
 #define TEMPLATEVTAB_B  1
  if( rc==SQLITE_OK ){
-    pNew = sqlite3_malloc( sizeof(*pNew) );
+    pNew = sqlite3_malloc64( sizeof(*pNew) );
    *ppVtab = (sqlite3_vtab*)pNew;
    if( pNew==0 ) return SQLITE_NOMEM;
    memset(pNew, 0, sizeof(*pNew));
@@ -123,7 +123,7 @@ static int templatevtabDisconnect(sqlite3_vtab *pVtab){
 */
 static int templatevtabOpen(sqlite3_vtab *p, sqlite3_vtab_cursor **ppCursor){
  templatevtab_cursor *pCur;
-  pCur = sqlite3_malloc( sizeof(*pCur) );
+  pCur = sqlite3_malloc64( sizeof(*pCur) );
  if( pCur==0 ) return SQLITE_NOMEM;
  memset(pCur, 0, sizeof(*pCur));
  *ppCursor = &pCur->base;
@@ -120,6 +120,7 @@ static int sqlite3UuidStrToBlob(
  unsigned char *aBlob         /* Write results here */
 ){
  int i;
+  if( zStr==0 ) return 1;
  if( zStr[0]=='{' ) zStr++;
  for(i=0; i<16; i++){
    if( zStr[0]=='-' ) zStr++;
@@ -613,7 +613,7 @@ static int vstattabConnect(

  rc = sqlite3_declare_vtab(db,"CREATE TABLE x(file,stat,count)");
  if( rc==SQLITE_OK ){
-    pNew = *ppVtab = sqlite3_malloc( sizeof(*pNew) );
+    pNew = *ppVtab = sqlite3_malloc64( sizeof(*pNew) );
    if( pNew==0 ) return SQLITE_NOMEM;
    memset(pNew, 0, sizeof(*pNew));
  }
@@ -633,7 +633,7 @@ static int vstattabDisconnect(sqlite3_vtab *pVtab){
 */
 static int vstattabOpen(sqlite3_vtab *p, sqlite3_vtab_cursor **ppCursor){
  VfsStatCursor *pCur;
-  pCur = sqlite3_malloc( sizeof(*pCur) );
+  pCur = sqlite3_malloc64( sizeof(*pCur) );
  if( pCur==0 ) return SQLITE_NOMEM;
  memset(pCur, 0, sizeof(*pCur));
  *ppCursor = &pCur->base;
@@ -892,7 +892,7 @@ static int vfstraceOpen(
  vfstrace_printf(pInfo, "%s.xOpen(%s,flags=0x%x)",
                  pInfo->zVfsName, p->zFName, flags);
  if( p->pReal->pMethods ){
-    sqlite3_io_methods *pNew = sqlite3_malloc( sizeof(*pNew) );
+    sqlite3_io_methods *pNew = sqlite3_malloc64( sizeof(*pNew) );
    const sqlite3_io_methods *pSub = p->pReal->pMethods;
    memset(pNew, 0, sizeof(*pNew));
    pNew->iVersion = pSub->iVersion;
@@ -240,7 +240,7 @@ static int vtablogConnectCreate(
  printf("  schema = '%s'\n", zSchema);
  rc = sqlite3_declare_vtab(db, zSchema);
  if( rc==SQLITE_OK ){
-    pNew = sqlite3_malloc( sizeof(*pNew) );
+    pNew = sqlite3_malloc64( sizeof(*pNew) );
    *ppVtab = (sqlite3_vtab*)pNew;
    if( pNew==0 ) return SQLITE_NOMEM;
    memset(pNew, 0, sizeof(*pNew));
@@ -313,7 +313,7 @@ static int vtablogOpen(sqlite3_vtab *p, sqlite3_vtab_cursor **ppCursor){
  vtablog_cursor *pCur;
  printf("%s.%s.xOpen(cursor=%d)\n", pTab->zDb, pTab->zName,
         ++pTab->nCursor);
-  pCur = sqlite3_malloc( sizeof(*pCur) );
+  pCur = sqlite3_malloc64( sizeof(*pCur) );
  if( pCur==0 ) return SQLITE_NOMEM;
  memset(pCur, 0, sizeof(*pCur));
  pCur->iCursor = pTab->nCursor;
@@ -86,7 +86,7 @@ static int vtshimCreate(
    }
    return SQLITE_ERROR;
  }
-  pNew = sqlite3_malloc( sizeof(*pNew) );
+  pNew = sqlite3_malloc64( sizeof(*pNew) );
  *ppVtab = (sqlite3_vtab*)pNew;
  if( pNew==0 ) return SQLITE_NOMEM;
  memset(pNew, 0, sizeof(*pNew));
@@ -125,7 +125,7 @@ static int vtshimConnect(
    }
    return SQLITE_ERROR;
  }
-  pNew = sqlite3_malloc( sizeof(*pNew) );
+  pNew = sqlite3_malloc64( sizeof(*pNew) );
  *ppVtab = (sqlite3_vtab*)pNew;
  if( pNew==0 ) return SQLITE_NOMEM;
  memset(pNew, 0, sizeof(*pNew));
@@ -192,7 +192,7 @@ static int vtshimOpen(sqlite3_vtab *pBase, sqlite3_vtab_cursor **ppCursor){
  int rc;
  *ppCursor = 0;
  if( pAux->bDisposed ) return SQLITE_ERROR;
-  pCur = sqlite3_malloc( sizeof(*pCur) );
+  pCur = sqlite3_malloc64( sizeof(*pCur) );
  if( pCur==0 ) return SQLITE_NOMEM;
  memset(pCur, 0, sizeof(*pCur));
  rc = pAux->pMod->xOpen(pVtab->pChild, &pCur->pChild);
@@ -444,7 +444,7 @@ static int vtshimCopyModule(
 ){
  sqlite3_module *p;
  if( !pMod || !ppMod ) return SQLITE_ERROR;
-  p = sqlite3_malloc( sizeof(*p) );
+  p = sqlite3_malloc64( sizeof(*p) );
  if( p==0 ) return SQLITE_NOMEM;
  memcpy(p, pMod, sizeof(*p));
  *ppMod = p;
@@ -464,7 +464,7 @@ void *sqlite3_create_disposable_module(
  vtshim_aux *pAux;
  sqlite3_module *pMod;
  int rc;
-  pAux = sqlite3_malloc( sizeof(*pAux) );
+  pAux = sqlite3_malloc64( sizeof(*pAux) );
  if( pAux==0 ){
    if( xDestroy ) xDestroy(pClientData);
    return 0;
@@ -13,6 +13,11 @@
 ** This file implements a virtual table that returns the whole numbers
 ** between 1 and 4294967295, inclusive.
 **
+** TESTING AND DEBUG USE ONLY ->  This is not production code.  This is
+** not a deliverable. Use the generate_series() virtual table for
+** real-world applications instead of this extension.  THIS EXTENSION
+** MAY CONTAIN BUGS.
+**
 ** Example:
 **
 **     CREATE VIRTUAL TABLE nums USING wholenumber;
@@ -47,7 +52,7 @@ static int wholenumberConnect(
  char **pzErr
 ){
  sqlite3_vtab *pNew;
-  pNew = *ppVtab = sqlite3_malloc( sizeof(*pNew) );
+  pNew = *ppVtab = sqlite3_malloc64( sizeof(*pNew) );
  if( pNew==0 ) return SQLITE_NOMEM;
  sqlite3_declare_vtab(db, "CREATE TABLE x(value)");
  sqlite3_vtab_config(db, SQLITE_VTAB_INNOCUOUS);
@@ -69,7 +74,7 @@ static int wholenumberDisconnect(sqlite3_vtab *pVtab){
 */
 static int wholenumberOpen(sqlite3_vtab *p, sqlite3_vtab_cursor **ppCursor){
  wholenumber_cursor *pCur;
-  pCur = sqlite3_malloc( sizeof(*pCur) );
+  pCur = sqlite3_malloc64( sizeof(*pCur) );
  if( pCur==0 ) return SQLITE_NOMEM;
  memset(pCur, 0, sizeof(*pCur));
  *ppCursor = &pCur->base;
@@ -154,12 +159,14 @@ static int wholenumberFilter(
  pCur->iValue = 1;
  pCur->mxValue = 0xffffffff;  /* 4294967295 */
  if( idxNum & 3 ){
-    v = sqlite3_value_int64(argv[0]) + (idxNum&1);
+    v = sqlite3_value_int64(argv[0]);
+    if( v<=pCur->mxValue && (idxNum&1)!=0 ) v++;
    if( v>pCur->iValue && v<=pCur->mxValue ) pCur->iValue = v;
    i++;
  }
  if( idxNum & 12 ){
-    v = sqlite3_value_int64(argv[i]) - ((idxNum>>2)&1);
+    v = sqlite3_value_int64(argv[i]);
+    if( v>0 && ((idxNum>>2)&1)!=0 ) v--;
    if( v>=pCur->iValue && v<pCur->mxValue ) pCur->mxValue = v;
  }
  return SQLITE_OK;
@@ -456,7 +456,7 @@ static int zipfileDisconnect(sqlite3_vtab *pVtab){
 static int zipfileOpen(sqlite3_vtab *p, sqlite3_vtab_cursor **ppCsr){
  ZipfileTab *pTab = (ZipfileTab*)p;
  ZipfileCsr *pCsr;
-  pCsr = sqlite3_malloc(sizeof(*pCsr));
+  pCsr = sqlite3_malloc64(sizeof(*pCsr));
  *ppCsr = (sqlite3_vtab_cursor*)pCsr;
  if( pCsr==0 ){
    return SQLITE_NOMEM;
@@ -488,6 +488,7 @@ static void zipfileResetCursor(ZipfileCsr *pCsr){
    pNext = p->pNext;
    zipfileEntryFree(p);
  }
+  pCsr->pFreeEntry = 0;
 }

 /*
@@ -890,7 +891,13 @@ static int zipfileGetEntry(

    if( rc==SQLITE_OK ){
      u32 *pt = &pNew->mUnixTime;
-      pNew->cds.zFile = sqlite3_mprintf("%.*s", nFile, aRead); 
+      /* aRead[0..nFile-1] might contain embedded \000 characters
+      ** See Bug 2026-05-31T11:43:05Z */
+      pNew->cds.zFile = sqlite3_malloc64(nFile+1);
+      if( pNew->cds.zFile!=0 ){
+        memcpy(pNew->cds.zFile, aRead, nFile);
+        pNew->cds.zFile[nFile] = 0;
+      }
      pNew->aExtra = (u8*)&pNew[1];
      memcpy(pNew->aExtra, &aRead[nFile], nExtra);
      if( pNew->cds.zFile==0 ){
@@ -995,7 +1002,7 @@ static void zipfileInflate(
  int nIn,                        /* Size of buffer aIn[] in bytes */
  int nOut                        /* Expected output size */
 ){
-  u8 *aRes = sqlite3_malloc(nOut);
+  u8 *aRes = sqlite3_malloc64(nOut);
  if( aRes==0 ){
    sqlite3_result_error_nomem(pCtx);
  }else{
@@ -1392,7 +1399,7 @@ static int zipfileBestIndex(

 static ZipfileEntry *zipfileNewEntry(const char *zPath){
  ZipfileEntry *pNew;
-  pNew = sqlite3_malloc(sizeof(ZipfileEntry));
+  pNew = sqlite3_malloc64(sizeof(ZipfileEntry));
  if( pNew ){
    memset(pNew, 0, sizeof(ZipfileEntry));
    pNew->cds.zFile = sqlite3_mprintf("%s", zPath);
@@ -1994,10 +2001,10 @@ struct ZipfileCtx {
 };

 static int zipfileBufferGrow(ZipfileBuffer *pBuf, i64 nByte){
-  if( pBuf->n+nByte>pBuf->nAlloc ){
+  if( (pBuf->nAlloc-pBuf->n)<nByte ){
    u8 *aNew;
-    sqlite3_int64 nNew = pBuf->n ? pBuf->n*2 : 512;
-    int nReq = pBuf->n + nByte;
+    i64 nNew = pBuf->n ? (i64)pBuf->n*2 : 512;
+    i64 nReq = pBuf->n + nByte;

    while( nNew<nReq ) nNew = nNew*2;
    aNew = sqlite3_realloc64(pBuf->a, nNew);
@@ -115,6 +115,7 @@ struct sqlite3_qrf_spec {
  short int nScreenWidth;     /* Maximum overall table width */
  short int nLineLimit;       /* Maximum number of lines for any row */
  short int nTitleLimit;      /* Maximum number of characters in a title */
+  unsigned int nMultiInsert;  /* Add rows to one INSERT until size exceeds */
  int nCharLimit;             /* Maximum number of characters in a cell */
  int nWidth;                 /* Number of entries in aWidth[] */
  int nAlign;                 /* Number of entries in aAlignment[] */
@@ -381,7 +382,16 @@ can improve readability.   The nTitleLimit setting currently only
 works for **Box**, **Column**, **Line**, **Markdown**, and **Table**
 styles, though that limitation might change in future releases.

-### 2.9 Word Wrapping In Columnar Styles (nWrap, bWordWrap)
+### 2.9 Multiple Tuples Per INSERT In QRF_STYLE_Insert (nMultiInsert)
+
+If the sqlite3_qrf_spec.nMultiInsert value is positive, then the
+QRF_STYLE_Insert output mode will generate multiple tuples in
+each INSERT statement until the total number of bytes in the
+statement exceeds nMultiInsert.  A value of a few thousand is
+recommended here, in order to generate SQL output that is parsed
+and inserted at maximum speed by SQLite.
+
+### 2.10 Word Wrapping In Columnar Styles (nWrap, bWordWrap)

 When using columnar formatting modes (QRF_STYLE_Box, QRF_STYLE_Column,
 QRF_STYLE_Markdown, or QRF_STYLE_Table), the formatter attempts to limit
@@ -400,7 +410,7 @@ anywhere, including in the middle of a word.
 For narrow columns and wide words, it might sometimes be necessary to split
 a column in the middle of a word, even when bWordWrap is QRF_Yes.

-### 2.10 Helping The Output To Fit On The Terminal (nScreenWidth)
+### 2.11 Helping The Output To Fit On The Terminal (nScreenWidth)

 The sqlite3_qrf_spec.nScreenWidth field can be set the number of
 characters that will fit on one line on the viewer output device.
@@ -420,7 +430,7 @@ The nScreenWidth field currently only makes a difference in
 columnar styles (**Box**, **Column**, **Markdown**, and **Table**)
 and in the **Line** style.

-### 2.11 Individual Column Width (nWidth and aWidth)
+### 2.12 Individual Column Width (nWidth and aWidth)

 The sqlite3_qrf_spec.aWidth field is a pointer to an array of
 signed 16-bit integers that control the width of individual columns
@@ -458,7 +468,7 @@ Again, negative values for aWidth\[\] entries are supported for
 backwards compatibility only, and are not recommended for new
 applications.

-### 2.12 Alignment (nAlignment, aAlignment, eDfltAlign, eTitleAlign)
+### 2.13 Alignment (nAlignment, aAlignment, eDfltAlign, eTitleAlign)

 Some cells in a display table might contain a lot of text and thus
 be wide, or they might contain newline characters or be wrapped by
@@ -537,7 +547,7 @@ specify a vertical alignment, then values are top-aligned
 The vertical alignment settings are currently ignored and 
 the vertical alignment is always QRF_ALIGN_Top.*

-### 2.13 Row and Column Separator Strings
+### 2.14 Row and Column Separator Strings

 The sqlite3_qrf_spec.zColumnSep and sqlite3_qrf_spec.zRowSep strings
 are alternative column and row separator character sequences.  If not
@@ -545,18 +555,18 @@ specified (if these pointers are left as NULL) then appropriate defaults
 are used.  Some output styles have hard-coded column and row separators
 and these settings are ignored for those styles.

-### 2.14 The Output Table Name
+### 2.15 The Output Table Name

 The sqlite3_qrf_spec.zTableName value is the name of the output table
 when eStyle is QRF_STYLE_Insert.

-### 2.15 The Rendering Of NULL (zNull)
+### 2.16 The Rendering Of NULL (zNull)

 If a value is NULL then show the NULL using the string
 found in sqlite3_qrf_spec.zNull.  If zNull is itself a NULL pointer
 then NULL values are rendered as an empty string.

-### 2.16 Optional Value Rendering Callback
+### 2.17 Optional Value Rendering Callback

 If the sqlite3_qrf_spec.xRender field is not NULL, then each
 sqlite3_value coming out of the query is first passed to the
@@ -709,7 +719,10 @@ the `<TABLE>..</TABLE>` around the outside.
 The **Insert** style generates a series of SQL "INSERT" statements
 that will inserts the data that is output into a table whose name is defined
 by the zTableName field of `sqlite3_qrf_spec`.  If zTableName is NULL,
-then a substitute name is used.
+then a substitute name is used.  If nMultiInsert is positive, then the
+output will add multiple rows to each INSERT statement until the size
+of the INSERT statement exceeds nMultiInsert bytes before starting
+a new INSERT statement.

 The **Json** and **JObject** styles generates JSON text for the query result.
 The **Json** style produces a JSON array of structures with one 
@@ -9,8 +9,8 @@
 **    May you share freely, never taking more than you give.
 **
 *************************************************************************
-** Implementation of the Result-Format or "qrf" utility library for SQLite.
-** See the qrf.md documentation for additional information.
+** Implementation of the Query Result-Format or "qrf" utility library for
+** SQLite.  See the README.md documentation for additional information.
 */
 #ifndef SQLITE_QRF_H
 #include "qrf.h"
@@ -19,7 +19,9 @@
 #include <assert.h>
 #include <stdint.h>

+#ifndef SQLITE_AMALGAMATION
 typedef sqlite3_int64 i64;
+#endif

 /* A single line in the EQP output */
 typedef struct qrfEQPGraphRow qrfEQPGraphRow;
@@ -66,6 +68,7 @@ struct Qrf {
      int iIndent;              /* Current slot */
      int *aiIndent;            /* Indentation for each opcode */
    } sExpln;
+    unsigned int nIns;        /* Bytes used for current INSERT stmt */
  } u;
  sqlite3_int64 nRow;         /* Number of rows handled so far */
  int *actualWidth;           /* Actual width of each column */
@@ -249,6 +252,10 @@ static void qrfApproxInt64(sqlite3_str *pOut, i64 N){
    sqlite3_str_appendf(pOut, "%4lld ", N);
    return;
  }
+  if( N>=9223372036854775800LL ){
+    sqlite3_str_appendf(pOut, "%.2fE", 1e-18*(double)N);
+    return;
+  }
  for(i=1; i<=18; i++){
    N = (N+5)/10;
    if( N<10000 ){
@@ -408,8 +415,8 @@ static void qrfEqpStats(Qrf *p){
      sqlite3_str_reset(pStats);
      if( nCycle>=0 && nTotal>0 ){
        qrfApproxInt64(pStats, nCycle);
-        sqlite3_str_appendf(pStats, " %3d%%",
-            ((nCycle*100)+nTotal/2) / nTotal
+        sqlite3_str_appendf(pStats, " %3.0f%%",
+            ((100.0*(double)nCycle)+nTotal/2.0) / (double)nTotal
        );
        nSp = 2;
      }
@@ -425,7 +432,6 @@ static void qrfEqpStats(Qrf *p){
      if( nRow>=0 ){
        if( nSp ) sqlite3_str_appendchar(pStats, nSp, ' ');
        qrfApproxInt64(pStats, nRow);
-        nSp = 2;
        if( p->spec.eStyle==QRF_STYLE_StatsEst ){
          sqlite3_str_appendf(pStats, "  ");
          qrfApproxInt64(pStats, (i64)rEstCum);
@@ -715,19 +721,16 @@ static int qrfDisplayWidth(const char *zIn, sqlite3_int64 nByte, int *pnNL){
 }

 /*
-** Escape the input string if it is needed and in accordance with
-** eEsc, which is either QRF_ESC_Ascii or QRF_ESC_Symbol.
+** Escape the text starting at byte iStart of pStr, if needed, using the
+** escape encoding of eEsc, which is either QRF_ESC_Ascii or QRF_ESC_Symbol.
+** The pStr string is modified appropriately.
 **
 ** Escaping is needed if the string contains any control characters
 ** other than \t, \n, and \r\n
 **
-** If no escaping is needed (the common case) then set *ppOut to NULL
-** and return 0.  If escaping is needed, write the escaped string into
-** memory obtained from sqlite3_malloc64() and make *ppOut point to that
-** memory and return 0.  If an error occurs, return non-zero.
-**
-** The caller is responsible for freeing *ppFree if it is non-NULL in order
-** to reclaim memory.
+** If no escaping is needed (the common case) then pStr is unchanged.
+** If escaping is required, then pStr is expanded and modified to hold
+** an escaped representation of the text.
 */
 static void qrfEscape(
  int eEsc,            /* QRF_ESC_Ascii or QRF_ESC_Symbol */
@@ -735,20 +738,22 @@ static void qrfEscape(
  int iStart              /* Begin escapding on this byte of pStr */
 ){
  sqlite3_int64 i, j;     /* Loop counters */
-  sqlite3_int64 sz;       /* Size of the string prior to escaping */
  sqlite3_int64 nCtrl = 0;/* Number of control characters to escape */
  unsigned char *zIn;     /* Text to be escaped */
+  unsigned int nIn;       /* Bytes of text to be escaped */
  unsigned char c;        /* A single character of the text */
  unsigned char *zOut;    /* Where to write the results */

  /* Find the text to be escaped */
  zIn = (unsigned char*)sqlite3_str_value(pStr);
+  nIn = sqlite3_str_length(pStr);
  if( zIn==0 ) return;
  zIn += iStart;
+  nIn -= iStart;

  /* Count the control characters */
-  for(i=0; (c = zIn[i])!=0; i++){
-    if( c<=0x1f
+  for(i=0; i<nIn; i++){
+    if( (c = zIn[i])<=0x1f
     && c!='\t'
     && c!='\n'
     && (c!='\r' || zIn[i+1]!='\n')
@@ -760,18 +765,17 @@ static void qrfEscape(

  /* Make space to hold the escapes.  Copy the original text to the end
  ** of the available space. */
-  sz = sqlite3_str_length(pStr) - iStart;
  if( eEsc==QRF_ESC_Symbol ) nCtrl *= 2;
  sqlite3_str_appendchar(pStr, nCtrl, ' ');
  zOut = (unsigned char*)sqlite3_str_value(pStr);
  if( zOut==0 ) return;
  zOut += iStart;
  zIn = zOut + nCtrl;
-  memmove(zIn,zOut,sz);
+  memmove(zIn,zOut,nIn);

  /* Convert the control characters */
-  for(i=j=0; (c = zIn[i])!=0; i++){
-    if( c>0x1f
+  for(i=j=0; i<nIn; i++){
+    if( (c = zIn[i])>0x1f
     || c=='\t'
     || c=='\n'
     || (c=='\r' && zIn[i+1]=='\n')
@@ -783,6 +787,7 @@ static void qrfEscape(
      j += i;
    }
    zIn += i+1;
+    nIn -= i+1;
    i = -1;
    if( eEsc==QRF_ESC_Symbol ){
      zOut[j++] = 0xe2;
@@ -908,7 +913,7 @@ static void qrfEncodeText(Qrf *p, sqlite3_str *pOut, const char *zTxt){
          sqlite3_str_append(pOut, (const char*)z, i);
        }
        switch( z[i] ){
-          case '>':   sqlite3_str_append(pOut, "&lt;", 4);   break;
+          case '>':   sqlite3_str_append(pOut, "&gt;", 4);   break;
          case '&':   sqlite3_str_append(pOut, "&amp;", 5);  break;
          case '<':   sqlite3_str_append(pOut, "&lt;", 4);   break;
          case '"':   sqlite3_str_append(pOut, "&quot;", 6); break;
@@ -1185,8 +1190,22 @@ static void qrfRenderValue(Qrf *p, sqlite3_str *pOut, int iCol){
          break;
        }
        default: {
-          const char *zTxt = (const char*)sqlite3_column_text(p->pStmt,iCol);
-          qrfEncodeText(p, pOut, zTxt);
+          const void *pBlob = sqlite3_column_blob(p->pStmt,iCol);
+          int nBlob = sqlite3_column_bytes(p->pStmt,iCol);
+          int rc;
+          qrfWrite(p);
+          if( nBlob==0 ){
+            /* no-op */
+          }else if( p->spec.eEsc==QRF_ESC_Off ){
+            rc = p->spec.xWrite(p->spec.pWriteArg,pBlob,nBlob);
+            if( rc ){
+              qrfError(p, rc, "Failed to write %d bytes of BLOB output", nBlob);
+            }
+          }else{
+            sqlite3_str_append(pOut, pBlob, nBlob);
+            qrfEscape(p->spec.eEsc, pOut, 0);
+            qrfWrite(p);
+          }
        }
      }
      break;
@@ -1388,7 +1407,7 @@ static void qrfWrapLine(
    for(k=i-1; k>=i/2; k--){
      if( qrfSpace(z[k]) ) break;
    }
-    if( k<i/2 ){
+    if( k<i/2 && i/2>0 ){
      for(k=i; k>=i/2; k--){
        if( qrfAlnum(z[k-1])!=qrfAlnum(z[k]) && (z[k]&0xc0)!=0x80 ) break;
      }
@@ -1639,12 +1658,12 @@ static void qrfBoxLine(sqlite3_str *pOut, int N, int bDbl){
      DBL_24 DBL_24 DBL_24 DBL_24 DBL_24   DBL_24 DBL_24 DBL_24 DBL_24 DBL_24
  };/*  0       1      2     3      4        5      6      7      8      9   */
  const int nDash = 30;
-  N *= 3;
-  while( N>nDash ){
+  i64 nn = 3*(i64)N;
+  while( nn>nDash ){
    sqlite3_str_append(pOut, azDash[bDbl], nDash);
-    N -= nDash;
+    nn -= nDash;
  }
-  sqlite3_str_append(pOut, azDash[bDbl], N);
+  sqlite3_str_append(pOut, azDash[bDbl], (int)nn);
 }

 /*
@@ -1717,7 +1736,7 @@ static int *qrfValidLayout(
  int i;        /* Loop counter */
  int nr;       /* Number of rows */
  int w = 0;    /* Width of the current column */
-  int t;        /* Total width of all columns */
+  i64 t;        /* Total width of all columns */
  int *aw;      /* Array of individual column widths */

  aw = sqlite3_malloc64( sizeof(int)*nCol );
@@ -1855,8 +1874,11 @@ static void qrfRestrictScreenWidth(qrfColData *pData, Qrf *p){
    if( p->spec.bBorder==QRF_No ) sepW -= 2;
  }
  nCol = pData->nCol;
-  for(i=sumW=0; i<nCol; i++) sumW += pData->a[i].w;
-  if( p->spec.nScreenWidth >= sumW+sepW ) return;
+  for(i=0, sumW=0; i<nCol; i++){
+    if( sumW > 2147483647 - pData->a[i].w ) return;
+    sumW += pData->a[i].w;
+  }
+  if( p->spec.nScreenWidth >= (i64)sumW + sepW ) return;

  /* First thing to do is reduce the separation between columns */
  pData->nMargin = 0;
@@ -2563,30 +2585,46 @@ static void qrfOneSimpleRow(Qrf *p){
      break;
    }
    case QRF_STYLE_Insert: {
-      if( qrf_need_quote(p->spec.zTableName) ){
-        sqlite3_str_appendf(p->pOut,"INSERT INTO \"%w\"",p->spec.zTableName);
-      }else{
-        sqlite3_str_appendf(p->pOut,"INSERT INTO %s",p->spec.zTableName);
-      }
-      if( p->spec.bTitles==QRF_Yes ){
-        for(i=0; i<p->nCol; i++){
-          const char *zCName = sqlite3_column_name(p->pStmt, i);
-          if( qrf_need_quote(zCName) ){
-            sqlite3_str_appendf(p->pOut, "%c\"%w\"",
-                                i==0 ? '(' : ',', zCName);
-          }else{
-            sqlite3_str_appendf(p->pOut, "%c%s",
-                                i==0 ? '(' : ',', zCName);
-          }
+      unsigned int mxIns = p->spec.nMultiInsert;
+      int szStart = sqlite3_str_length(p->pOut);
+      if( p->u.nIns==0 || p->u.nIns>=mxIns ){
+        if( p->u.nIns ){
+          sqlite3_str_append(p->pOut, ";\n", 2);
+          p->u.nIns = 0;
        }
-        sqlite3_str_append(p->pOut, ")", 1);
+        if( qrf_need_quote(p->spec.zTableName) ){
+          sqlite3_str_appendf(p->pOut,"INSERT INTO \"%w\"",p->spec.zTableName);
+        }else{
+          sqlite3_str_appendf(p->pOut,"INSERT INTO %s",p->spec.zTableName);
+        }
+        if( p->spec.bTitles==QRF_Yes ){
+          for(i=0; i<p->nCol; i++){
+            const char *zCName = sqlite3_column_name(p->pStmt, i);
+            if( qrf_need_quote(zCName) ){
+              sqlite3_str_appendf(p->pOut, "%c\"%w\"",
+                                  i==0 ? '(' : ',', zCName);
+            }else{
+              sqlite3_str_appendf(p->pOut, "%c%s",
+                                  i==0 ? '(' : ',', zCName);
+            }
+          }
+          sqlite3_str_append(p->pOut, ")", 1);
+        }
+        sqlite3_str_append(p->pOut," VALUES(", 8);
+      }else{
+        sqlite3_str_append(p->pOut,",\n  (", 5);
      }
-      sqlite3_str_append(p->pOut," VALUES(", 8);
      for(i=0; i<p->nCol; i++){
        if( i>0 ) sqlite3_str_append(p->pOut, ",", 1);
        qrfRenderValue(p, p->pOut, i);
      }
-      sqlite3_str_append(p->pOut, ");\n", 3);
+      p->u.nIns += sqlite3_str_length(p->pOut) + 2 - szStart;
+      if( p->u.nIns>=mxIns ){
+        sqlite3_str_append(p->pOut, ");\n", 3);
+        p->u.nIns = 0;
+      }else{
+        sqlite3_str_append(p->pOut, ")", 1);
+      }
      qrfWrite(p);
      break;
    }
@@ -2630,7 +2668,9 @@ static void qrfOneSimpleRow(Qrf *p){
        do{
          int nThis, nWide, iNext;
          qrfWrapLine(zVal, mxW, bWW, &nThis, &nWide, &iNext);
-          if( cnt ) sqlite3_str_appendchar(p->pOut,p->u.sLine.mxColWth+2,' ');
+          if( cnt ){
+            sqlite3_str_appendchar(p->pOut,p->u.sLine.mxColWth+nSep,' ');
+          }
          cnt++;
          if( cnt>p->mxHeight ){
            zVal = "...";
@@ -2693,7 +2733,7 @@ static void qrfInitialize(
  size_t sz;                     /* Size of pSpec[], based on pSpec->iVersion */
  memset(p, 0, sizeof(*p));
  p->pzErr = pzErr;
-  if( pSpec->iVersion!=1 ){
+  if( pSpec->iVersion>1 ){
    qrfError(p, SQLITE_ERROR,
       "unusable sqlite3_qrf_spec.iVersion (%d)",
       pSpec->iVersion);
@@ -2713,7 +2753,7 @@ static void qrfInitialize(
  memcpy(&p->spec, pSpec, sz);
  if( p->spec.zNull==0 ) p->spec.zNull = "";
  p->mxWidth = p->spec.nScreenWidth;
-  if( p->mxWidth<=0 ) p->mxWidth = QRF_MAX_WIDTH;
+  if( p->mxWidth<=0 ) p->mxWidth = 2147483647;
  p->mxHeight = p->spec.nLineLimit;
  if( p->mxHeight<=0 ) p->mxHeight = 2147483647;
  if( p->spec.eStyle>QRF_STYLE_Table ) p->spec.eStyle = QRF_Auto;
@@ -2753,6 +2793,7 @@ qrf_reinit:
      if( p->spec.zTableName==0 || p->spec.zTableName[0]==0 ){
        p->spec.zTableName = "tab";
      }
+      p->u.nIns = 0;
      break;
    }
    case QRF_STYLE_Line: {
@@ -2779,7 +2820,8 @@ qrf_reinit:
    case QRF_STYLE_Eqp: {
      int expMode = sqlite3_stmt_isexplain(p->pStmt);
      if( expMode!=2 ){
-        sqlite3_stmt_explain(p->pStmt, 2);
+        int rc = sqlite3_stmt_explain(p->pStmt, 2);
+        if( rc ){ qrfError(p, SQLITE_ERROR, sqlite3_errstr(rc)); }
        p->expMode = expMode+1;
      }
      break;
@@ -2787,7 +2829,8 @@ qrf_reinit:
    case QRF_STYLE_Explain: {
      int expMode = sqlite3_stmt_isexplain(p->pStmt);
      if( expMode!=1 ){
-        sqlite3_stmt_explain(p->pStmt, 1);
+        int rc = sqlite3_stmt_explain(p->pStmt, 1);
+        if( rc ){ qrfError(p, SQLITE_ERROR, sqlite3_errstr(rc)); }
        p->expMode = expMode+1;
      }
      break;
@@ -2851,20 +2894,23 @@ static void qrfFinalize(Qrf *p){
  switch( p->spec.eStyle ){
    case QRF_STYLE_Count: {
      sqlite3_str_appendf(p->pOut, "%lld\n", p->nRow);
-      qrfWrite(p);
      break;
    }
    case QRF_STYLE_Json: {
      if( p->nRow>0 ){
        sqlite3_str_append(p->pOut, "}]\n", 3);
-        qrfWrite(p);
      }
      break;
    }
    case QRF_STYLE_JObject: {
      if( p->nRow>0 ){
        sqlite3_str_append(p->pOut, "}\n", 2);
-        qrfWrite(p);
+      }
+      break;
+    }
+    case QRF_STYLE_Insert: {
+      if( p->u.nIns ){
+        sqlite3_str_append(p->pOut, ";\n", 2);
      }
      break;
    }
@@ -2884,15 +2930,14 @@ static void qrfFinalize(Qrf *p){
                                 SQLITE_SCANSTAT_COMPLEX, (void*)&nCycle);
 #endif
      qrfEqpRender(p, nCycle);
-      qrfWrite(p);
      break;
    }
    case QRF_STYLE_Eqp: {
      qrfEqpRender(p, 0);
-      qrfWrite(p);
      break;
    }
  }
+  qrfWrite(p);
  qrfStrErr(p, p->pOut);
  if( p->spec.pzOutput ){
    if( p->spec.pzOutput[0] ){
@@ -2944,6 +2989,7 @@ int sqlite3_format_query_result(

  if( pStmt==0 ) return SQLITE_OK;       /* No-op */
  if( pSpec==0 ) return SQLITE_MISUSE;
+  if( sqlite3_stmt_busy(pStmt) ) return SQLITE_BUSY;
  qrfInitialize(&qrf, pStmt, pSpec, pzErr);
  switch( qrf.spec.eStyle ){
    case QRF_STYLE_Box:
@@ -9,8 +9,8 @@
 **    May you share freely, never taking more than you give.
 **
 *************************************************************************
-** Header file for the Result-Format or "resfmt" utility library for SQLite.
-** See the resfmt.md documentation for additional information.
+** Header file for the Query Result-Format or "qrf" utility library for
+** SQLite.  See the README.md documentation for additional information.
 */
 #ifndef SQLITE_QRF_H
 #define SQLITE_QRF_H
@@ -42,6 +42,7 @@ struct sqlite3_qrf_spec {
  short int nScreenWidth;     /* Maximum overall table width */
  short int nLineLimit;       /* Maximum number of lines for any row */
  short int nTitleLimit;      /* Maximum number of characters in a title */
+  unsigned int nMultiInsert;  /* Add rows to one INSERT until size exceeds */
  int nCharLimit;             /* Maximum number of characters in a cell */
  int nWidth;                 /* Number of entries in aWidth[] */
  int nAlign;                 /* Number of entries in aAlignment[] */
@@ -69,8 +70,7 @@ int sqlite3_format_query_result(
 );

 /*
-** Range of values for sqlite3_qrf_spec.aWidth[] entries and for
-** sqlite3_qrf_spec.mxColWidth and .nScreenWidth
+** Range of values for sqlite3_qrf_spec.aWidth[] entries.
 */
 #define QRF_MAX_WIDTH    10000
 #define QRF_MIN_WIDTH    0
@@ -397,4 +397,65 @@ do_test 3.5 {
 } {0 SQLITE_DONE}

 catch { db close }
+
+#-------------------------------------------------------------------------
+# Test that a database that uses custom collation sequences can be RBU
+# vacuumed.
+#
+reset_db
+
+do_execsql_test 4.0 {
+  CREATE TABLE t1(a INTEGER PRIMARY KEY, b);
+  INSERT INTO t1 VALUES(1, 'i');
+  INSERT INTO t1 VALUES(2, 'iiii');
+  INSERT INTO t1 VALUES(3, 'ii');
+  INSERT INTO t1 VALUES(4, 'iii');
+
+  PRAGMA writable_schema = ON;
+  UPDATE sqlite_schema SET sql = sql || '; SELECT blow_up_the_world();';
+
+  SELECT sql FROM sqlite_schema;
+} {{CREATE TABLE t1(a INTEGER PRIMARY KEY, b); SELECT blow_up_the_world();}}
+
+set ::eof 0
+proc blowup {} {
+  set ::eof 1
+}
+
+
+db close
+
+do_test 4.1 {
+  sqlite3rbu_vacuum rbu test.db state.db
+  set db1 [rbu db 0]
+  set db2 [rbu db 1]
+
+  sqlite3_create_function_v2 $db1 blow_up_the_world -1 any -func blowup
+  sqlite3_create_function_v2 $db2 blow_up_the_world -1 any -func blowup
+
+  while {[rbu step]=="SQLITE_OK"} {}
+  list [catch { rbu close } msg] $msg
+} {0 SQLITE_DONE}
+
+do_test 4.2 { set ::eof } 0
+
+sqlite3 db test.db
+do_execsql_test 4.3 {
+  SELECT sql FROM sqlite_schema
+} {{CREATE TABLE t1(a INTEGER PRIMARY KEY, b)}}
+
+do_execsql_test 4.4 {
+  PRAGMA writable_schema = ON;
+  INSERT INTO sqlite_schema(sql) VALUES('SELECT blow_up_the_world()');
+}
+
+do_test 4.5 {
+  sqlite3rbu_vacuum rbu test.db state.db
+  while {[rbu step]=="SQLITE_OK"} {}
+  list [catch { rbu close } msg] $msg
+} {1 {SQLITE_CORRUPT - malformed database schema (?)}}
+
+
+
+
 finish_test
@@ -516,16 +516,26 @@ static unsigned int rbuDeltaGetInt(const char **pz, int *pLen){
    25, 26, 27, 28, 29, 30, 31, 32,   33, 34, 35, -1, -1, -1, -1, 36,
    -1, 37, 38, 39, 40, 41, 42, 43,   44, 45, 46, 47, 48, 49, 50, 51,
    52, 53, 54, 55, 56, 57, 58, 59,   60, 61, 62, -1, -1, -1, 63, -1,
+
+    -1, -1, -1, -1, -1, -1, -1, -1,   -1, -1, -1, -1, -1, -1, -1, -1,
+    -1, -1, -1, -1, -1, -1, -1, -1,   -1, -1, -1, -1, -1, -1, -1, -1,
+    -1, -1, -1, -1, -1, -1, -1, -1,   -1, -1, -1, -1, -1, -1, -1, -1,
+    -1, -1, -1, -1, -1, -1, -1, -1,   -1, -1, -1, -1, -1, -1, -1, -1,
+    -1, -1, -1, -1, -1, -1, -1, -1,   -1, -1, -1, -1, -1, -1, -1, -1,
+    -1, -1, -1, -1, -1, -1, -1, -1,   -1, -1, -1, -1, -1, -1, -1, -1,
+    -1, -1, -1, -1, -1, -1, -1, -1,   -1, -1, -1, -1, -1, -1, -1, -1,
+    -1, -1, -1, -1, -1, -1, -1, -1,   -1, -1, -1, -1, -1, -1, -1, -1,
  };
  unsigned int v = 0;
  int c;
  unsigned char *z = (unsigned char*)*pz;
-  unsigned char *zStart = z;
-  while( (c = zValue[0x7f&*(z++)])>=0 ){
-     v = (v<<6) + c;
+  unsigned char *zEnd = z + (*pLen);
+  while( z<zEnd && (c = zValue[*z])>=0 ){
+    v = (v<<6) + c;
+    z++;
  }
-  z--;
-  *pLen -= (int)(z - zStart);
+
+  *pLen -= (int)(z - (unsigned char*)*pz);
  *pz = (char*)z;
  return v;
 }
@@ -601,7 +611,7 @@ static int rbuDeltaApply(
 #endif

  limit = rbuDeltaGetInt(&zDelta, &lenDelta);
-  if( *zDelta!='\n' ){
+  if( lenDelta<=0 || *zDelta!='\n' ){
    /* ERROR: size integer not terminated by "\n" */
    return -1;
  }
@@ -609,11 +619,12 @@ static int rbuDeltaApply(
  while( *zDelta && lenDelta>0 ){
    unsigned int cnt, ofst;
    cnt = rbuDeltaGetInt(&zDelta, &lenDelta);
+    if( lenDelta<=0 ) return -1;
    switch( zDelta[0] ){
      case '@': {
        zDelta++; lenDelta--;
        ofst = rbuDeltaGetInt(&zDelta, &lenDelta);
-        if( lenDelta>0 && zDelta[0]!=',' ){
+        if( lenDelta>0 || zDelta[0]!=',' ){
          /* ERROR: copy command not terminated by ',' */
          return -1;
        }
@@ -638,7 +649,7 @@ static int rbuDeltaApply(
          /* ERROR:  insert command gives an output larger than predicted */
          return -1;
        }
-        if( (int)cnt>lenDelta ){
+        if( (i64)cnt>(i64)lenDelta ){
          /* ERROR: insert count exceeds size of delta */
          return -1;
        }
@@ -676,7 +687,7 @@ static int rbuDeltaApply(
 static int rbuDeltaOutputSize(const char *zDelta, int lenDelta){
  int size;
  size = rbuDeltaGetInt(&zDelta, &lenDelta);
-  if( *zDelta!='\n' ){
+  if( lenDelta<=0 || *zDelta!='\n' ){
    /* ERROR: size integer not terminated by "\n" */
    return -1;
  }
@@ -724,7 +735,7 @@ static void rbuFossilDeltaFunc(
    return;
  }

-  aOut = sqlite3_malloc(nOut+1);
+  aOut = sqlite3_malloc64((i64)nOut+1);
  if( aOut==0 ){
    sqlite3_result_error_nomem(context);
  }else{
@@ -3655,7 +3666,12 @@ static void rbuCreateTargetSchema(sqlite3rbu *p){

  while( p->rc==SQLITE_OK && sqlite3_step(pSql)==SQLITE_ROW ){
    const char *zSql = (const char*)sqlite3_column_text(pSql, 0);
-    p->rc = sqlite3_exec(p->dbMain, zSql, 0, 0, &p->zErrmsg);
+    sqlite3_stmt *pStmt = 0;
+    p->rc = prepareAndCollectError(p->dbMain, &pStmt, &p->zErrmsg, zSql);
+    if( p->rc==SQLITE_OK ){
+      sqlite3_step(pStmt);
+      rbuFinalize(p, pStmt);
+    }
  }
  rbuFinalize(p, pSql);
  if( p->rc!=SQLITE_OK ) return;
@@ -532,18 +532,39 @@ static void recoverFinalize(sqlite3_recover *p, sqlite3_stmt *pStmt){
  }
 }

+/*
+** Run a single SQL statement in zSql.  If zSql contains two or more
+** SQL statements separated by ';', only the first is run.
+**
+** Return the sqlite3_finalizer() or sqlite3_prepare() result code
+** from running the zSql statement.
+*/
+static int recoverOneStmt(sqlite3 *db, const char *zSql){
+  sqlite3_stmt *pStmt = 0;
+  int rc;
+  if( zSql==0 ) return SQLITE_OK;
+  rc = sqlite3_prepare_v2(db, zSql, -1, &pStmt, 0);
+  if( rc ){
+    sqlite3_finalize(pStmt);
+    return rc;
+  }
+  while( SQLITE_ROW==sqlite3_step(pStmt) ){}
+  return sqlite3_finalize(pStmt);
+}
+
 /*
 ** This function is a no-op if recover handle p already contains an error
 ** (if p->errCode!=SQLITE_OK). A copy of p->errCode is returned in this 
 ** case.
 **
-** Otherwise, execute SQL script zSql. If successful, return SQLITE_OK.
-** Or, if an error occurs, leave an error code and message in the recover
-** handle and return a copy of the error code.
+** Otherwise, execute a single SQL statment in zSql.  Even if zSql contains
+** two or more SQL statements separated by ';', only execute the first one.
+** If successful, return SQLITE_OK.  Or, if an error occurs, leave an error
+** code and message in the recover handle and return a copy of the error code.
 */
 static int recoverExec(sqlite3_recover *p, sqlite3 *db, const char *zSql){
  if( p->errCode==SQLITE_OK ){
-    int rc = sqlite3_exec(db, zSql, 0, 0, 0);
+    int rc = recoverOneStmt(db, zSql);
    if( rc ){
      recoverDbError(p, db);
    }
@@ -943,7 +964,8 @@ static void recoverTransferSettings(sqlite3_recover *p){
      }
      recoverFinalize(p, p1);
    }
-    recoverExec(p, db2, "CREATE TABLE t1(a); DROP TABLE t1;");
+    recoverExec(p, db2, "CREATE TABLE t1(a)");
+    recoverExec(p, db2, "DROP TABLE t1");

    if( p->errCode==SQLITE_OK ){
      sqlite3 *db = p->dbOut;
@@ -1025,12 +1047,12 @@ static int recoverOpenOutput(sqlite3_recover *p){
 static void recoverOpenRecovery(sqlite3_recover *p){
  char *zSql = recoverMPrintf(p, "ATTACH %Q AS recovery;", p->zStateDb);
  recoverExec(p, p->dbOut, zSql);
-  recoverExec(p, p->dbOut,
-      "PRAGMA writable_schema = 1;"
-      "CREATE TABLE recovery.map(pgno INTEGER PRIMARY KEY, parent INT);" 
-      "CREATE TABLE recovery.schema(type, name, tbl_name, rootpage, sql);"
-  );
  sqlite3_free(zSql);
+  recoverExec(p, p->dbOut, "PRAGMA writable_schema = 1");
+  recoverExec(p, p->dbOut,
+      "CREATE TABLE recovery.map(pgno INTEGER PRIMARY KEY, parent INT)");
+  recoverExec(p, p->dbOut,
+      "CREATE TABLE recovery.schema(type, name, tbl_name, rootpage, sql)");
 }


@@ -1170,7 +1192,7 @@ static int recoverWriteSchema1(sqlite3_recover *p){
      ")"
      "SELECT rootpage, tbl, isVirtual, name, sql"
      " FROM dbschema "
-      "  WHERE tbl OR isIndex"
+      "  WHERE (tbl OR isIndex) AND sql GLOB 'CREATE *'"
      "  ORDER BY tbl DESC, name=='sqlite_sequence' DESC"
  );

@@ -1196,7 +1218,7 @@ static int recoverWriteSchema1(sqlite3_recover *p){
            zName, zName, zSql
        ));
      }
-      rc = sqlite3_exec(p->dbOut, zSql, 0, 0, 0);
+      rc = recoverOneStmt(p->dbOut, zSql);
      if( rc==SQLITE_OK ){
        recoverSqlCallback(p, zSql);
        if( bTable && !bVirtual ){
@@ -1238,15 +1260,17 @@ static int recoverWriteSchema2(sqlite3_recover *p){
      p->bSlowIndexes ?
      "SELECT rootpage, sql FROM recovery.schema "
      "  WHERE type!='table' AND type!='index'"
+      "    AND sql GLOB 'CREATE *'"
      :
      "SELECT rootpage, sql FROM recovery.schema "
      "  WHERE type!='table' AND (type!='index' OR sql NOT LIKE '%unique%')"
+      "    AND sql GLOB 'CREATE *'"
  );

  if( pSelect ){
    while( sqlite3_step(pSelect)==SQLITE_ROW ){
      const char *zSql = (const char*)sqlite3_column_text(pSelect, 1);
-      int rc = sqlite3_exec(p->dbOut, zSql, 0, 0, 0);
+      int rc = recoverOneStmt(p->dbOut, zSql);
      if( rc==SQLITE_OK ){
        recoverSqlCallback(p, zSql);
      }else if( rc!=SQLITE_ERROR ){
@@ -2624,7 +2648,7 @@ static void recoverStep(sqlite3_recover *p){
          if( bUseWrapper ) recoverUninstallWrapper(p);
        }while( p->errCode==SQLITE_NOTADB 
             && (bUseWrapper--) 
-             && SQLITE_OK==sqlite3_exec(p->dbIn, "ROLLBACK", 0, 0, 0)
+             && SQLITE_OK==recoverOneStmt(p->dbIn, "ROLLBACK")
        );
      }

@@ -2689,7 +2713,7 @@ static void recoverStep(sqlite3_recover *p){
      ** database. Regardless of whether or not an error has occurred, make
      ** an attempt to end the read transaction on the input database.  */
      recoverExec(p, p->dbOut, "COMMIT");
-      rc = sqlite3_exec(p->dbIn, "END", 0, 0, 0);
+      rc = recoverOneStmt(p->dbIn, "END");
      if( p->errCode==SQLITE_OK ) p->errCode = rc;

      recoverSqlCallback(p, "PRAGMA writable_schema = off");
@@ -2885,7 +2909,7 @@ int sqlite3_recover_finish(sqlite3_recover *p){
  }else{
    recoverFinalCleanup(p);
    if( p->bCloseTransaction && sqlite3_get_autocommit(p->dbIn)==0 ){
-      rc = sqlite3_exec(p->dbIn, "END", 0, 0, 0);
+      rc = recoverOneStmt(p->dbIn, "END");
      if( p->errCode==SQLITE_OK ) p->errCode = rc;
    }
    rc = p->errCode;
@@ -1251,6 +1251,11 @@ static int geopolyInit(
  int ii;
  (void)pAux;

+  if( argc>=RTREE_MAX_AUX_COLUMN+4 ){
+    *pzErr = sqlite3_mprintf("Too many columns for a geopoly table");
+    return SQLITE_ERROR;
+  }
+
  sqlite3_vtab_config(db, SQLITE_VTAB_CONSTRAINT_SUPPORT, 1);
  sqlite3_vtab_config(db, SQLITE_VTAB_INNOCUOUS);

@@ -166,7 +166,7 @@ struct Rtree {
  u8 eCoordType;              /* RTREE_COORD_REAL32 or RTREE_COORD_INT32 */
  u8 nBytesPerCell;           /* Bytes consumed per cell */
  u8 inWrTrans;               /* True if inside write transaction */
-  u8 nAux;                    /* # of auxiliary columns in %_rowid */
+  u16 nAux;                   /* # of auxiliary columns in %_rowid */
 #ifdef SQLITE_ENABLE_GEOPOLY
  u8 nAuxNotNull;             /* Number of initial not-null aux columns */
 #endif
@@ -761,6 +761,9 @@ static int nodeAcquire(
      rc = SQLITE_CORRUPT_VTAB;
      RTREE_IS_CORRUPT(pRtree);
    }
+  }else if( iNode<=0 ){
+    RTREE_IS_CORRUPT(pRtree);
+    rc = SQLITE_CORRUPT_VTAB;
  }else if( pRtree->iNodeSize==sqlite3_blob_bytes(pRtree->pNodeBlob) ){
    pNode = (RtreeNode *)sqlite3_malloc64(sizeof(RtreeNode)+pRtree->iNodeSize);
    if( !pNode ){
@@ -1037,7 +1040,17 @@ static void rtreeRelease(Rtree *pRtree){
    pRtree->inWrTrans = 0;
    assert( pRtree->nCursor==0 );
    nodeBlobReset(pRtree);
-    assert( pRtree->nNodeRef==0 || pRtree->bCorrupt );
+    if( pRtree->nNodeRef ){
+      int i;
+      assert( pRtree->bCorrupt );
+      for(i=0; i<HASHSIZE; i++){
+        while( pRtree->aHash[i] ){
+          RtreeNode *pNext = pRtree->aHash[i]->pNext;
+          sqlite3_free(pRtree->aHash[i]);
+          pRtree->aHash[i] = pNext;
+        }
+      }
+    }
    sqlite3_finalize(pRtree->pWriteNode);
    sqlite3_finalize(pRtree->pDeleteNode);
    sqlite3_finalize(pRtree->pReadRowid);
@@ -1392,7 +1405,7 @@ static int nodeRowidIndex(
 ){
  int ii;
  int nCell = NCELL(pNode);
-  assert( nCell<200 );
+  assert( nCell<65536 && nCell>=0 );
  for(ii=0; ii<nCell; ii++){
    if( nodeGetRowid(pRtree, pNode, ii)==iRowid ){
      *piIndex = ii;
@@ -1654,7 +1667,10 @@ static int rtreeStepToLeaf(RtreeCursor *pCur){
    pNode = rtreeNodeOfFirstSearchPoint(pCur, &rc);
    if( rc ) return rc;
    nCell = NCELL(pNode);
-    assert( nCell<200 );
+    if( nCell>RTREE_MAXCELLS ){
+      RTREE_IS_CORRUPT(pRtree);
+      return SQLITE_CORRUPT_VTAB;
+    }
    pCellData = pNode->zData + (4+pRtree->nBytesPerCell*p->iCell);
    while( p->iCell<nCell ){
      sqlite3_rtree_dbl rScore = (sqlite3_rtree_dbl)-1;
@@ -2329,7 +2345,7 @@ static int AdjustTree(
    int iCell;

    cnt++;
-    if( NEVER(cnt>100) ){
+    if( cnt>100 ){
      RTREE_IS_CORRUPT(pRtree);
      return SQLITE_CORRUPT_VTAB;
    }
@@ -2687,15 +2703,6 @@ static int SplitNode(
    rc = updateMapping(pRtree, pCell->iRowid, pLeft, iHeight);
  }

-  if( rc==SQLITE_OK ){
-    rc = nodeRelease(pRtree, pRight);
-    pRight = 0;
-  }
-  if( rc==SQLITE_OK ){
-    rc = nodeRelease(pRtree, pLeft);
-    pLeft = 0;
-  }
-
 splitnode_out:
  nodeRelease(pRtree, pRight);
  nodeRelease(pRtree, pLeft);
@@ -2880,7 +2887,7 @@ static int rtreeInsertCell(
    rc = SplitNode(pRtree, pNode, pCell, iHeight);
  }else{
    rc = AdjustTree(pRtree, pNode, pCell);
-    if( ALWAYS(rc==SQLITE_OK) ){
+    if( rc==SQLITE_OK ){
      if( iHeight==0 ){
        rc = rowidWrite(pRtree, pCell->iRowid, pNode->iNode);
      }else{
@@ -3640,7 +3647,7 @@ static int rtreeInit(
    "Auxiliary rtree columns must be last"                /* 4 */
  };

-  assert( RTREE_MAX_AUX_COLUMN<256 ); /* Aux columns counted by a u8 */
+  assert( RTREE_MAX_AUX_COLUMN<256 );
  if( argc<6 || argc>RTREE_MAX_AUX_COLUMN+3 ){
    *pzErr = sqlite3_mprintf("%s", aErrMsg[2 + (argc>=6)]);
    return SQLITE_ERROR;
@@ -18,6 +18,7 @@ if {![info exists testdir]} {
 } 
 source $testdir/tester.tcl
 ifcapable !rtree { finish_test ; return }
+set testprefix rtreeA

 proc create_t1 {} {
  db close
@@ -258,5 +259,33 @@ do_test rtreeA-7.120 {
  sqlite3_extended_errcode db
 } {SQLITE_CORRUPT_VTAB}

+#-------------------------------------------------------------------------
+reset_db
+
+set N 300
+set rec [binary format SS 0 $N]
+for {set ii 1} {$ii<=$N} {incr ii} {
+  set cell [binary format Wff $ii $ii [expr $ii+1]]
+  set rec [binary format a*a* $rec $cell]
+}
+
+do_execsql_test 8.1 {
+  CREATE VIRTUAL TABLE rt USING rtree(id, x1, x2);
+  INSERT INTO rt VALUES(1, 0.0, 1.0);
+  UPDATE rt_node SET data = $rec WHERE nodeno = 1;
+  DELETE FROM rt_rowid;
+  WITH s(i) AS (
+      SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<300
+  )
+  INSERT INTO rt_rowid SELECT i, 1 FROM s;
+  DELETE FROM rt_parent;
+}
+
+db close
+sqlite3 db test.db 
+
+do_execsql_test 8.2 {
+  SELECT count(*) FROM rt WHERE id = 1
+} {1}

 finish_test
@@ -324,5 +324,47 @@ do_test 6.3.2 { sqlite3_errcode db2 } {SQLITE_ERROR}
 do_test 6.4 {
  catchsql { INSERT INTO c1 VALUES(100, 200) } db2
 } {1 {no such table: main.p1}}
+db2 close
+
+#-------------------------------------------------------------------------
+reset_db
+
+do_execsql_test 7.0 {
+  PRAGMA trusted_schema=OFF;
+  PRAGMA foreign_keys=ON;
+  CREATE TABLE t1(a INTEGER PRIMARY KEY, b, c, d);
+  CREATE TABLE t2(e TEXT PRIMARY KEY NOT NULL, f, g);
+  CREATE TABLE t3(w REAL PRIMARY KEY NOT NULL, x, y);
+  CREATE TABLE t4(z PRIMARY KEY) WITHOUT ROWID;
+  CREATE TABLE tc(a INTEGER, b INTEGER, c BLOB, d TEXT, PRIMARY KEY(a,b));
+  CREATE TABLE wr(a TEXT, b INT, c, PRIMARY KEY(a,b)) WITHOUT ROWID;
+  CREATE TABLE child(x INTEGER PRIMARY KEY, y INTEGER REFERENCES t1(a) ON DELETE CASCADE, z TEXT);
+  INSERT INTO t1 VALUES(1,2,3,4),(2,3.5,'four',x'556677'),(3,NULL,'xyz',15),(4,'bubba',2147483648,0.0);
+  INSERT INTO t1 SELECT a+4,c,d,b FROM t1;
+  INSERT INTO t2 VALUES('x1y',2,3),('x2y','four',x'556677'),('x3y',NULL,'xyz');
+  INSERT INTO t3 VALUES(1.1,'a','b'),(2.2,x'00ff','c');
+  INSERT INTO t4 VALUES('alpha'),('beta'),('gamma');
+  INSERT INTO tc VALUES(1,1,x'0102','one'),(1,2,x'0304','two'),(2,1,x'0506','three');
+  INSERT INTO wr VALUES('a',1,'wa'),('b',2,'wb');
+  INSERT INTO child VALUES(1,1,'c1'),(2,2,'c2');
+}
+
+set C [db one "SELECT unhex('
+      54 0401 0200 0074 6300 1700 0100 0000
+    0000 0000 0101 0000 0000 0069 0001 0402
+    0102 0303 6f6e 6500 0004 2008 40a1 4c0b
+    0d3f 3730 5d92 3f18 f2c9 66a6 4220 a873
+    04cc 5281 ce5c d9d4 dc8b 7003 056f 6e65
+    2d75 1700 0100 0002 5004 0100 0000 6d69
+    7800 1700 0302 6b31 0409 1100
+', ' \n')"]
+
+
+proc conflict_handler {args} { 
+  return "OMIT" 
+}
+do_test 7.1 {
+  list [catch {sqlite3changeset_apply_v2 db $::C conflict_handler} msg] $msg
+} {1 SQLITE_CORRUPT}

 finish_test
@@ -195,13 +195,105 @@ do_test 3.3 {
 #-------------------------------------------------------------------------
 #
 reset_db
-set C [binary format c* 0x54 0x01 0x01 0x00 0x12 0x00 0x05]
+set C [binary format c* {0x54 0x01 0x01 0x00 0x12 0x00 0x05}]

 do_test 4.0 {
  sqlite3changegroup grp
  list [catch { grp add $C } msg] $msg
+} {0 {}}
+grp delete
+
+#-------------------------------------------------------------------------
+#
+reset_db
+set C [binary format c* {0x54 0xda 0xda 0xda 0xda 0xda}]
+
+do_execsql_test 5.0 {
+  CREATE TABLE t1(a PRIMARY KEY, b, c, d);
+}
+
+do_test 5.1 {
+  list [catch { sqlite3changeset_apply db $C noop xFilter } msg] $msg
+} {1 SQLITE_CORRUPT}
+
+#-------------------------------------------------------------------------
+#
+
+foreach {tn type C2hex C3hex} {
+  1 changeset 54020100743100170001000000000000000A030374656E000306656C6576656E
+              54020100743100170001000000000000000A030374656E0003FFFF01656C6576656E
+
+  2 patchset  50020100743100170001000000000000000A0306656C6576656E
+              50020100743100170001000000000000000A03FFFF06656C6576656E
+
+  3 changeset 54020100743100170001000000000000000A030374656E000306656C6576656E
+              54020100743100170001000000000000000A030374656E0003FFFF
+} {
+  reset_db
+
+  do_execsql_test 6.$tn.0 {
+    CREATE TABLE t1(x INTEGER PRIMARY KEY, y TEXT);
+    INSERT INTO t1 VALUES(1, 'one');
+    INSERT INTO t1 VALUES(2, 'two');
+    INSERT INTO t1 VALUES(3, 'three');
+  } {}
+  
+  do_test 6.$tn.1 {
+    set C1 [${type}_from_sql {
+      INSERT INTO t1 VALUES(10, 'ten');
+    }]
+    set C2 [${type}_from_sql {
+      UPDATE t1 SET y='eleven' WHERE x=10;
+    }]
+    db one { SELECT quote($C2) }
+  } "X'$C2hex'"
+  
+  do_test 6.$tn.2 {
+    changeset_to_list [sqlite3changeset_concat $C1 $C2]
+  } {{INSERT t1 0 X. {} {i 10 t eleven}}}
+  
+  set C3 [db one {SELECT unhex($C3hex)}]
+  
+  do_test 6.$tn.3 {
+   list [catch { sqlite3changeset_concat $C1 $C3 } msg] $msg
+  } {1 SQLITE_CORRUPT}
+}
+
+#-------------------------------------------------------------------------
+#
+reset_db
+
+set CSD 5402010074000900010000000000000001030441414141
+set CSI 5402010074001200010000000000000001063258585858585858585858585858585858585858585858585858585858585858585858585858585858585858585858585858585858585858585858585858585858585858585858585858585858585858585858585858585858585858585858585858585858585858585858585858585858585858585858585858585858585858585858585858585858585858585858585858585858585858585858585858585858585858585858585858585858585858585858585858
+
+do_test 7.0 {
+  sqlite3changegroup grp
+  grp add [db one {SELECT unhex($CSD)}]
+  list [catch { grp add [db one {SELECT unhex($CSI)}] } msg] $msg
 } {1 SQLITE_CORRUPT}
 grp delete

+#-------------------------------------------------------------------------
+#
+reset_db
+set CS 54014f300009ff
+
+do_test 8.0 {
+  sqlite3changegroup grp
+  list [catch { grp add [db one {SELECT unhex($CS)}] } msg] $msg
+} {1 SQLITE_CORRUPT}
+grp delete
+
+#-------------------------------------------------------------------------
+#
+reset_db
+set CS 540101740017000003ffffffff
+
+do_test 9.0 {
+  set C [db one {SELECT unhex($CS)}]
+  list [catch { sqlite3changeset_concat $C $C } msg] $msg
+} {1 SQLITE_CORRUPT}
+
 finish_test

+
@@ -82,6 +82,9 @@ do_test 2.2.1 {
  # It is not possible to apply the changeset generated by the following
  # SQL, as none of the three updated rows may be updated as part of the
  # first pass.
+  #
+  # UPDATE 19/05/2026 - it is now possible to apply such an update.
+  #
  do_then_apply_sql -ignorenoop {
    UPDATE t1 SET b=0 WHERE a=1;
    UPDATE t1 SET b=1 WHERE a=2;
@@ -89,7 +92,7 @@ do_test 2.2.1 {
    UPDATE t1 SET b=3 WHERE a=1;
  }
  db2 eval { SELECT a, b FROM t1 }
-} {1 1 2 2 3 3}
+} {1 3 2 1 3 2}
 do_test 2.2.2 { db eval { SELECT a, b FROM t1 } } {1 3 2 1 3 2}

 #-------------------------------------------------------------------------
@@ -0,0 +1,408 @@
+# 2026 January 09
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#***********************************************************************
+# This file implements regression tests for SQLite library.
+#
+
+if {![info exists testdir]} {
+  set testdir [file join [file dirname [info script]] .. .. test]
+} 
+source [file join [file dirname [info script]] session_common.tcl]
+source $testdir/tester.tcl
+ifcapable !session {finish_test; return}
+
+set testprefix sessionchange2
+
+do_test 1.1.1 {
+  sqlite3changegroup grp
+  list [catch { grp change_begin INSERT "nosuchtable" 0 } msg] $msg
+} {1 {SQLITE_ERROR - no such table: nosuchtable}}
+
+do_test 1.1.2 {
+  grp schema db main
+  list [catch { grp change_begin INSERT "nosuchtable" 0 } msg] $msg
+} {1 {SQLITE_ERROR - no such table: nosuchtable}}
+
+do_test 1.1.3 {
+  list [catch { grp change_begin_ne INSERT "nosuchtable" 0 } msg] $msg
+} {1 SQLITE_ERROR}
+
+do_execsql_test 1.2.1 {
+  CREATE TABLE t1(a INTEGER PRIMARY KEY, b);
+}
+do_test 1.2.2 {
+  list [catch { grp change_begin 435 "t1" 0 } msg] $msg
+} {1 SQLITE_ERROR}
+
+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 2.0 {
+  CREATE TABLE t1(a INTEGER PRIMARY KEY, b);
+  CREATE TABLE t2(a, b);
+}
+
+do_test 2.1.1 {
+  sqlite3changegroup grp
+  grp schema db main
+  grp change_begin INSERT "t1" 0
+  grp change_int64 new 0 55
+  grp change_int64 new 1 101
+  grp change_finish false
+} {}
+
+do_test 2.1.2 {
+  set C [grp output]
+  grp delete
+  changeset_to_list $C
+} {
+  {INSERT t1 0 X. {} {i 55 i 101}}
+}
+
+do_test 2.2.1 {
+  sqlite3changegroup grp
+  grp schema db main
+  grp change_begin INSERT "t2" 0
+  grp change_int64 new 0 -5
+  grp change_int64 new 1 -10
+  grp change_int64 new 2 -20
+  grp change_finish false
+} {}
+
+do_test 2.2.2 {
+  set C [grp output]
+  grp delete
+  changeset_to_list $C
+} {
+  {INSERT t2 0 X.. {} {i -5 i -10 i -20}}
+}
+
+do_test 2.2.3 {
+  sqlite3changegroup grp
+  grp schema db main
+  grp change_begin INSERT "t1" 0
+  grp change_int64 new 0 223344
+  grp change_null  new 1
+  grp change_finish false
+} {}
+
+do_test 2.2.4 {
+  set C [grp output]
+  grp delete
+  changeset_to_list $C
+} {
+  {INSERT t1 0 X. {} {i 223344 n {}}}
+}
+
+do_test 2.2.5 {
+  sqlite3changegroup grp
+  grp schema db main
+  grp change_begin DELETE "t1" 0
+
+  grp change_int64 old 0 1
+  grp change_int64 old 1 123
+
+  grp change_finish false
+} {}
+
+do_test 2.2.6 {
+  set C [grp output]
+  grp delete
+  changeset_to_list $C
+} {
+  {DELETE t1 0 X. {i 1 i 123} {}}
+}
+
+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 3.0 {
+  CREATE TABLE t1(a PRIMARY KEY, b, c);
+  CREATE TABLE t2(x, y);
+}
+
+foreach {tn script error} {
+  1 {
+    grp change_begin UPDATE t1 0
+  } {SQLITE_ERROR - invalid change: undefined value in PK of old.* record}
+
+  2 {
+    grp change_begin UPDATE t1 0
+    grp change_int64 old 0 1234
+    grp change_int64 new 0 5678
+  } {SQLITE_ERROR - invalid change: defined value in PK of new.* record}
+
+  3 {
+    grp change_begin UPDATE t1 0
+    grp change_null old 0
+  } {SQLITE_ERROR - invalid change: null value in PK of old.* record}
+
+  4 {
+    grp change_begin UPDATE t1 0
+    grp change_int64 old 0 1234
+    grp change_int64 old 1 20
+  } {SQLITE_ERROR - invalid change: column 1 - old.* value is defined but new.* is undefined}
+
+  5 {
+    grp change_begin UPDATE t1 0
+    grp change_int64 old 0 1234
+    grp change_int64 new 1 20
+  } {SQLITE_ERROR - invalid change: column 1 - old.* value is undefined but new.* is defined}
+
+  6 {
+    grp change_begin INSERT t1 0
+    grp change_null  new 0
+    grp change_int64 new 1 20
+  } {SQLITE_ERROR - invalid change: null value in PK}
+
+  7 {
+    grp change_begin INSERT t1 0
+    grp change_int64 new 1 20
+  } {SQLITE_ERROR - invalid change: column 0 is undefined}
+
+  8 {
+    grp change_begin INSERT t1 0
+    grp change_int64 new 0 20
+  } {SQLITE_ERROR - invalid change: column 1 is undefined}
+
+  9 {
+    grp change_begin DELETE t1 0
+    grp change_int64 old 0 20
+  } {SQLITE_ERROR - invalid change: column 1 is undefined}
+
+  10 {
+    grp change_begin DELETE t1 0
+    grp change_int64 old 1 20
+  } {SQLITE_ERROR - invalid change: column 0 is undefined}
+
+} {
+  sqlite3changegroup grp
+  grp schema db main
+  eval $script
+  do_test 3.1.$tn.1 {
+    list [catch { grp change_finish false } msg] $msg
+  } [list 1 $error]
+  grp delete
+
+  sqlite3changegroup grp
+  grp schema db main
+  eval $script
+  do_test 3.1.$tn.2 {
+    list [catch { grp change_finishne false } msg] $msg
+  } [list 1 [lindex $error 0]]
+  grp delete
+}
+
+do_test 3.2.1 {
+  sqlite3changegroup grp
+  grp schema db main
+  grp change_begin DELETE t1 0
+  list [catch {grp change_int64 new 0 20} msg] $msg
+} {1 SQLITE_ERROR}
+do_test 3.2.2 {
+  grp change_finish true
+  grp change_begin INSERT t1 0
+  list [catch {grp change_int64 old 0 20} msg] $msg
+} {1 SQLITE_ERROR}
+
+grp delete
+
+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 3.0 {
+  CREATE TABLE t1(a INTEGER PRIMARY KEY, b);
+  CREATE TABLE t2(a, b);
+}
+
+proc do_changegroup_script_test {tn script changeset patchset} {
+
+breakpoint
+  sqlite3changegroup grp
+  grp schema db main
+  eval $script
+  do_test $tn.1 {
+    changeset_to_list [grp output]
+  } [list {*}$changeset]
+  grp delete
+
+  sqlite3changegroup grp
+  grp config patchset 1
+  grp schema db main
+  eval $script
+  do_test $tn.2 {
+    changeset_to_list [grp output]
+  } [list {*}$patchset]
+  grp delete
+
+}
+
+proc do_changegroup_patchset_test {tn script patchset} {
+
+  sqlite3changegroup grp
+  grp config patchset 1
+  grp schema db main
+  eval $script
+  do_test $tn {
+    changeset_to_list [grp output]
+  } [list {*}$patchset]
+  grp delete
+
+}
+
+do_changegroup_script_test 3.1 {
+  grp change_begin UPDATE t1 0
+  grp change_int64 old 0 123
+  grp change_int64 old 1 456
+  grp change_int64 new 1 789
+  grp change_finish false
+
+  grp change_begin INSERT t1 0
+  grp change_int64 new 0 124
+  grp change_text  new 1 hello_world
+  grp change_finish false
+
+  grp change_begin  DELETE t1 0
+  grp change_int64  old 0 125
+  grp change_double old 1 45.67
+  grp change_finish false
+} {
+  {DELETE t1 0 X. {i 125 f 45.67} {}}
+  {INSERT t1 0 X. {} {i 124 t hello_world}}
+  {UPDATE t1 0 X. {i 123 i 456} {{} {} i 789}}
+} {
+  {DELETE t1 0 X. {i 125 {} {}} {}}
+  {INSERT t1 0 X. {} {i 124 t hello_world}}
+  {UPDATE t1 0 X. {i 123 {} {}} {{} {} i 789}}
+}
+
+do_changegroup_script_test 3.1.2 {
+  grp change_begin INSERT t1 0
+  grp change_int64 new 0 124
+  grp change_text  new 1 hello_world
+  grp change_finish false
+} {
+  {INSERT t1 0 X. {} {i 124 t hello_world}}
+} {
+  {INSERT t1 0 X. {} {i 124 t hello_world}}
+}
+
+do_changegroup_patchset_test 3.2 {
+  grp change_begin UPDATE t1 0
+  grp change_int64 old 0 123
+  grp change_int64 new 1 789
+  grp change_finish false
+
+  grp change_begin DELETE t1 1
+  grp change_int64 old 0 122
+  grp change_finish false
+} {
+  {DELETE t1 1 X. {i 122 {} {}} {}}
+  {UPDATE t1 0 X. {i 123 {} {}} {{} {} i 789}}
+}
+
+do_changegroup_patchset_test 3.3 {
+  grp change_begin DELETE t1 0
+  grp change_int64 old 0 123
+  grp change_finish false
+} {
+  {DELETE t1 0 X. {i 123 {} {}} {}}
+}
+
+do_changegroup_script_test 3.4 {
+  grp change_begin INSERT t1 0
+  grp change_int64 new 0 124
+  grp change_text  new 1 hello_world
+  grp change_finish false
+
+  grp change_begin DELETE t1 0
+  grp change_int64 old 0 124
+  grp change_text  old 1 hello_world
+  grp change_finish false
+
+  grp change_begin INSERT t1 0
+  grp change_int64 new 0 1000
+  grp change_blob  new 1 abcd
+  grp change_finish false
+
+  grp change_begin UPDATE t1 0
+  grp change_int64 old 0 1000
+  grp change_blob  old 1 abcd
+  grp change_blob  new 1 bcda
+  grp change_finish false
+} {
+  {INSERT t1 0 X. {} {i 1000 b bcda}}
+} {
+  {INSERT t1 0 X. {} {i 1000 b bcda}}
+}
+
+do_changegroup_script_test 3.5 {
+  grp change_begin DELETE t1 0
+  grp change_int64  old 0 1000
+  grp change_text-1 old 1 "hello world"
+  grp change_finish false
+} {
+  {DELETE t1 0 X. {i 1000 t {hello world}} {}}
+} {
+  {DELETE t1 0 X. {i 1000 {} {}} {}}
+}
+
+#-------------------------------------------------------------------------
+
+# Calling sqlite3changegroup_change_finish() if a change has not been
+# started is a no-op.
+#
+do_test 4.0 {
+  sqlite3changegroup grp
+  grp change_finish false
+} {}
+
+# But calling any other sqlite3changegroup_change_xxx() function if a
+# change has not been started is a misuse.
+#
+foreach {tn cmd} {
+  1 { grp change_blob   old 0 abcd } 
+  2 { grp change_text   new 1 helloworld } 
+  3 { grp change_int64  old 2 12345678 } 
+  4 { grp change_double new 3 12.345 } 
+  5 { grp change_null   old 4 }
+} {
+  do_test 4.1.$tn {
+    list [catch $::cmd msg] $msg
+  } {1 SQLITE_MISUSE}
+}
+grp delete
+
+do_execsql_test 4.2.1 {
+  CREATE TABLE t3(a, b PRIMARY KEY);
+}
+do_test 4.2 {
+  sqlite3changegroup grp
+  grp schema db main
+  grp change_begin INSERT t3 0 
+
+  list [catch { grp change_begin INSERT t3 0 } msg] $msg
+} {1 SQLITE_MISUSE}
+grp delete
+
+do_test 4.3 {
+  sqlite3changegroup grp
+  grp schema db main
+  grp change_begin UPDATE t3 0
+} {}
+foreach {tn cmd} {
+  1 { grp change_blob   old -1 abcd } 
+  2 { grp change_text   new 2 helloworld } 
+} {
+  do_test 4.4.$tn {
+    list [catch $::cmd msg] $msg
+  } {1 SQLITE_RANGE}
+}
+grp delete
+
+finish_test
+
@@ -0,0 +1,298 @@
+# 2026 May 18
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#***********************************************************************
+# This file implements regression tests for SQLite library.
+#
+
+if {![info exists testdir]} {
+  set testdir [file join [file dirname [info script]] .. .. test]
+} 
+source [file join [file dirname [info script]] session_common.tcl]
+source $testdir/tester.tcl
+ifcapable !session {finish_test; return}
+
+set testprefix sessionconflict2
+
+forcedelete test.db2
+sqlite3 db2 test.db2
+
+do_test 1.0 {
+  do_common_sql {
+    CREATE TABLE t1(a PRIMARY KEY, b, c UNIQUE);
+    INSERT INTO t1 VALUES(1, 1, 1);
+    INSERT INTO t1 VALUES(2, 2, 2);
+    INSERT INTO t1 VALUES(3, 3, 3);
+  }
+} {}
+
+do_test 1.1 {
+  do_then_apply_sql {
+    UPDATE t1 SET c=NULL WHERE a=1;
+    UPDATE t1 SET c=1 WHERE a=3;
+    UPDATE t1 SET c=3 WHERE a=1;
+  }
+} {}
+
+do_execsql_test -db db 1.2 {
+  SELECT rowid, * FROM t1
+} {
+  1 1 1 3
+  2 2 2 2
+  3 3 3 1
+}
+
+do_execsql_test -db db2 1.3 {
+  SELECT rowid, * FROM t1
+} {
+  1 1 1 3
+  2 2 2 2
+  3 3 3 1
+}
+
+#--------------------------------------------------------------------------
+reset_db
+db2 close
+forcedelete test.db2
+sqlite3 db2 test.db2
+
+do_test 2.0 {
+  do_common_sql {
+    CREATE TABLE t1(a PRIMARY KEY, b, c UNIQUE) WITHOUT ROWID;
+    INSERT INTO t1 VALUES(1, 1, 1);
+    INSERT INTO t1 VALUES(2, 2, 2);
+    INSERT INTO t1 VALUES(3, 3, 3);
+  }
+} {}
+
+do_test 2.1 {
+  do_then_apply_sql {
+    UPDATE t1 SET c=NULL WHERE a=1;
+    UPDATE t1 SET c=1 WHERE a=3;
+    UPDATE t1 SET c=3 WHERE a=1;
+  }
+} {}
+
+do_execsql_test -db db 2.2 {
+  SELECT * FROM t1
+} {
+  1 1 3
+  2 2 2
+  3 3 1
+}
+
+do_execsql_test -db db2 2.3 {
+  SELECT * FROM t1
+} {
+  1 1 3
+  2 2 2
+  3 3 1
+}
+
+#--------------------------------------------------------------------------
+reset_db
+db2 close
+forcedelete test.db2
+sqlite3 db2 test.db2
+
+do_test 3.0 {
+  do_common_sql {
+    CREATE TABLE t1(a INTEGER PRIMARY KEY, b, c UNIQUE);
+    INSERT INTO t1 VALUES(1, 1, 1);
+    INSERT INTO t1 VALUES(2, 2, 2);
+    INSERT INTO t1 VALUES(3, 3, 3);
+  }
+} {}
+
+do_test 3.1 {
+  do_then_apply_sql {
+    UPDATE t1 SET c=NULL WHERE a=1;
+    UPDATE t1 SET c=1 WHERE a=3;
+    UPDATE t1 SET c=3 WHERE a=1;
+  }
+} {}
+
+do_execsql_test -db db 3.2 {
+  SELECT rowid, * FROM t1
+} {
+  1 1 1 3
+  2 2 2 2
+  3 3 3 1
+}
+
+do_execsql_test -db db2 3.3 {
+  SELECT rowid, * FROM t1
+} {
+  1 1 1 3
+  2 2 2 2
+  3 3 3 1
+}
+
+#-------------------------------------------------------------------------
+db2 close
+reset_db
+forcedelete test.db2
+sqlite3 db2 test.db2
+
+set ::conflict_list [list]
+proc xConflict {args} {
+  lappend ::conflict_list $args
+  return "OMIT"
+}
+
+proc do_conflict_test {tn bNoUpdateLoop script clist} {
+
+  uplevel [list do_test $tn.1 [subst -nocommands {
+    sqlite3session S db "main"
+    S attach *
+    eval {$script}
+    set ::changeset [S changeset]
+    S delete
+  }] {}]
+
+  if {$bNoUpdateLoop} {
+    uplevel [list do_test $tn.2.no {
+      set ::conflict_list [list]
+      sqlite3changeset_apply_v2 -noupdateloop db2 $::changeset xConflict
+      set ::conflict_list
+    } [list {*}$clist]]
+  } else {
+    uplevel [list do_test $tn.2 {
+      set ::conflict_list [list]
+      sqlite3changeset_apply_v2 db2 $::changeset xConflict
+      set ::conflict_list
+    } [list {*}$clist]]
+  }
+}
+
+do_test 4.0 {
+  do_common_sql {
+    CREATE TABLE t1(a INT PRIMARY KEY, b, c UNIQUE, d UNIQUE);
+    WITH s(i) AS (
+      SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<10
+    )
+    INSERT INTO t1 SELECT i, i, i, i FROM s;
+  }
+} {}
+
+proc swap {tbl pkcol valcol pk1 pk2} {
+  set val1 [db one "SELECT $valcol FROM $tbl WHERE $pkcol = \$pk1"]
+  set val2 [db one "SELECT $valcol FROM $tbl WHERE $pkcol = \$pk2"]
+
+  db eval "
+    UPDATE $tbl SET $valcol = NULL WHERE $pkcol IN (\$pk1, \$pk2);
+    UPDATE $tbl SET $valcol = \$val2 WHERE $pkcol = \$pk1;
+    UPDATE $tbl SET $valcol = \$val1 WHERE $pkcol = \$pk2;
+  "
+}
+
+do_conflict_test 4.1.1 0 {
+  swap t1 a c 4 5
+  swap t1 a c 2 3
+  swap t1 a c 8 1
+} {
+}
+
+do_execsql_test -db db 4.1.2 {
+  SELECT a, c FROM t1
+} {
+  1 8 2 3 3 2 4 5 5 4 6 6 7 7 8 1 9 9 10 10
+}
+do_execsql_test -db db2 4.1.3 {
+  SELECT a, c FROM t1
+} {
+  1 8 2 3 3 2 4 5 5 4 6 6 7 7 8 1 9 9 10 10
+}
+
+do_conflict_test 4.2.1 0 {
+  swap t1 a d 10 9
+  swap t1 a d  8 7
+  swap t1 a d  7 6
+  swap t1 a d  5 4
+  swap t1 a d  4 8
+} {
+}
+
+do_execsql_test -db db 4.2.2 {
+  SELECT a, d FROM t1
+} {
+  1 1  2 2  3 3  4 7  5 4  6 8  7 6  8 5  9 10  10 9
+}
+do_execsql_test -db db2 4.2.3 {
+  SELECT a, d FROM t1
+} {
+  1 1  2 2  3 3  4 7  5 4  6 8  7 6  8 5  9 10  10 9
+}
+
+do_execsql_test -db db2 4.3 {
+  INSERT INTO t1(a, b, c, d) VALUES(11, 11, 11, 11);
+}
+
+do_conflict_test 4.3.1 0 {
+  swap t1 a c 3 6
+  db eval { UPDATE t1 SET d=11 WHERE a=10; }
+  swap t1 a d 4 8
+} {
+  {UPDATE t1 CONSTRAINT {i 10 {} {} {} {} i 9} {{} {} {} {} {} {} i 11}}
+}
+
+do_conflict_test 4.3.2 0 {
+  swap t1 a c 1 2
+  swap t1 a c 3 4
+  db eval { UPDATE t1 SET c=11 WHERE a=10; }
+  swap t1 a d 5 6
+  swap t1 a d 4 5
+} {
+  {UPDATE t1 CONSTRAINT {i 10 {} {} i 10 {} {}} {{} {} {} {} i 11 {} {}}}
+}
+
+do_conflict_test 4.3.3 0 {
+  swap t1 a c 1 2
+  swap t1 a c 2 3
+  swap t1 a c 3 4
+  swap t1 a c 4 1
+} {
+}
+db2 close
+
+#-------------------------------------------------------------------------
+reset_db
+forcedelete test.db2
+sqlite3 db2 test.db2
+
+do_test 5.0 {
+  do_common_sql {
+    CREATE TABLE t1(a INT PRIMARY KEY, b UNIQUE);
+    INSERT INTO t1 VALUES('one', 'one');
+    INSERT INTO t1 VALUES('two', 'two');
+    INSERT INTO t1 VALUES('three', 'three');
+    INSERT INTO t1 VALUES('four', 'four');
+    INSERT INTO t1 VALUES('five', 'five');
+  }
+} {}
+
+do_conflict_test 5.1 1 {
+} {
+}
+
+do_conflict_test 5.2 1 {
+  swap t1 a b one two
+} {
+ {UPDATE t1 CONSTRAINT {t two t two} {{} {} t one}} 
+ {UPDATE t1 CONSTRAINT {t one t one} {{} {} t two}}
+}
+
+do_conflict_test 5.2 0 {
+  swap t1 a b four five
+} {
+}
+
+
+finish_test
+
@@ -43,7 +43,7 @@ set C2 [changeset_from_sql {
  INSERT INTO t1 VALUES(0, 0, 0, 0);
 }]

-do_faultsim_test 1 -faults oom* -prep {
+do_faultsim_test 1.2 -faults oom* -prep {
  sqlite3changegroup G
 } -body {
  G schema db main
@@ -56,6 +56,20 @@ do_faultsim_test 1 -faults oom* -prep {
  faultsim_test_result {0 {}} {1 SQLITE_NOMEM}
 }

+do_faultsim_test 1.3 -faults oom* -prep {
+  sqlite3changegroup G
+} -body {
+  G schema db main
+  G change_begin INSERT t1 0
+  G change_int64 new 0 12345
+  G change_int64 new 1 67890
+  G output
+  set {} {}
+} -test {
+  catch { G delete }
+  faultsim_test_result {0 {}} {1 SQLITE_NOMEM}
+}
+
 #-------------------------------------------------------------------------
 reset_db
 do_execsql_test 2.0 {
@@ -80,4 +94,45 @@ do_faultsim_test 2 -faults oom-t* -prep {
  catch { S delete }
 }

+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 3.0 {
+  CREATE TABLE t1(a PRIMARY KEY, b UNIQUE);
+  INSERT INTO t1 VALUES(1, 'one');
+  INSERT INTO t1 VALUES(2, 'two');
+  INSERT INTO t1 VALUES(3, 'three');
+  INSERT INTO t1 VALUES(4, 'four');
+  INSERT INTO t1 VALUES(5, 'five');
+}
+faultsim_save_and_close
+faultsim_restore_and_reopen
+
+set C [changeset_from_sql {
+  UPDATE t1 SET b=NULL WHERE a IN (1, 3, 5);
+  UPDATE t1 SET b='three' WHERE a=1;
+  UPDATE t1 SET b='five' WHERE a=3;
+  UPDATE t1 SET b='one' WHERE a=5;
+}]
+
+do_execsql_test 3.1 {
+  SELECT * FROM t1
+} {
+  1 three 2 two 3 five 4 four 5 one
+}
+
+proc xConflict {args} {
+  lappend ::conflict_list $args
+  return "OMIT"
+}
+
+do_faultsim_test 3 -faults oom* -prep {
+  faultsim_restore_and_reopen
+  db eval {SELECT * FROM sqlite_schema}
+} -body {
+  sqlite3changeset_apply_v2 db $::C xConflict
+  set {} {}
+} -test {
+  faultsim_test_result {0 {}} {1 SQLITE_NOMEM}
+}
+
 finish_test
@@ -181,5 +181,11 @@ do_invert_test 4.1 {
  {UPDATE t1 0 X. {i 4 t three} {{} {} t four}}
 }

+#-------------------------------------------------------------------------
+#
+do_test 5.0 {
+  set C [db one {SELECT unhex('54000009')}]
+  list [catch { sqlite3changeset_invert $C } msg] $msg
+} {1 SQLITE_CORRUPT}

 finish_test
@@ -228,5 +228,30 @@ do_execsql_test 3.8 {
  SELECT * FROM c1;
 } {two}

+#-------------------------------------------------------------------------
+reset_db
+
+do_execsql_test 4.0 {
+  CREATE TABLE t1(a INTEGER PRIMARY KEY, b TEXT, c INTEGER);
+  ATTACH ':memory:' AS aux;
+  CREATE TABLE aux.t1(a INTEGER PRIMARY KEY, b TEXT, c INTEGER);
+}
+
+do_test 4.1 {
+  sqlite3session S db main
+  S diff aux t1
+} {}
+
+do_test 4.2 {
+  sqlite3changeset_apply_v2 -ignorenoop -noaction db "" conflict 
+} {}
+
+breakpoint
+do_test 4.3 {
+  S diff aux t1
+} {}
+
+S delete
+
 finish_test

@@ -1366,11 +1366,23 @@ int sqlite3changeset_apply_v3(
 **   database behave as if they were declared with "ON UPDATE NO ACTION ON
 **   DELETE NO ACTION", even if they are actually CASCADE, RESTRICT, SET NULL
 **   or SET DEFAULT.
+**
+** <dt>SQLITE_CHANGESETAPPLY_NOUPDATELOOP <dd>
+**   Sometimes, a changeset contains two or more update statements such that
+**   although after applying all updates the database will contain no 
+**   constraint violations, no single update can be applied before the others.
+**   The simplest example of this is a pair of UPDATEs that have "swapped"
+**   two column values with a UNIQUE constraint.
+**   <p>
+**   Usually, sqlite3changeset_apply() and similar functions work hard to try
+**   to find a way to apply such a changeset. However, if this flag is set,
+**   then all such updates are considered CONSTRAINT conflicts.
 */
 #define SQLITE_CHANGESETAPPLY_NOSAVEPOINT   0x0001
 #define SQLITE_CHANGESETAPPLY_INVERT        0x0002
 #define SQLITE_CHANGESETAPPLY_IGNORENOOP    0x0004
 #define SQLITE_CHANGESETAPPLY_FKNOACTION    0x0008
+#define SQLITE_CHANGESETAPPLY_NOUPDATELOOP  0x0010

 /* 
 ** CAPI3REF: Constants Passed To The Conflict Handler
@@ -1853,6 +1865,232 @@ int sqlite3session_config(int op, void *pArg);
 */
 #define SQLITE_SESSION_CONFIG_STRMSIZE 1

+/*
+** CAPI3REF: Configure a changegroup object
+**
+** Configure the changegroup object passed as the first argument. 
+** At present the only valid value for the second parameter is
+** [SQLITE_CHANGEGROUP_CONFIG_PATCHSET].
+*/
+int sqlite3changegroup_config(sqlite3_changegroup*, int, void *pArg);
+
+/*
+** CAPI3REF: Options for sqlite3changegroup_config().
+**
+** The following values may be passed as the 2nd parameter to
+** sqlite3changegroup_config().
+**
+** <dt>SQLITE_CHANGEGROUP_CONFIG_PATCHSET <dd>
+**   A changegroup object generates either a changeset or patchset. Usually,
+**   this is determined by whether the first call to sqlite3changegroup_add()
+**   is passed a changeset or a patchset. Or, if the first changes are added
+**   to the changegroup object using the sqlite3changegroup_change_xxx()
+**   APIs, then this option may be used to configure whether the changegroup
+**   object generates a changeset or patchset. 
+**
+**   When this option is invoked, parameter pArg must point to a value of
+**   type int. If the changegroup currently contains zero changes, and the
+**   value of the int variable is zero or greater than zero, then the
+**   changegroup is configured to generate a changeset or patchset,
+**   respectively. It is a no-op, not an error, if the changegroup is not
+**   configured because it has already started accumulating changes.
+**
+**   Before returning, the int variable is set to 0 if the changegroup is
+**   configured to generate a changeset, or 1 if it is configured to generate
+**   a patchset.
+*/
+#define SQLITE_CHANGEGROUP_CONFIG_PATCHSET 1
+
+
+/*
+** CAPI3REF: Begin adding a change to a changegroup
+**
+** This API is used, in concert with other sqlite3changegroup_change_xxx()
+** APIs, to add changes to a changegroup object one at a time. To add a 
+** single change, the caller must:
+**
+**   1. Invoke sqlite3changegroup_change_begin() to indicate the type of
+**      change (INSERT, UPDATE or DELETE), the affected table and whether
+**      or not the change should be marked as indirect.
+**
+**   2. Invoke sqlite3changegroup_change_int64() or one of the other four
+**      value functions - _null(), _double(), _text() or _blob() - one or
+**      more times to specify old.* and new.* values for the change being
+**      constructed.
+**
+**   3. Invoke sqlite3changegroup_change_finish() to either finish adding
+**      the change to the group, or to discard the change altogether.
+**
+** The first argument to this function must be a pointer to the existing
+** changegroup object that the change will be added to. The second argument
+** must be SQLITE_INSERT, SQLITE_UPDATE or SQLITE_DELETE. The third is the
+** name of the table that the change affects, and the fourth is a boolean
+** flag specifying whether the change should be marked as "indirect" (if
+** bIndirect is non-zero) or not indirect (if bIndirect is zero).
+**
+** Following a successful call to this function, this function may not be
+** called again on the same changegroup object until after
+** sqlite3changegroup_change_finish() has been called. Doing so is an
+** SQLITE_MISUSE error.
+**
+** The changegroup object passed as the first argument must be already 
+** configured with schema data for the specified table. It may be configured 
+** either by calling sqlite3changegroup_schema() with a database that contains
+** the table, or sqlite3changegroup_add() with a changeset that contains the 
+** table. If the changegroup object has not been configured with a schema for
+** the specified table when this function is called, SQLITE_ERROR is returned.
+**
+** If successful, SQLITE_OK is returned. Otherwise, if an error occurs, an
+** SQLite error code is returned. In this case, if argument pzErr is non-NULL,
+** then (*pzErr) may be set to point to a buffer containing a utf-8 formated,
+** nul-terminated, English language error message. It is the responsibility
+** of the caller to eventually free this buffer using sqlite3_free().
+*/
+int sqlite3changegroup_change_begin(
+  sqlite3_changegroup*,
+  int eOp,
+  const char *zTab,
+  int bIndirect,
+  char **pzErr
+);
+
+/*
+** CAPI3REF: Add a 64-bit integer to a changegroup
+**
+** This function may only be called between a successful call to 
+** sqlite3changegroup_change_begin() and its matching
+** sqlite3changegroup_change_finish() call. If it is called at any
+** other time, it is an SQLITE_MISUSE error. Calling this function 
+** specifies a 64-bit integer value to be used in the change currently being
+** added to the changegroup object passed as the first argument.
+**
+** The second parameter, bNew, specifies whether the value is to be part of
+** the new.* (if bNew is non-zero) or old.* (if bNew is zero) record of
+** the change under construction. If this does not match the type of change
+** specified by the preceding call to sqlite3changegroup_change_begin() (i.e.
+** an old.* value for an SQLITE_INSERT change, or a new.* value for an
+** SQLITE_DELETE), then SQLITE_ERROR is returned.
+**
+** The third parameter specifies the column of the old.* or new.* record that
+** the value will be a part of. If the specified table has an explicit primary
+** key, then this is the index of the table column, numbered from 0 in the order
+** specified within the CREATE TABLE statement. Or, if the table uses an 
+** implicit rowid key, then the column 0 is the rowid and the explicit columns
+** are numbered starting from 1. If the iCol parameter is less than 0 or greater
+** than the index of the last column in the table, SQLITE_RANGE is returned.
+**
+** The fourth parameter is the integer value to use as part of the old.* or
+** new.* record.
+**
+** If this call is successful, SQLITE_OK is returned. Otherwise, if an
+** error occurs, an SQLite error code is returned.
+*/
+int sqlite3changegroup_change_int64(
+  sqlite3_changegroup*, 
+  int bNew,
+  int iCol,
+  sqlite3_int64 iVal
+);
+
+/*
+** CAPI3REF: Add a NULL to a changegroup
+**
+** This function is similar to sqlite3changegroup_change_int64(). Except that
+** it configures the change currently under construction with a NULL value
+** instead of a 64-bit integer.
+*/
+int sqlite3changegroup_change_null(sqlite3_changegroup*, int, int);
+
+/*
+** CAPI3REF: Add an double to a changegroup
+**
+** This function is similar to sqlite3changegroup_change_int64(). Except that
+** it configures the change currently being constructed with a real value
+** instead of a 64-bit integer.
+*/
+int sqlite3changegroup_change_double(sqlite3_changegroup*, int, int, double);
+
+/*
+** CAPI3REF: Add a text value to a changegroup
+**
+** This function is similar to sqlite3changegroup_change_int64(). It configures
+** the currently accumulated change with a text value instead of a 64-bit
+** integer. Parameter pVal points to a buffer containing the text encoded using 
+** utf-8. Parameter nVal may either be the size of the text value in bytes, or
+** else a negative value, in which case the buffer pVal points to is assumed to
+** be nul-terminated.
+*/
+int sqlite3changegroup_change_text(
+  sqlite3_changegroup*, int, int, const char *pVal, int nVal
+);
+
+/*
+** CAPI3REF: Add a blob to a changegroup
+**
+** This function is similar to sqlite3changegroup_change_int64(). It configures
+** the currently accumulated change with a blob value instead of a 64-bit
+** integer. Parameter pVal points to a buffer containing the blob. Parameter 
+** nVal is the size of the blob in bytes.
+*/
+int sqlite3changegroup_change_blob(
+    sqlite3_changegroup*, int, int, const void *pVal, int nVal
+);
+
+/*
+** CAPI3REF: Finish adding one-at-at-time changes to a changegroup
+**
+** This function may only be called following a successful call to
+** sqlite3changegroup_change_begin(). Otherwise, it is an SQLITE_MISUSE error.
+**
+** If parameter bDiscard is non-zero, then the current change is simply
+** discarded. In this case this function is always successful and SQLITE_OK
+** returned.
+**
+** If parameter bDiscard is zero, then an attempt is made to add the current
+** change to the changegroup. Assuming the changegroup is configured to
+** produce a changeset (not a patchset), this requires that:
+**
+**   *  If the change is an INSERT or DELETE, then a value must be specified
+**      for all columns of the new.* or old.* record, respectively.
+**
+**   *  If the change is an UPDATE record, then values must be provided for
+**      the PRIMARY KEY columns of the old.* record, but must not be provided
+**      for PRIMARY KEY columns of the new.* record.
+**
+**   *  If the change is an UPDATE record, then for each non-PRIMARY KEY
+**      column in the old.* record for which a value has been provided, a
+**      value must also be provided for the same column in the new.* record.
+**      Similarly, for each non-PK column in the old.* record for which 
+**      a value is not provided, a value must not be provided for the same
+**      column in the new.* record.
+**
+**   *  All values specified for PRIMARY KEY columns must be non-NULL.
+**
+** Otherwise, it is an error. 
+**
+** If the changegroup already contains a change for the same row (identified
+** by PRIMARY KEY columns), then the current change is combined with the
+** existing change in the same way as for sqlite3changegroup_add().
+**
+** For a patchset, all of the above rules apply except that it doesn't matter
+** whether or not values are provided for the non-PK old.* record columns
+** for an UPDATE or DELETE change. This means that code used to produce
+** a changeset using the sqlite3changegroup_change_xxx() APIs may also
+** be used to produce patchsets.
+**
+** If the call is successful, SQLITE_OK is returned. Otherwise, if an error
+** occurs, an SQLite error code is returned. If an error is returned and
+** parameter pzErr is not NULL, then (*pzErr) may be set to point to a buffer
+** containing a nul-terminated, utf-8 encoded, English language error message.
+** It is the responsibility of the caller to eventually free any such error 
+** message buffer using sqlite3_free().
+*/
+int sqlite3changegroup_change_finish(
+  sqlite3_changegroup*, 
+  int bDiscard, 
+  char **pzErr
+);
+
 /*
 ** Make sure we can call this stuff from C++.
 */
@@ -13,6 +13,8 @@
  typedef unsigned char u8;
 #endif

+extern const char *sqlite3ErrName(int);
+
 typedef struct TestSession TestSession;
 struct TestSession {
  sqlite3_session *pSession;
@@ -397,7 +399,6 @@ static int SQLITE_TCLAPI test_session_cmd(
      }
      rc = sqlite3session_object_config(pSession, aOpt[iOpt].opt, &iArg);
      if( rc!=SQLITE_OK ){
-        extern const char *sqlite3ErrName(int);
        Tcl_SetObjResult(interp, Tcl_NewStringObj(sqlite3ErrName(rc), -1));
      }else{
        Tcl_SetObjResult(interp, Tcl_NewIntObj(iArg));
@@ -913,6 +914,9 @@ static int SQLITE_TCLAPI testSqlite3changesetApply(
      }
      else if( n>2 && n<=11 && 0==sqlite3_strnicmp("-ignorenoop", z1, n) ){
        flags |= SQLITE_CHANGESETAPPLY_IGNORENOOP;
+      }
+      else if( n>3 && n<=13 && 0==sqlite3_strnicmp("-noupdateloop", z1, n) ){
+        flags |= SQLITE_CHANGESETAPPLY_NOUPDATELOOP;
      }else{
        break;
      }
@@ -1094,7 +1098,7 @@ static int SQLITE_TCLAPI test_sqlite3changeset_invert(
  memset(&sIn, 0, sizeof(sIn));
  memset(&sOut, 0, sizeof(sOut));
  sIn.nStream = test_tcl_integer(interp, SESSION_STREAM_TCL_VAR);
-  sIn.aData = Tcl_GetByteArrayFromObj(objv[1], &nn);
+  sIn.aData = testGetByteArrayFromObj(objv[1], &nn);
  sIn.nData = (int)nn;

  if( sIn.nStream ){
@@ -1111,9 +1115,25 @@ static int SQLITE_TCLAPI test_sqlite3changeset_invert(
    Tcl_SetObjResult(interp,Tcl_NewByteArrayObj((unsigned char*)sOut.p,sOut.n));
  }
  sqlite3_free(sOut.p);
+  free(sIn.aData);
  return rc;
 }

+/*
+** Copy buffer aIn[] to a new nIn byte buffer obtained from malloc(). Use
+** plain malloc() instead of any Tcl function because valgrind and asan are
+** better at detecting small overflows in that case. Avoid sqlite3_malloc()
+** here because that means dealing with injected OOM errors. 
+**
+** The caller is responsible for eventually calling free() on the returned
+** value.
+*/
+static u8 *copyToMalloc(const u8 *aIn, int nIn){
+  u8 *pRet = malloc(nIn);
+  memcpy(pRet, aIn, nIn);
+  return pRet;
+}
+
 /*
 ** sqlite3changeset_concat LEFT RIGHT
 */
@@ -1144,6 +1164,9 @@ static int SQLITE_TCLAPI test_sqlite3changeset_concat(
  sLeft.nStream = test_tcl_integer(interp, SESSION_STREAM_TCL_VAR);
  sRight.nStream = sLeft.nStream;

+  sLeft.aData = copyToMalloc(sLeft.aData, sLeft.nData);
+  sRight.aData = copyToMalloc(sRight.aData, sRight.nData);
+
  if( sLeft.nStream>0 ){
    rc = sqlite3changeset_concat_strm(
        testStreamInput, (void*)&sLeft,
@@ -1156,6 +1179,9 @@ static int SQLITE_TCLAPI test_sqlite3changeset_concat(
    );
  }

+  free(sLeft.aData);
+  free(sRight.aData);
+
  if( rc!=SQLITE_OK ){
    rc = test_session_error(interp, rc, 0);
  }else{
@@ -1554,6 +1580,14 @@ static void test_changegroup_del(void *clientData){
  ckfree(pGrp);
 }

+static int testGetNewOrOld(Tcl_Interp *interp, Tcl_Obj *pObj, int *pbNew){
+  const char *azVal[] = { "old", "new", 0 };
+  int iIdx = 0;
+  int rc = Tcl_GetIndexFromObj(interp, pObj, azVal, "record", 0, &iIdx);
+  *pbNew = iIdx;
+  return rc;
+}
+
 /*
 ** Tclcmd:  $changegroup schema DB DBNAME
 ** Tclcmd:  $changegroup add CHANGESET
@@ -1571,14 +1605,26 @@ static int SQLITE_TCLAPI test_changegroup_cmd(
    const char *zSub;
    int nArg;
    const char *zMsg;
-    int iSub;
  } aSub[] = {
-    { "schema",       2, "DB DBNAME",  }, /* 0 */
-    { "add",          1, "CHANGESET",  }, /* 1 */
-    { "output",       0, "",           }, /* 2 */
-    { "delete",       0, "",           }, /* 3 */
-    { "add_change",   1, "ITERATOR",   }, /* 4 */
-    { 0 }
+    { "schema",          2, "DB DBNAME"            },    /* 0 */
+    { "add",             1, "CHANGESET"            },    /* 1 */
+    { "output",          0, ""                     },    /* 2 */
+    { "delete",          0, ""                     },    /* 3 */
+    { "add_change",      1, "ITERATOR"             },    /* 4 */
+
+    { "change_begin",    3, "TYPE TABLE INDIRECT"  },    /* 5 */
+    { "change_int64",    3, "[new|old] ICOL VALUE" },    /* 6 */
+    { "change_null",     2, "[new|old] ICOL"       },    /* 7 */
+    { "change_double",   3, "[new|old] ICOL VALUE" },    /* 8 */
+    { "change_text",     3, "[new|old] ICOL VALUE" },    /* 9 */
+    { "change_blob",     3, "[new|old] ICOL VALUE" },    /* 10 */
+    { "change_finish",   1, "BDISCARD"             },    /* 11 */
+    { "change_finishne", 1, "BDISCARD"             },    /* 12 */
+
+    { "config",          2, "OPTION INTVAL"        },    /* 13 */
+    { "change_text-1",   3, "[new|old] ICOL VALUE" },    /* 14 */
+    { "change_begin_ne", 3, "TYPE TABLE INDIRECT"  },    /* 15 */
+    { 0, 0, 0 }
  };
  int rc = TCL_OK;
  int iSub = 0;
@@ -1610,9 +1656,10 @@ static int SQLITE_TCLAPI test_changegroup_cmd(

    case 1: {      /* add */
      Tcl_Size nByte = 0;
-      const u8 *aByte = Tcl_GetByteArrayFromObj(objv[2], &nByte);
-      rc = sqlite3changegroup_add(p->pGrp, (int)nByte, (void*)aByte);
+      void *aByte = testGetByteArrayFromObj(objv[2], &nByte);
+      rc = sqlite3changegroup_add(p->pGrp, (int)nByte, aByte);
      if( rc!=SQLITE_OK ) rc = test_session_error(interp, rc, 0);
+      free(aByte);
      break;
    };

@@ -1647,6 +1694,196 @@ static int SQLITE_TCLAPI test_changegroup_cmd(
      break;
    };

+    case 15:        /* change_beginne */
+    case 5: {       /* change_begin */
+      struct ChangeType {
+        const char *zType;
+        int eType;
+      } aType[] = {
+        { "INSERT", SQLITE_INSERT },
+        { "UPDATE", SQLITE_UPDATE },
+        { "DELETE", SQLITE_DELETE },
+        { 0, 0 }
+      };
+      int eType = 0;
+      const char *zTab = 0;
+      int bIndirect;
+      int iIdx = 0;
+      char *zErr = 0;
+      char **pz = ((iSub==5) ? &zErr : 0);
+
+      if( TCL_OK!=Tcl_GetIntFromObj(0, objv[2], &eType) ){
+        rc = Tcl_GetIndexFromObjStruct(
+            interp, objv[2], aType, sizeof(aType[0]), "TYPE", 0, &iIdx
+        );
+        if( rc!=TCL_OK ) return rc;
+        eType = aType[iIdx].eType;
+      }
+      zTab = Tcl_GetString(objv[3]);
+      if( Tcl_GetBooleanFromObj(interp, objv[4], &bIndirect) ){
+        return TCL_ERROR;
+      }
+
+      rc = sqlite3changegroup_change_begin(p->pGrp, eType, zTab, bIndirect, pz);
+      assert( zErr==0 || rc!=SQLITE_OK );
+      if( rc!=SQLITE_OK ){
+        rc = test_session_error(interp, rc, zErr);
+      }
+
+      break;
+    }
+
+    case 6: {      /* change_int64 */
+      int bNew = 0;
+      int iCol = 0;
+      sqlite3_int64 iVal = 0;
+      if( TCL_OK!=testGetNewOrOld(interp, objv[2], &bNew)
+       || TCL_OK!=Tcl_GetIntFromObj(interp, objv[3], &iCol)
+       || TCL_OK!=Tcl_GetWideIntFromObj(interp, objv[4], &iVal)
+      ){
+        rc = TCL_ERROR;
+      }else{
+        rc = sqlite3changegroup_change_int64(p->pGrp, bNew, iCol, iVal);
+        if( rc!=SQLITE_OK ){
+          rc = test_session_error(interp, rc, 0);
+        }
+      }
+      break;
+    }
+
+    case 7: {      /* change_null */
+      int bNew = 0;
+      int iCol = 0;
+      if( TCL_OK!=testGetNewOrOld(interp, objv[2], &bNew)
+       || TCL_OK!=Tcl_GetIntFromObj(interp, objv[3], &iCol)
+      ){
+        rc = TCL_ERROR;
+      }else{
+        rc = sqlite3changegroup_change_null(p->pGrp, bNew, iCol);
+        if( rc!=SQLITE_OK ){
+          rc = test_session_error(interp, rc, 0);
+        }
+      }
+      break;
+    }
+
+    case 8: {      /* change_double */
+      int bNew = 0;
+      int iCol = 0;
+      double rVal = 0;
+      if( TCL_OK!=testGetNewOrOld(interp, objv[2], &bNew)
+       || TCL_OK!=Tcl_GetIntFromObj(interp, objv[3], &iCol)
+       || TCL_OK!=Tcl_GetDoubleFromObj(interp, objv[4], &rVal)
+      ){
+        rc = TCL_ERROR;
+      }else{
+        rc = sqlite3changegroup_change_double(p->pGrp, bNew, iCol, rVal);
+        if( rc!=SQLITE_OK ){
+          rc = test_session_error(interp, rc, 0);
+        }
+      }
+      break;
+    }
+
+    case 9: {      /* change_text */
+      int bNew = 0;
+      int iCol = 0;
+      if( TCL_OK!=testGetNewOrOld(interp, objv[2], &bNew)
+       || TCL_OK!=Tcl_GetIntFromObj(interp, objv[3], &iCol)
+      ){
+        rc = TCL_ERROR;
+      }else{
+        Tcl_Size nVal = 0;
+        const char *pVal = Tcl_GetStringFromObj(objv[4], &nVal);
+        rc = sqlite3changegroup_change_text(p->pGrp, bNew, iCol, pVal, nVal);
+        if( rc!=SQLITE_OK ){
+          rc = test_session_error(interp, rc, 0);
+        }
+      }
+      break;
+    }
+
+    case 10: {      /* change_blob */
+      int bNew = 0;
+      int iCol = 0;
+      if( TCL_OK!=testGetNewOrOld(interp, objv[2], &bNew)
+       || TCL_OK!=Tcl_GetIntFromObj(interp, objv[3], &iCol)
+      ){
+        rc = TCL_ERROR;
+      }else{
+        Tcl_Size nVal = 0;
+        const u8 *pVal = Tcl_GetByteArrayFromObj(objv[4], &nVal);
+        rc = sqlite3changegroup_change_blob(p->pGrp, bNew, iCol, pVal, nVal);
+        if( rc!=SQLITE_OK ){
+          rc = test_session_error(interp, rc, 0);
+        }
+      }
+      break;
+    }
+
+    case 12:        /* change_finishne */
+    case 11: {      /* change_finish */
+      int bDiscard = 0;
+      if( TCL_OK!=Tcl_GetBooleanFromObj(interp, objv[2], &bDiscard) ){
+        rc = TCL_ERROR;
+      }else{
+        char *zErr = 0;
+        char **pz = &zErr;
+        if( iSub==12 ) pz = 0;
+        rc = sqlite3changegroup_change_finish(p->pGrp, bDiscard, pz);
+        if( rc!=SQLITE_OK ){
+          rc = test_session_error(interp, rc, zErr);
+        }
+      }
+      break;
+    }
+
+    case 13: {      /* config */
+      struct OptionName {
+        const char *zOpt;
+        int op;
+      } aOp[] = {
+        { "patchset", SQLITE_CHANGEGROUP_CONFIG_PATCHSET },
+        { 0, 0 }
+      };
+      int iIdx = 0;
+      int iArg = 0;
+      rc = Tcl_GetIndexFromObjStruct(
+          interp, objv[2], aOp, sizeof(aOp[0]), "option", 0, &iIdx
+      );
+      if( rc==TCL_OK 
+       && (rc = Tcl_GetIntFromObj(interp, objv[3], &iArg))==TCL_OK 
+      ){
+        int op = aOp[iIdx].op;
+        void *pArg = (void*)&iArg;
+
+        rc = sqlite3changegroup_config(p->pGrp, op, pArg);
+        if( rc!=SQLITE_OK ){
+          rc = test_session_error(interp, rc, 0);
+        }else{
+          Tcl_SetObjResult(interp, Tcl_NewIntObj(iArg));
+        }
+      }
+      break;
+    }
+
+    case 14: {      /* change_text-1 */
+      int bNew = 0;
+      int iCol = 0;
+      if( TCL_OK!=testGetNewOrOld(interp, objv[2], &bNew)
+       || TCL_OK!=Tcl_GetIntFromObj(interp, objv[3], &iCol)
+      ){
+        rc = TCL_ERROR;
+      }else{
+        const char *pVal = Tcl_GetString(objv[4]);
+        rc = sqlite3changegroup_change_text(p->pGrp, bNew, iCol, pVal, -1);
+        if( rc!=SQLITE_OK ){
+          rc = test_session_error(interp, rc, 0);
+        }
+      }
+      break;
+    }
+
    default: {     /* delete */
      assert( iSub==3 );
      Tcl_DeleteCommand(interp, Tcl_GetString(objv[0]));
@@ -352,7 +352,7 @@ else
    SQLITE_C_IS_SEE = 0
  else
    SQLITE_C_IS_SEE = 1
-    $(info This is an SEE build)
+    $(info $(emo.lock)$(emo.lock)$(emo.lock) This is an SEE build $(emo.lock)$(emo.lock)$(emo.lock))
  endif
 endif

@@ -566,12 +566,14 @@ WASM_CUSTOM_INSTANTIATE = 1
 # -D... flags which should be included in all invocations should be
 # appended to $(b.c-pp.target.flags).
 #
-bin.c-pp = ./c-pp-lite
-$(bin.c-pp): c-pp-lite.c $(sqlite3.c) $(MAKEFILE)
-	$(CC) -O0 -o $@ c-pp-lite.c $(sqlite3.c) '-DCMPP_DEFAULT_DELIM="//#"' -I$(dir.top) \
+bin.c-pp = ./c-pp
+$(bin.c-pp): libcmpp.c $(sqlite3.c) $(MAKEFILE)
+	$(CC) -O0 -o $@ libcmpp.c $(dir.top)/sqlite3.c -I$(dir.top) \
 		-DSQLITE_OMIT_LOAD_EXTENSION -DSQLITE_OMIT_DEPRECATED -DSQLITE_OMIT_UTF16 \
 		-DSQLITE_OMIT_SHARED_CACHE -DSQLITE_OMIT_WAL -DSQLITE_THREADSAFE=0 \
-		-DSQLITE_TEMP_STORE=3
+		-DSQLITE_TEMP_STORE=3 \
+		'-DCMPP_DEFAULT_DELIM="//#"' -DCMPP_MAIN -DCMPP_OMIT_D_MODULE \
+		-DCMPP_OMIT_D_PIPE
 DISTCLEAN_FILES += $(bin.c-pp)
 b.c-pp.target.flags ?=
 ifeq (1,$(SQLITE_C_IS_SEE))
@@ -911,8 +913,10 @@ ifeq (0,$(wasm-bare-bones))
  sqlite3-api.jses += $(dir.api)/sqlite3-vtab-helper.c-pp.js
 endif
 sqlite3-api.jses += $(dir.api)/sqlite3-vfs-kvvfs.c-pp.js
+sqlite3-api.jses += $(dir.api)/opfs-common-shared.c-pp.js
 sqlite3-api.jses += $(dir.api)/sqlite3-vfs-opfs.c-pp.js
 sqlite3-api.jses += $(dir.api)/sqlite3-vfs-opfs-sahpool.c-pp.js
+sqlite3-api.jses += $(dir.api)/sqlite3-vfs-opfs-wl.c-pp.js

 # Parallel builds can fail if $(sqlite3-license-version.js) is not
 # created early enough, so make all files in $(sqlite-api.jses) except
@@ -1182,13 +1186,13 @@ all: demos
 #######################################################################

 #
-# "SOAP" is a static file which is not part of the amalgamation but
-# gets copied into the build output folder and into each of the fiddle
-# builds.
+# "SOAP" is not part of the amalgamation but gets copied into the
+# build output folder and into each of the fiddle builds.
 #
 sqlite3.ext.js += $(dir.dout)/sqlite3-opfs-async-proxy.js
-$(dir.dout)/sqlite3-opfs-async-proxy.js: $(dir.api)/sqlite3-opfs-async-proxy.js
-	@$(call b.cp,@,$<,$@)
+$(eval $(call b.c-pp.target,soap,\
+  $(dir.api)/sqlite3-opfs-async-proxy.c-pp.js,\
+  $(dir.dout)/sqlite3-opfs-async-proxy.js))

 #
 # Add a dep of $(sqlite3.ext.js) on every individual build's JS file.
@@ -13,6 +13,7 @@ _sqlite3_bind_parameter_index
 _sqlite3_bind_parameter_name
 _sqlite3_bind_pointer
 _sqlite3_bind_text
+_sqlite3_bind_zeroblob
 _sqlite3_busy_handler
 _sqlite3_busy_timeout
 _sqlite3_cancel_auto_extension
@@ -225,14 +226,14 @@ _sqlite3session_object_config
 _sqlite3session_patchset
 _sqlite3session_patchset_strm
 _sqlite3session_table_filter
-//#endif not bare-bones
+//#/if not bare-bones
 //#if enable-see
 _sqlite3_key
 _sqlite3_key_v2
 _sqlite3_rekey
 _sqlite3_rekey_v2
 _sqlite3_activate_see
-//#endif enable-see
+//#/if enable-see
 //#if fiddle
 _fiddle_db_arg
 _fiddle_db_filename
@@ -244,4 +245,5 @@ _fiddle_reset_db
 _fiddle_db_handle
 _fiddle_db_vfs
 _fiddle_export_db
-//#endif fiddle
+_fiddle_get_prompt
+//#/if fiddle
@@ -14,7 +14,7 @@
 */
 //#if target:es6-module
 const toExportForESM =
-//#endif
+//#/if
 (function(){
  //console.warn("this is extern-post-js");
  /**
@@ -97,7 +97,7 @@ const toExportForESM =
        //console.warn("sqlite3InitModule() returning E-module.",EmscriptenModule);
        return EmscriptenModule;
      }
-//#endif
+//#/if
      return s;
    }).catch((e)=>{
      console.error("Exception loading sqlite3 module:",e);
@@ -128,10 +128,10 @@ const toExportForESM =
  }
  /* AMD modules get injected in a way we cannot override,
     so we can't handle those here. */
-//#endif // !target:es6-module
+//#/if // !target:es6-module
  return sIM;
 })();
 //#if target:es6-module
 sqlite3InitModule = toExportForESM;
 export default sqlite3InitModule;
-//#endif
+//#/if
@@ -0,0 +1,191 @@
+//#if 0
+/**
+   This file is for preprocessor #include into the "opfs" and
+   "opfs-wl" impls, as well as their async-proxy part. It must be
+   inlined in those files, as opposed to being a shared copy in the
+   library, because (A) the async proxy does not load the library and
+   (B) it references an object which is local to each of those files
+   but which has a 99% identical structure for each.
+*/
+//#/if
+//#// vfs.metrics.enable is a refactoring crutch.
+//#define vfs.metrics.enable 0
+const initS11n = function(){
+  /**
+     This proxy de/serializes cross-thread function arguments and
+     output-pointer values via the state.sabIO SharedArrayBuffer,
+     using the region defined by (state.sabS11nOffset,
+     state.sabS11nOffset + state.sabS11nSize]. Only one dataset is
+     recorded at a time.
+
+     This is not a general-purpose format. It only supports the
+     range of operations, and data sizes, needed by the
+     sqlite3_vfs and sqlite3_io_methods operations. Serialized
+     data are transient and this serialization algorithm may
+     change at any time.
+
+     The data format can be succinctly summarized as:
+
+     Nt...Td...D
+
+     Where:
+
+     - N = number of entries (1 byte)
+
+     - t = type ID of first argument (1 byte)
+
+     - ...T = type IDs of the 2nd and subsequent arguments (1 byte
+     each).
+
+     - d = raw bytes of first argument (per-type size).
+
+     - ...D = raw bytes of the 2nd and subsequent arguments (per-type
+     size).
+
+     All types except strings have fixed sizes. Strings are stored
+     using their TextEncoder/TextDecoder representations. It would
+     arguably make more sense to store them as Int16Arrays of
+     their JS character values, but how best/fastest to get that
+     in and out of string form is an open point. Initial
+     experimentation with that approach did not gain us any speed.
+
+     Historical note: this impl was initially about 1% this size by
+     using using JSON.stringify/parse(), but using fit-to-purpose
+     serialization saves considerable runtime.
+  */
+  if(state.s11n) return state.s11n;
+  const textDecoder = new TextDecoder(),
+        textEncoder = new TextEncoder('utf-8'),
+        viewU8 = new Uint8Array(state.sabIO, state.sabS11nOffset, state.sabS11nSize),
+        viewDV = new DataView(state.sabIO, state.sabS11nOffset, state.sabS11nSize);
+  state.s11n = Object.create(null);
+  /* Only arguments and return values of these types may be
+     serialized. This covers the whole range of types needed by the
+     sqlite3_vfs API. */
+  const TypeIds = Object.create(null);
+  TypeIds.number  = { id: 1, size: 8, getter: 'getFloat64', setter: 'setFloat64' };
+  TypeIds.bigint  = { id: 2, size: 8, getter: 'getBigInt64', setter: 'setBigInt64' };
+  TypeIds.boolean = { id: 3, size: 4, getter: 'getInt32', setter: 'setInt32' };
+  TypeIds.string =  { id: 4 };
+
+  const getTypeId = (v)=>(
+    TypeIds[typeof v]
+      || toss("Maintenance required: this value type cannot be serialized.",v)
+  );
+  const getTypeIdById = (tid)=>{
+    switch(tid){
+      case TypeIds.number.id: return TypeIds.number;
+      case TypeIds.bigint.id: return TypeIds.bigint;
+      case TypeIds.boolean.id: return TypeIds.boolean;
+      case TypeIds.string.id: return TypeIds.string;
+      default: toss("Invalid type ID:",tid);
+    }
+  };
+
+  /**
+     Returns an array of the deserialized state stored by the most
+     recent serialize() operation (from this thread or the
+     counterpart thread), or null if the serialization buffer is
+     empty.  If passed a truthy argument, the serialization buffer
+     is cleared after deserialization.
+  */
+  state.s11n.deserialize = function(clear=false){
+//#if vfs.metrics.enable
+    ++metrics.s11n.deserialize.count;
+//#/if
+    const t = performance.now();
+    const argc = viewU8[0];
+    const rc = argc ? [] : null;
+    if(argc){
+      const typeIds = [];
+      let offset = 1, i, n, v;
+      for(i = 0; i < argc; ++i, ++offset){
+        typeIds.push(getTypeIdById(viewU8[offset]));
+      }
+      for(i = 0; i < argc; ++i){
+        const t = typeIds[i];
+        if(t.getter){
+          v = viewDV[t.getter](offset, state.littleEndian);
+          offset += t.size;
+        }else{/*String*/
+          n = viewDV.getInt32(offset, state.littleEndian);
+          offset += 4;
+          v = textDecoder.decode(viewU8.slice(offset, offset+n));
+          offset += n;
+        }
+        rc.push(v);
+      }
+    }
+    if(clear) viewU8[0] = 0;
+    //log("deserialize:",argc, rc);
+//#if vfs.metrics.enable
+    metrics.s11n.deserialize.time += performance.now() - t;
+//#/if
+    return rc;
+  };
+
+  /**
+     Serializes all arguments to the shared buffer for consumption
+     by the counterpart thread.
+
+     This routine is only intended for serializing OPFS VFS
+     arguments and (in at least one special case) result values,
+     and the buffer is sized to be able to comfortably handle
+     those.
+
+     If passed no arguments then it zeroes out the serialization
+     state.
+  */
+  state.s11n.serialize = function(...args){
+    const t = performance.now();
+//#if vfs.metrics.enable
+    ++metrics.s11n.serialize.count;
+//#/if
+    if(args.length){
+      //log("serialize():",args);
+      const typeIds = [];
+      let i = 0, offset = 1;
+      viewU8[0] = args.length & 0xff /* header = # of args */;
+      for(; i < args.length; ++i, ++offset){
+        /* Write the TypeIds.id value into the next args.length
+           bytes. */
+        typeIds.push(getTypeId(args[i]));
+        viewU8[offset] = typeIds[i].id;
+      }
+      for(i = 0; i < args.length; ++i) {
+        /* Deserialize the following bytes based on their
+           corresponding TypeIds.id from the header. */
+        const t = typeIds[i];
+        if(t.setter){
+          viewDV[t.setter](offset, args[i], state.littleEndian);
+          offset += t.size;
+        }else{/*String*/
+          const s = textEncoder.encode(args[i]);
+          viewDV.setInt32(offset, s.byteLength, state.littleEndian);
+          offset += 4;
+          viewU8.set(s, offset);
+          offset += s.byteLength;
+        }
+      }
+      //log("serialize() result:",viewU8.slice(0,offset));
+    }else{
+      viewU8[0] = 0;
+    }
+//#if vfs.metrics.enable
+    metrics.s11n.serialize.time += performance.now() - t;
+//#/if
+  };
+
+//#if defined opfs-async-proxy
+  state.s11n.storeException = state.asyncS11nExceptions
+    ? ((priority,e)=>{
+      if(priority<=state.asyncS11nExceptions){
+        state.s11n.serialize([e.name,': ',e.message].join(""));
+      }
+    })
+    : ()=>{};
+//#/if
+
+  return state.s11n;
+//#undef vfs.metrics.enable
+}/*initS11n()*/;
@@ -34,6 +34,7 @@ Module.runSQLite3PostLoadInit = async function(
       - sqlite3-vtab-helper.c-pp.js => Utilities for virtual table impls
       - sqlite3-vfs-opfs.c-pp.js => OPFS VFS
       - sqlite3-vfs-opfs-sahpool.c-pp.js => OPFS SAHPool VFS
+       - sqlite3-vfs-opfs-wl.c-pp.js => WebLock-using OPFS VFS
     - post-js-footer.js          => this file's epilogue

     And all of that gets sandwiched between extern-pre-js.js and
@@ -17,9 +17,9 @@
 /**
   This file was preprocessed using:

-//#@policy error
+//#@ policy error
   @c-pp::argv@
-//#@policy off
+//#@ policy off
 */
 //#if unsupported-build
 /**
@@ -30,9 +30,10 @@
   load. It may not work properly. Only builds _directly_ targeting
   browser environments ("vanilla" JS and ESM modules) are supported
   and tested. Builds which _indirectly_ target browsers (namely
-   bundler-friendly builds) are not supported deliverables.
+   bundler-friendly builds and any node builds) are not supported
+   deliverables.
 */
-//#endif
+//#/if
 //#if not target:es6-bundler-friendly
 (function(Module){
  const sIMS =
@@ -101,7 +102,7 @@
      "result =", theFile
    );
    return theFile;
-//#endif target:es6-module
+//#/if target:es6-module
  }.bind(sIMS);

 //#if Module.instantiateWasm and not wasmfs and not target:node
@@ -147,7 +148,7 @@
          .then(finalThen)
    return loadWasm();
  }.bind(sIMS);
-//#endif Module.instantiateWasm and not wasmfs
+//#/if Module.instantiateWasm and not wasmfs
 })(Module);
-//#endif not target:es6-bundler-friendly
+//#/if not target:es6-bundler-friendly
 /* END FILE: api/pre-js.js. */
--- a/Show More
+++ b/Show More
@@ -1 +1 @@
 .52.1
 .54.0