Version 3.12.0

FossilOrigin-Name: e9bb4cf40f4971974a74468ef922bdee481c988b
Fix the multiplexor so that it does not assume that the xGetLastError method
2016-03-29 10:14:15 +00:00 · 2016-03-28 11:01:54 +00:00 · 2016-03-26 15:36:36 +00:00 · 2016-03-26 14:41:13 +00:00 · 2016-03-26 13:26:35 +00:00 · 2016-03-25 20:10:20 +00:00
66 changed files with 1872 additions and 552 deletions
@@ -5,14 +5,17 @@
 <body bgcolor=white>
 <h1 align=center>The Lemon Parser Generator</h1>

-<p>Lemon is an LALR(1) parser generator for C or C++.  
-It does the same job as ``bison'' and ``yacc''.
-But lemon is not another bison or yacc clone.  It
+<p>Lemon is an LALR(1) parser generator for C.
+It does the same job as "bison" and "yacc".
+But lemon is not a bison or yacc clone.  Lemon
 uses a different grammar syntax which is designed to
-reduce the number of coding errors.  Lemon also uses a more
-sophisticated parsing engine that is faster than yacc and
-bison and which is both reentrant and thread-safe.
-Furthermore, Lemon implements features that can be used
+reduce the number of coding errors.  Lemon also uses a
+parsing engine that is faster than yacc and
+bison and which is both reentrant and threadsafe.
+(Update: Since the previous sentence was written, bison
+has also been updated so that it too can generate a
+reentrant and threadsafe parser.)
+Lemon also implements features that can be used
 to eliminate resource leaks, making is suitable for use
 in long-running programs such as graphical user interfaces
 or embedded controllers.</p>
@@ -44,18 +47,18 @@ one and three files of outputs.
    automaton.
 </ul>
 By default, all three of these output files are generated.
-The header file is suppressed if the ``-m'' command-line option is
-used and the report file is omitted when ``-q'' is selected.</p>
+The header file is suppressed if the "-m" command-line option is
+used and the report file is omitted when "-q" is selected.</p>

-<p>The grammar specification file uses a ``.y'' suffix, by convention.
+<p>The grammar specification file uses a ".y" suffix, by convention.
 In the examples used in this document, we'll assume the name of the
-grammar file is ``gram.y''.  A typical use of Lemon would be the
+grammar file is "gram.y".  A typical use of Lemon would be the
 following command:
 <pre>
   lemon gram.y
 </pre>
-This command will generate three output files named ``gram.c'',
-``gram.h'' and ``gram.out''.
+This command will generate three output files named "gram.c",
+"gram.h" and "gram.out".
 The first is C code to implement the parser.  The second
 is the header file that defines numerical values for all
 terminal symbols, and the last is the report that explains
@@ -71,39 +74,35 @@ with a brief explanation of what each does by typing
 </pre>
 As of this writing, the following command-line options are supported:
 <ul>
-<li><tt>-b</tt>
-<li><tt>-c</tt>
-<li><tt>-g</tt>
-<li><tt>-m</tt>
-<li><tt>-q</tt>
-<li><tt>-s</tt>
-<li><tt>-x</tt>
+<li><b>-b</b>
+Show only the basis for each parser state in the report file.
+<li><b>-c</b>
+Do not compress the generated action tables.
+<li><b>-D<i>name</i></b>
+Define C preprocessor macro <i>name</i>.  This macro is useable by
+"%ifdef" lines in the grammar file.
+<li><b>-g</b>
+Do not generate a parser.  Instead write the input grammar to standard
+output with all comments, actions, and other extraneous text removed.
+<li><b>-l</b>
+Omit "#line" directives int the generated parser C code.
+<li><b>-m</b>
+Cause the output C source code to be compatible with the "makeheaders"
+program. 
+<li><b>-p</b>
+Display all conflicts that are resolved by 
+<a href='#precrules'>precedence rules</a>.
+<li><b>-q</b>
+Suppress generation of the report file.
+<li><b>-r</b>
+Do not sort or renumber the parser states as part of optimization.
+<li><b>-s</b>
+Show parser statistics before existing.
+<li><b>-T<i>file</i></b>
+Use <i>file</i> as the template for the generated C-code parser implementation.
+<li><b>-x</b>
+Print the Lemon version number.
 </ul>
-The ``-b'' option reduces the amount of text in the report file by
-printing only the basis of each parser state, rather than the full
-configuration.
-The ``-c'' option suppresses action table compression.  Using -c
-will make the parser a little larger and slower but it will detect
-syntax errors sooner.
-The ``-g'' option causes no output files to be generated at all.
-Instead, the input grammar file is printed on standard output but
-with all comments, actions and other extraneous text deleted.  This
-is a useful way to get a quick summary of a grammar.
-The ``-m'' option causes the output C source file to be compatible
-with the ``makeheaders'' program.
-Makeheaders is a program that automatically generates header files
-from C source code.  When the ``-m'' option is used, the header
-file is not output since the makeheaders program will take care
-of generated all header files automatically.
-The ``-q'' option suppresses the report file.
-Using ``-s'' causes a brief summary of parser statistics to be
-printed.  Like this:
-<pre>
-   Parser statistics: 74 terminals, 70 nonterminals, 179 rules
-                      340 states, 2026 parser table entries, 0 conflicts
-</pre>
-Finally, the ``-x'' option causes Lemon to print its version number
-and then stops without attempting to read the grammar or generate a parser.</p>

 <h3>The Parser Interface</h3>

@@ -121,12 +120,12 @@ A new parser is created as follows:
 </pre>
 The ParseAlloc() routine allocates and initializes a new parser and
 returns a pointer to it.
-The actual data structure used to represent a parser is opaque --
+The actual data structure used to represent a parser is opaque &mdash;
 its internal structure is not visible or usable by the calling routine.
 For this reason, the ParseAlloc() routine returns a pointer to void
 rather than a pointer to some particular structure.
 The sole argument to the ParseAlloc() routine is a pointer to the
-subroutine used to allocate memory.  Typically this means ``malloc()''.</p>
+subroutine used to allocate memory.  Typically this means malloc().</p>

 <p>After a program is finished using a parser, it can reclaim all
 memory allocated by that parser by calling
@@ -151,13 +150,13 @@ type of the next token in the data stream.
 There is one token type for each terminal symbol in the grammar.
 The gram.h file generated by Lemon contains #define statements that
 map symbolic terminal symbol names into appropriate integer values.
-(A value of 0 for the second argument is a special flag to the
-parser to indicate that the end of input has been reached.)
+A value of 0 for the second argument is a special flag to the
+parser to indicate that the end of input has been reached.
 The third argument is the value of the given token.  By default,
 the type of the third argument is integer, but the grammar will
 usually redefine this type to be some kind of structure.
 Typically the second argument will be a broad category of tokens
-such as ``identifier'' or ``number'' and the third argument will
+such as "identifier" or "number" and the third argument will
 be the name of the identifier or the value of the number.</p>

 <p>The Parse() function may have either three or four arguments,
@@ -193,7 +192,7 @@ following:
 </pre>
 This example shows a user-written routine that parses a file of
 text and returns a pointer to the parse tree.
-(We've omitted all error-handling from this example to keep it
+(All error-handling code is omitted from this example to keep it
 simple.)
 We assume the existence of some kind of tokenizer which is created
 using TokenizerCreate() on line 8 and deleted by TokenizerFree()
@@ -287,7 +286,7 @@ tokens) and it honors the same commenting conventions as C and C++.</p>
 <h3>Terminals and Nonterminals</h3>

 <p>A terminal symbol (token) is any string of alphanumeric
-and underscore characters
+and/or underscore characters
 that begins with an upper case letter.
 A terminal can contain lowercase letters after the first character,
 but the usual convention is to make terminals all upper case.
@@ -314,7 +313,7 @@ must have alphanumeric names.</p>
 <p>The main component of a Lemon grammar file is a sequence of grammar
 rules.
 Each grammar rule consists of a nonterminal symbol followed by
-the special symbol ``::='' and then a list of terminals and/or nonterminals.
+the special symbol "::=" and then a list of terminals and/or nonterminals.
 The rule is terminated by a period.
 The list of terminals and nonterminals on the right-hand side of the
 rule can be empty.
@@ -330,9 +329,9 @@ A typical sequence of grammar rules might look something like this:
 </pre>
 </p>

-<p>There is one non-terminal in this example, ``expr'', and five
-terminal symbols or tokens: ``PLUS'', ``TIMES'', ``LPAREN'',
-``RPAREN'' and ``VALUE''.</p>
+<p>There is one non-terminal in this example, "expr", and five
+terminal symbols or tokens: "PLUS", "TIMES", "LPAREN",
+"RPAREN" and "VALUE".</p>

 <p>Like yacc and bison, Lemon allows the grammar to specify a block
 of C code that will be executed whenever a grammar rule is reduced
@@ -348,15 +347,15 @@ For example:

 <p>In order to be useful, grammar actions must normally be linked to
 their associated grammar rules.
-In yacc and bison, this is accomplished by embedding a ``$$'' in the
+In yacc and bison, this is accomplished by embedding a "$$" in the
 action to stand for the value of the left-hand side of the rule and
-symbols ``$1'', ``$2'', and so forth to stand for the value of
+symbols "$1", "$2", and so forth to stand for the value of
 the terminal or nonterminal at position 1, 2 and so forth on the
 right-hand side of the rule.
 This idea is very powerful, but it is also very error-prone.  The
 single most common source of errors in a yacc or bison grammar is
 to miscount the number of symbols on the right-hand side of a grammar
-rule and say ``$7'' when you really mean ``$8''.</p>
+rule and say "$7" when you really mean "$8".</p>

 <p>Lemon avoids the need to count grammar symbols by assigning symbolic
 names to each symbol in a grammar rule and then using those symbolic
@@ -386,7 +385,7 @@ For example, the rule
 <pre>
  expr(A) ::= expr(B) PLUS expr(C).  { A = B; }
 </pre>
-will generate an error because the linking symbol ``C'' is used
+will generate an error because the linking symbol "C" is used
 in the grammar rule but not in the reduce action.</p>

 <p>The Lemon notation for linking grammar rules to reduce actions
@@ -394,6 +393,7 @@ also facilitates the use of destructors for reclaiming memory
 allocated by the values of terminals and nonterminals on the
 right-hand side of a rule.</p>

+<a name='precrules'></a>
 <h3>Precedence Rules</h3>

 <p>Lemon resolves parsing ambiguities in exactly the same way as
@@ -405,7 +405,10 @@ whichever rule comes first in the grammar file.</p>
 yacc and bison, Lemon allows a measure of control 
 over the resolution of paring conflicts using precedence rules.
 A precedence value can be assigned to any terminal symbol
-using the %left, %right or %nonassoc directives.  Terminal symbols
+using the 
+<a href='#pleft'>%left</a>,
+<a href='#pright'>%right</a> or
+<a href='#pnonassoc'>%nonassoc</a> directives.  Terminal symbols
 mentioned in earlier directives have a lower precedence that
 terminal symbols mentioned in later directives.  For example:</p>

@@ -525,7 +528,11 @@ other than that, the order of directives in Lemon is arbitrary.</p>
 <li><tt>%default_destructor</tt>
 <li><tt>%default_type</tt>
 <li><tt>%destructor</tt>
+<li><tt>%endif</tt>
 <li><tt>%extra_argument</tt>
+<li><tt>%fallback</tt>
+<li><tt>%ifdef</tt>
+<li><tt>%ifndef</tt>
 <li><tt>%include</tt>
 <li><tt>%left</tt>
 <li><tt>%name</tt>
@@ -537,49 +544,57 @@ other than that, the order of directives in Lemon is arbitrary.</p>
 <li><tt>%stack_size</tt>
 <li><tt>%start_symbol</tt>
 <li><tt>%syntax_error</tt>
+<li><tt>%token_class</tt>
 <li><tt>%token_destructor</tt>
 <li><tt>%token_prefix</tt>
 <li><tt>%token_type</tt>
 <li><tt>%type</tt>
+<li><tt>%wildcard</tt>
 </ul>
 Each of these directives will be described separately in the
 following sections:</p>

+<a name='pcode'></a>
 <h4>The <tt>%code</tt> directive</h4>

-<p>The %code directive is used to specify addition C/C++ code that
+<p>The %code directive is used to specify addition C code that
 is added to the end of the main output file.  This is similar to
-the %include directive except that %include is inserted at the
-beginning of the main output file.</p>
+the <a href='#pinclude'>%include</a> directive except that %include
+is inserted at the beginning of the main output file.</p>

 <p>%code is typically used to include some action routines or perhaps
-a tokenizer as part of the output file.</p>
+a tokenizer or even the "main()" function 
+as part of the output file.</p>

+<a name='default_destructor'></a>
 <h4>The <tt>%default_destructor</tt> directive</h4>

 <p>The %default_destructor directive specifies a destructor to 
 use for non-terminals that do not have their own destructor
 specified by a separate %destructor directive.  See the documentation
-on the %destructor directive below for additional information.</p>
+on the <a name='#destructor'>%destructor</a> directive below for
+additional information.</p>

 <p>In some grammers, many different non-terminal symbols have the
 same datatype and hence the same destructor.  This directive is
 a convenience way to specify the same destructor for all those
 non-terminals using a single statement.</p>

+<a name='default_type'></a>
 <h4>The <tt>%default_type</tt> directive</h4>

 <p>The %default_type directive specifies the datatype of non-terminal
 symbols that do no have their own datatype defined using a separate
-%type directive.  See the documentation on %type below for addition
-information.</p>
+<a href='#ptype'>%type</a> directive.  
+</p>

+<a name='destructor'></a>
 <h4>The <tt>%destructor</tt> directive</h4>

 <p>The %destructor directive is used to specify a destructor for
 a non-terminal symbol.
-(See also the %token_destructor directive which is used to
-specify a destructor for terminal symbols.)</p>
+(See also the <a href='#token_destructor'>%token_destructor</a>
+directive which is used to specify a destructor for terminal symbols.)</p>

 <p>A non-terminal's destructor is called to dispose of the
 non-terminal's value whenever the non-terminal is popped from
@@ -602,26 +617,25 @@ or other resources held by that non-terminal.</p>
 </pre>
 This example is a bit contrived but it serves to illustrate how
 destructors work.  The example shows a non-terminal named
-``nt'' that holds values of type ``void*''.  When the rule for
-an ``nt'' reduces, it sets the value of the non-terminal to
+"nt" that holds values of type "void*".  When the rule for
+an "nt" reduces, it sets the value of the non-terminal to
 space obtained from malloc().  Later, when the nt non-terminal
 is popped from the stack, the destructor will fire and call
 free() on this malloced space, thus avoiding a memory leak.
-(Note that the symbol ``$$'' in the destructor code is replaced
+(Note that the symbol "$$" in the destructor code is replaced
 by the value of the non-terminal.)</p>

 <p>It is important to note that the value of a non-terminal is passed
 to the destructor whenever the non-terminal is removed from the
 stack, unless the non-terminal is used in a C-code action.  If
 the non-terminal is used by C-code, then it is assumed that the
-C-code will take care of destroying it if it should really
-be destroyed.  More commonly, the value is used to build some
+C-code will take care of destroying it.
+More commonly, the value is used to build some
 larger structure and we don't want to destroy it, which is why
 the destructor is not called in this circumstance.</p>

-<p>By appropriate use of destructors, it is possible to
-build a parser using Lemon that can be used within a long-running
-program, such as a GUI, that will not leak memory or other resources.
+<p>Destructors help avoid memory leaks by automatically freeing
+allocated objects when they go out of scope.
 To do the same using yacc or bison is much more difficult.</p>

 <a name="extraarg"></a>
@@ -638,17 +652,66 @@ and so forth.  For example, if the grammar file contains:</p>
 </pre></p>

 <p>Then the Parse() function generated will have an 4th parameter
-of type ``MyStruct*'' and all action routines will have access to
-a variable named ``pAbc'' that is the value of the 4th parameter
+of type "MyStruct*" and all action routines will have access to
+a variable named "pAbc" that is the value of the 4th parameter
 in the most recent call to Parse().</p>

+<a name='pfallback'></a>
+<h4>The <tt>%fallback</tt> directive</h4>
+
+<p>The %fallback directive specifies an alternative meaning for one
+or more tokens.  The alternative meaning is tried if the original token
+would have generated a syntax error.
+
+<p>The %fallback directive was added to support robust parsing of SQL
+syntax in <a href="https://www.sqlite.org/">SQLite</a>.
+The SQL language contains a large assortment of keywords, each of which
+appears as a different token to the language parser.  SQL contains so
+many keywords, that it can be difficult for programmers to keep up with
+them all.  Programmers will, therefore, sometimes mistakenly use an
+obscure language keyword for an identifier.  The %fallback directive
+provides a mechanism to tell the parser:  "If you are unable to parse
+this keyword, try treating it as an identifier instead."
+
+<p>The syntax of %fallback is as follows:
+
+<blockquote>
+<tt>%fallback</tt>  <i>ID</i> <i>TOKEN...</i> <b>.</b>
+</blockquote>
+
+<p>In words, the %fallback directive is followed by a list of token names
+terminated by a period.  The first token name is the fallback token - the
+token to which all the other tokens fall back to.  The second and subsequent
+arguments are tokens which fall back to the token identified by the first
+argument.
+
+<a name='pifdef'></a>
+<h4>The <tt>%ifdef</tt>, <tt>%ifndef</tt>, and <tt>%endif</tt> directives.</h4>
+
+<p>The %ifdef, %ifndef, and %endif directives are similar to
+#ifdef, #ifndef, and #endif in the C-preprocessor, just not as general.
+Each of these directives must begin at the left margin.  No whitespace
+is allowed between the "%" and the directive name.
+
+<p>Grammar text in between "%ifdef MACRO" and the next nested "%endif" is
+ignored unless the "-DMACRO" command-line option is used.  Grammar text
+betwen "%ifndef MACRO" and the next nested "%endif" is included except when
+the "-DMACRO" command-line option is used.
+
+<p>Note that the argument to %ifdef and %ifndef must be a single 
+preprocessor symbol name, not a general expression.  There is no "%else"
+directive.
+
+
+<a name='pinclude'></a>
 <h4>The <tt>%include</tt> directive</h4>

 <p>The %include directive specifies C code that is included at the
 top of the generated parser.  You can include any text you want --
 the Lemon parser generator copies it blindly.  If you have multiple
-%include directives in your grammar file the value of the last
-%include directive overwrites all the others.</p.
+%include directives in your grammar file, their values are concatenated
+so that all %include code ultimately appears near the top of the
+generated parser, in the same order as it appeared in the grammer.</p>

 <p>The %include directive is very handy for getting some extra #include
 preprocessor statements at the beginning of the generated parser.
@@ -661,12 +724,13 @@ For example:</p>
 <p>This might be needed, for example, if some of the C actions in the
 grammar call functions that are prototyed in unistd.h.</p>

+<a name='pleft'></a>
 <h4>The <tt>%left</tt> directive</h4>

-The %left directive is used (along with the %right and
-%nonassoc directives) to declare precedences of terminal
-symbols.  Every terminal symbol whose name appears after
-a %left directive but before the next period (``.'') is
+The %left directive is used (along with the <a href='#pright'>%right</a> and
+<a href='#pnonassoc'>%nonassoc</a> directives) to declare precedences of 
+terminal symbols.  Every terminal symbol whose name appears after
+a %left directive but before the next period (".") is
 given the same left-associative precedence value.  Subsequent
 %left directives have higher precedence.  For example:</p>

@@ -687,10 +751,11 @@ a large amount of stack space if you make heavy use or right-associative
 operators.  For this reason, it is recommended that you use %left
 rather than %right whenever possible.</p>

+<a name='pname'></a>
 <h4>The <tt>%name</tt> directive</h4>

 <p>By default, the functions generated by Lemon all begin with the
-five-character string ``Parse''.  You can change this string to something
+five-character string "Parse".  You can change this string to something
 different using the %name directive.  For instance:</p>

 <p><pre>
@@ -709,16 +774,19 @@ The %name directive allows you to generator two or more different
 parsers and link them all into the same executable.
 </p>

+<a name='pnonassoc'></a>
 <h4>The <tt>%nonassoc</tt> directive</h4>

 <p>This directive is used to assign non-associative precedence to
-one or more terminal symbols.  See the section on precedence rules
-or on the %left directive for additional information.</p>
+one or more terminal symbols.  See the section on 
+<a href='#precrules'>precedence rules</a>
+or on the <a href='#pleft'>%left</a> directive for additional information.</p>

+<a name='parse_accept'></a>
 <h4>The <tt>%parse_accept</tt> directive</h4>

 <p>The %parse_accept directive specifies a block of C code that is
-executed whenever the parser accepts its input string.  To ``accept''
+executed whenever the parser accepts its input string.  To "accept"
 an input string means that the parser was able to process all tokens
 without error.</p>

@@ -730,7 +798,7 @@ without error.</p>
   }
 </pre></p>

-
+<a name='parse_failure'></a>
 <h4>The <tt>%parse_failure</tt> directive</h4>

 <p>The %parse_failure directive specifies a block of C code that
@@ -745,12 +813,15 @@ only invoked when parsing is unable to continue.</p>
   }
 </pre></p>

+<a name='pright'></a>
 <h4>The <tt>%right</tt> directive</h4>

 <p>This directive is used to assign right-associative precedence to
-one or more terminal symbols.  See the section on precedence rules
-or on the %left directive for additional information.</p>
+one or more terminal symbols.  See the section on 
+<a href='#precrules'>precedence rules</a>
+or on the <a href='#pleft'>%left</a> directive for additional information.</p>

+<a name='stack_overflow'></a>
 <h4>The <tt>%stack_overflow</tt> directive</h4>

 <p>The %stack_overflow directive specifies a block of C code that
@@ -779,6 +850,7 @@ Not like this:
   list ::= .
 </pre>

+<a name='stack_size'></a>
 <h4>The <tt>%stack_size</tt> directive</h4>

 <p>If stack overflow is a problem and you can't resolve the trouble
@@ -791,6 +863,7 @@ with a stack of the requested size.  The default value is 100.</p>
   %stack_size 2000
 </pre></p>

+<a name='start_symbol'></a>
 <h4>The <tt>%start_symbol</tt> directive</h4>

 <p>By default, the start-symbol for the grammar that Lemon generates
@@ -801,6 +874,7 @@ can choose a different start-symbol using the %start_symbol directive.</p>
   %start_symbol  prog
 </pre></p>

+<a name='token_destructor'></a>
 <h4>The <tt>%token_destructor</tt> directive</h4>

 <p>The %destructor directive assigns a destructor to a non-terminal
@@ -813,6 +887,7 @@ the %token_type directive) and so they use a common destructor.  Other
 than that, the token destructor works just like the non-terminal
 destructors.</p>

+<a name='token_prefix'></a>
 <h4>The <tt>%token_prefix</tt> directive</h4>

 <p>Lemon generates #defines that assign small integer constants
@@ -838,6 +913,7 @@ to cause Lemon to produce these symbols instead:
    #define TOKEN_PLUS       4
 </pre>

+<a name='token_type'></a><a name='ptype'></a>
 <h4>The <tt>%token_type</tt> and <tt>%type</tt> directives</h4>

 <p>These directives are used to specify the data types for values
@@ -853,7 +929,7 @@ token structure.  Like this:</p>
 </pre></p>

 <p>If the data type of terminals is not specified, the default value
-is ``int''.</p>
+is "int".</p>

 <p>Non-terminal symbols can each have their own data types.  Typically
 the data type  of a non-terminal is a pointer to the root of a parse-tree
@@ -874,6 +950,17 @@ non-terminal whose data type requires 1K of storage, then your 100
 entry parser stack will require 100K of heap space.  If you are willing
 and able to pay that price, fine.  You just need to know.</p>

+<a name='pwildcard'></a>
+<h4>The <tt>%wildcard</tt> directive</h4>
+
+<p>The %wildcard directive is followed by a single token name and a
+period.  This directive specifies that the identified token should 
+match any input token.
+
+<p>When the generated parser has the choice of matching an input against
+the wildcard token and some other token, the other token is always used.
+The wildcard token is only matched if there are no other alternatives.
+
 <h3>Error Processing</h3>

 <p>After extensive experimentation over several years, it has been
@@ -885,7 +972,7 @@ first invokes the code specified by the %syntax_error directive, if
 any.  It then enters its error recovery strategy.  The error recovery
 strategy is to begin popping the parsers stack until it enters a
 state where it is permitted to shift a special non-terminal symbol
-named ``error''.  It then shifts this non-terminal and continues
+named "error".  It then shifts this non-terminal and continues
 parsing.  But the %syntax_error routine will not be called again
 until at least three new tokens have been successfully shifted.</p>

@@ -894,7 +981,7 @@ is unable to shift the error symbol, then the %parse_failed routine
 is invoked and the parser resets itself to its start state, ready
 to begin parsing a new file.  This is what will happen at the very
 first syntax error, of course, if there are no instances of the 
-``error'' non-terminal in your grammar.</p>
+"error" non-terminal in your grammar.</p>

 </body>
 </html>
@@ -47,6 +47,10 @@ typedef sqlite3_uint64 u64;

 #endif

+/* Truncate very long tokens to this many bytes. Hard limit is 
+** (65536-1-1-4-9)==65521 bytes. The limiting factor is the 16-bit offset
+** field that occurs at the start of each leaf page (see fts5_index.c). */
+#define FTS5_MAX_TOKEN_SIZE 32768

 /*
 ** Maximum number of prefix indexes on single FTS5 table. This must be
@@ -480,6 +484,7 @@ int sqlite3Fts5IndexReads(Fts5Index *p);
 int sqlite3Fts5IndexReinit(Fts5Index *p);
 int sqlite3Fts5IndexOptimize(Fts5Index *p);
 int sqlite3Fts5IndexMerge(Fts5Index *p, int nMerge);
+int sqlite3Fts5IndexReset(Fts5Index *p);

 int sqlite3Fts5IndexLoadConfig(Fts5Index *p);

@@ -622,6 +627,7 @@ int sqlite3Fts5StorageDeleteAll(Fts5Storage *p);
 int sqlite3Fts5StorageRebuild(Fts5Storage *p);
 int sqlite3Fts5StorageOptimize(Fts5Storage *p);
 int sqlite3Fts5StorageMerge(Fts5Storage *p, int nMerge);
+int sqlite3Fts5StorageReset(Fts5Storage *p);

 /*
 ** End of interface to code in fts5_storage.c.
@@ -1493,6 +1493,7 @@ static int fts5ParseTokenize(

  /* If an error has already occurred, this is a no-op */
  if( pCtx->rc!=SQLITE_OK ) return pCtx->rc;
+  if( nToken>FTS5_MAX_TOKEN_SIZE ) nToken = FTS5_MAX_TOKEN_SIZE;

  if( pPhrase && pPhrase->nTerm>0 && (tflags & FTS5_TOKEN_COLOCATED) ){
    Fts5ExprTerm *pSyn;
@@ -2495,6 +2496,7 @@ static int fts5ExprPopulatePoslistsCb(

  UNUSED_PARAM2(iUnused1, iUnused2);

+  if( nToken>FTS5_MAX_TOKEN_SIZE ) nToken = FTS5_MAX_TOKEN_SIZE;
  if( (tflags & FTS5_TOKEN_COLOCATED)==0 ) p->iOff++;
  for(i=0; i<pExpr->nPhrase; i++){
    Fts5ExprTerm *pTerm;
@@ -304,6 +304,10 @@ struct Fts5Index {
  sqlite3_stmt *pIdxDeleter;      /* "DELETE FROM %_idx WHERE segid=? */
  sqlite3_stmt *pIdxSelect;
  int nRead;                      /* Total number of blocks read */
+
+  sqlite3_stmt *pDataVersion;
+  i64 iStructVersion;             /* data_version when pStruct read */
+  Fts5Structure *pStruct;         /* Current db structure (or NULL) */
 };

 struct Fts5DoclistIter {
@@ -959,6 +963,50 @@ static void fts5StructureExtendLevel(
  }
 }

+static Fts5Structure *fts5StructureReadUncached(Fts5Index *p){
+  Fts5Structure *pRet = 0;
+  Fts5Config *pConfig = p->pConfig;
+  int iCookie;                    /* Configuration cookie */
+  Fts5Data *pData;
+
+  pData = fts5DataRead(p, FTS5_STRUCTURE_ROWID);
+  if( p->rc==SQLITE_OK ){
+    /* TODO: Do we need this if the leaf-index is appended? Probably... */
+    memset(&pData->p[pData->nn], 0, FTS5_DATA_PADDING);
+    p->rc = fts5StructureDecode(pData->p, pData->nn, &iCookie, &pRet);
+    if( p->rc==SQLITE_OK && pConfig->iCookie!=iCookie ){
+      p->rc = sqlite3Fts5ConfigLoad(pConfig, iCookie);
+    }
+    fts5DataRelease(pData);
+    if( p->rc!=SQLITE_OK ){
+      fts5StructureRelease(pRet);
+      pRet = 0;
+    }
+  }
+
+  return pRet;
+}
+
+static i64 fts5IndexDataVersion(Fts5Index *p){
+  i64 iVersion = 0;
+
+  if( p->rc==SQLITE_OK ){
+    if( p->pDataVersion==0 ){
+      p->rc = fts5IndexPrepareStmt(p, &p->pDataVersion, 
+          sqlite3_mprintf("PRAGMA %Q.data_version", p->pConfig->zDb)
+          );
+      if( p->rc ) return 0;
+    }
+
+    if( SQLITE_ROW==sqlite3_step(p->pDataVersion) ){
+      iVersion = sqlite3_column_int64(p->pDataVersion, 0);
+    }
+    p->rc = sqlite3_reset(p->pDataVersion);
+  }
+
+  return iVersion;
+}
+
 /*
 ** Read, deserialize and return the structure record.
 **
@@ -971,26 +1019,49 @@ static void fts5StructureExtendLevel(
 ** is called, it is a no-op.
 */
 static Fts5Structure *fts5StructureRead(Fts5Index *p){
-  Fts5Config *pConfig = p->pConfig;
-  Fts5Structure *pRet = 0;        /* Object to return */
-  int iCookie;                    /* Configuration cookie */
-  Fts5Data *pData;

-  pData = fts5DataRead(p, FTS5_STRUCTURE_ROWID);
-  if( p->rc ) return 0;
-  /* TODO: Do we need this if the leaf-index is appended? Probably... */
-  memset(&pData->p[pData->nn], 0, FTS5_DATA_PADDING);
-  p->rc = fts5StructureDecode(pData->p, pData->nn, &iCookie, &pRet);
-  if( p->rc==SQLITE_OK && pConfig->iCookie!=iCookie ){
-    p->rc = sqlite3Fts5ConfigLoad(pConfig, iCookie);
+  if( p->pStruct==0 ){
+    p->iStructVersion = fts5IndexDataVersion(p);
+    if( p->rc==SQLITE_OK ){
+      p->pStruct = fts5StructureReadUncached(p);
+    }
  }

-  fts5DataRelease(pData);
-  if( p->rc!=SQLITE_OK ){
-    fts5StructureRelease(pRet);
-    pRet = 0;
+#if 0
+  else{
+    Fts5Structure *pTest = fts5StructureReadUncached(p);
+    if( pTest ){
+      int i, j;
+      assert_nc( p->pStruct->nSegment==pTest->nSegment );
+      assert_nc( p->pStruct->nLevel==pTest->nLevel );
+      for(i=0; i<pTest->nLevel; i++){
+        assert_nc( p->pStruct->aLevel[i].nMerge==pTest->aLevel[i].nMerge );
+        assert_nc( p->pStruct->aLevel[i].nSeg==pTest->aLevel[i].nSeg );
+        for(j=0; j<pTest->aLevel[i].nSeg; j++){
+          Fts5StructureSegment *p1 = &pTest->aLevel[i].aSeg[j];
+          Fts5StructureSegment *p2 = &p->pStruct->aLevel[i].aSeg[j];
+          assert_nc( p1->iSegid==p2->iSegid );
+          assert_nc( p1->pgnoFirst==p2->pgnoFirst );
+          assert_nc( p1->pgnoLast==p2->pgnoLast );
+        }
+      }
+      fts5StructureRelease(pTest);
+    }
+  }
+#endif
+
+  if( p->rc!=SQLITE_OK ) return 0;
+  assert( p->iStructVersion!=0 );
+  assert( p->pStruct!=0 );
+  fts5StructureRef(p->pStruct);
+  return p->pStruct;
+}
+
+static void fts5StructureInvalidate(Fts5Index *p){
+  if( p->pStruct ){
+    fts5StructureRelease(p->pStruct);
+    p->pStruct = 0;
  }
-  return pRet;
 }

 /*
@@ -2250,6 +2321,18 @@ static void fts5LeafSeek(
  fts5SegIterLoadNPos(p, pIter);
 }

+static sqlite3_stmt *fts5IdxSelectStmt(Fts5Index *p){
+  if( p->pIdxSelect==0 ){
+    Fts5Config *pConfig = p->pConfig;
+    fts5IndexPrepareStmt(p, &p->pIdxSelect, sqlite3_mprintf(
+          "SELECT pgno FROM '%q'.'%q_idx' WHERE "
+          "segid=? AND term<=? ORDER BY term DESC LIMIT 1",
+          pConfig->zDb, pConfig->zName
+    ));
+  }
+  return p->pIdxSelect;
+}
+
 /*
 ** Initialize the object pIter to point to term pTerm/nTerm within segment
 ** pSeg. If there is no such term in the index, the iterator is set to EOF.
@@ -2267,6 +2350,7 @@ static void fts5SegIterSeekInit(
  int iPg = 1;
  int bGe = (flags & FTS5INDEX_QUERY_SCAN);
  int bDlidx = 0;                 /* True if there is a doclist-index */
+  sqlite3_stmt *pIdxSelect = 0;

  assert( bGe==0 || (flags & FTS5INDEX_QUERY_DESC)==0 );
  assert( pTerm && nTerm );
@@ -2275,23 +2359,16 @@ static void fts5SegIterSeekInit(

  /* This block sets stack variable iPg to the leaf page number that may
  ** contain term (pTerm/nTerm), if it is present in the segment. */
-  if( p->pIdxSelect==0 ){
-    Fts5Config *pConfig = p->pConfig;
-    fts5IndexPrepareStmt(p, &p->pIdxSelect, sqlite3_mprintf(
-          "SELECT pgno FROM '%q'.'%q_idx' WHERE "
-          "segid=? AND term<=? ORDER BY term DESC LIMIT 1",
-          pConfig->zDb, pConfig->zName
-    ));
-  }
+  pIdxSelect = fts5IdxSelectStmt(p);
  if( p->rc ) return;
-  sqlite3_bind_int(p->pIdxSelect, 1, pSeg->iSegid);
-  sqlite3_bind_blob(p->pIdxSelect, 2, pTerm, nTerm, SQLITE_STATIC);
-  if( SQLITE_ROW==sqlite3_step(p->pIdxSelect) ){
-    i64 val = sqlite3_column_int(p->pIdxSelect, 0);
+  sqlite3_bind_int(pIdxSelect, 1, pSeg->iSegid);
+  sqlite3_bind_blob(pIdxSelect, 2, pTerm, nTerm, SQLITE_STATIC);
+  if( SQLITE_ROW==sqlite3_step(pIdxSelect) ){
+    i64 val = sqlite3_column_int(pIdxSelect, 0);
    iPg = (int)(val>>1);
    bDlidx = (val & 0x0001);
  }
-  p->rc = sqlite3_reset(p->pIdxSelect);
+  p->rc = sqlite3_reset(pIdxSelect);

  if( iPg<pSeg->pgnoFirst ){
    iPg = pSeg->pgnoFirst;
@@ -3453,18 +3530,46 @@ static int fts5AllocateSegid(Fts5Index *p, Fts5Structure *pStruct){
    if( pStruct->nSegment>=FTS5_MAX_SEGMENT ){
      p->rc = SQLITE_FULL;
    }else{
-      while( iSegid==0 ){
-        int iLvl, iSeg;
-        sqlite3_randomness(sizeof(u32), (void*)&iSegid);
-        iSegid = iSegid & ((1 << FTS5_DATA_ID_B)-1);
-        for(iLvl=0; iLvl<pStruct->nLevel; iLvl++){
-          for(iSeg=0; iSeg<pStruct->aLevel[iLvl].nSeg; iSeg++){
-            if( iSegid==pStruct->aLevel[iLvl].aSeg[iSeg].iSegid ){
-              iSegid = 0;
-            }
+      /* FTS5_MAX_SEGMENT is currently defined as 2000. So the following
+      ** array is 63 elements, or 252 bytes, in size.  */
+      u32 aUsed[(FTS5_MAX_SEGMENT+31) / 32];
+      int iLvl, iSeg;
+      int i;
+      u32 mask;
+      memset(aUsed, 0, sizeof(aUsed));
+      for(iLvl=0; iLvl<pStruct->nLevel; iLvl++){
+        for(iSeg=0; iSeg<pStruct->aLevel[iLvl].nSeg; iSeg++){
+          int iId = pStruct->aLevel[iLvl].aSeg[iSeg].iSegid;
+          if( iId<=FTS5_MAX_SEGMENT ){
+            aUsed[(iId-1) / 32] |= 1 << ((iId-1) % 32);
          }
        }
      }
+
+      for(i=0; aUsed[i]==0xFFFFFFFF; i++);
+      mask = aUsed[i];
+      for(iSegid=0; mask & (1 << iSegid); iSegid++);
+      iSegid += 1 + i*32;
+
+#ifdef SQLITE_DEBUG
+      for(iLvl=0; iLvl<pStruct->nLevel; iLvl++){
+        for(iSeg=0; iSeg<pStruct->aLevel[iLvl].nSeg; iSeg++){
+          assert( iSegid!=pStruct->aLevel[iLvl].aSeg[iSeg].iSegid );
+        }
+      }
+      assert( iSegid>0 && iSegid<=FTS5_MAX_SEGMENT );
+
+      {
+        sqlite3_stmt *pIdxSelect = fts5IdxSelectStmt(p);
+        if( p->rc==SQLITE_OK ){
+          u8 aBlob[2] = {0xff, 0xff};
+          sqlite3_bind_int(pIdxSelect, 1, iSegid);
+          sqlite3_bind_blob(pIdxSelect, 2, aBlob, 2, SQLITE_STATIC);
+          assert( sqlite3_step(pIdxSelect)!=SQLITE_ROW );
+          p->rc = sqlite3_reset(pIdxSelect);
+        }
+      }
+#endif
    }
  }

@@ -3710,6 +3815,9 @@ static void fts5WriteFlushLeaf(Fts5Index *p, Fts5SegWriter *pWriter){
  Fts5PageWriter *pPage = &pWriter->writer;
  i64 iRowid;

+static int nCall = 0;
+nCall++;
+
  assert( (pPage->pgidx.n==0)==(pWriter->bFirstTermInPage) );

  /* Set the szLeaf header field. */
@@ -3909,7 +4017,9 @@ static void fts5WriteFinish(
      fts5WriteFlushLeaf(p, pWriter);
    }
    *pnLeaf = pLeaf->pgno-1;
-    fts5WriteFlushBtree(p, pWriter);
+    if( pLeaf->pgno>1 ){
+      fts5WriteFlushBtree(p, pWriter);
+    }
  }
  fts5BufferFree(&pLeaf->term);
  fts5BufferFree(&pLeaf->buf);
@@ -4328,6 +4438,7 @@ static void fts5FlushOneHash(Fts5Index *p){
  ** for the new level-0 segment.  */
  pStruct = fts5StructureRead(p);
  iSegid = fts5AllocateSegid(p, pStruct);
+  fts5StructureInvalidate(p);

  if( iSegid ){
    const int pgsz = p->pConfig->pgsz;
@@ -4512,7 +4623,7 @@ static Fts5Structure *fts5IndexOptimizeStruct(

  if( pNew ){
    Fts5StructureLevel *pLvl;
-    int nByte = nSeg * sizeof(Fts5StructureSegment);
+    nByte = nSeg * sizeof(Fts5StructureSegment);
    pNew->nLevel = pStruct->nLevel+1;
    pNew->nRef = 1;
    pNew->nWriteCounter = pStruct->nWriteCounter;
@@ -4547,6 +4658,7 @@ int sqlite3Fts5IndexOptimize(Fts5Index *p){
  assert( p->rc==SQLITE_OK );
  fts5IndexFlush(p);
  pStruct = fts5StructureRead(p);
+  fts5StructureInvalidate(p);

  if( pStruct ){
    pNew = fts5IndexOptimizeStruct(p, pStruct);
@@ -4577,6 +4689,7 @@ int sqlite3Fts5IndexMerge(Fts5Index *p, int nMerge){
  Fts5Structure *pStruct = fts5StructureRead(p);
  if( pStruct ){
    int nMin = p->pConfig->nUsermerge;
+    fts5StructureInvalidate(p);
    if( nMerge<0 ){
      Fts5Structure *pNew = fts5IndexOptimizeStruct(p, pStruct);
      fts5StructureRelease(pStruct);
@@ -5004,6 +5117,7 @@ int sqlite3Fts5IndexSync(Fts5Index *p, int bCommit){
 int sqlite3Fts5IndexRollback(Fts5Index *p){
  fts5CloseReader(p);
  fts5IndexDiscardData(p);
+  fts5StructureInvalidate(p);
  /* assert( p->rc==SQLITE_OK ); */
  return SQLITE_OK;
 }
@@ -5015,6 +5129,7 @@ int sqlite3Fts5IndexRollback(Fts5Index *p){
 */
 int sqlite3Fts5IndexReinit(Fts5Index *p){
  Fts5Structure s;
+  fts5StructureInvalidate(p);
  memset(&s, 0, sizeof(Fts5Structure));
  fts5DataWrite(p, FTS5_AVERAGES_ROWID, (const u8*)"", 0);
  fts5StructureWrite(p, &s);
@@ -5073,11 +5188,13 @@ int sqlite3Fts5IndexClose(Fts5Index *p){
  int rc = SQLITE_OK;
  if( p ){
    assert( p->pReader==0 );
+    fts5StructureInvalidate(p);
    sqlite3_finalize(p->pWriter);
    sqlite3_finalize(p->pDeleter);
    sqlite3_finalize(p->pIdxWriter);
    sqlite3_finalize(p->pIdxDeleter);
    sqlite3_finalize(p->pIdxSelect);
+    sqlite3_finalize(p->pDataVersion);
    sqlite3Fts5HashFree(p->pHash);
    sqlite3_free(p->zDataTbl);
    sqlite3_free(p);
@@ -6333,3 +6450,12 @@ int sqlite3Fts5IndexInit(sqlite3 *db){
  }
  return rc;
 }
+
+
+int sqlite3Fts5IndexReset(Fts5Index *p){
+  assert( p->pStruct==0 || p->iStructVersion!=0 );
+  if( fts5IndexDataVersion(p)!=p->iStructVersion ){
+    fts5StructureInvalidate(p);
+  }
+  return fts5IndexReturn(p);
+}
@@ -597,27 +597,38 @@ static int fts5BestIndexMethod(sqlite3_vtab *pVTab, sqlite3_index_info *pInfo){
  return SQLITE_OK;
 }

+static int fts5NewTransaction(Fts5Table *pTab){
+  Fts5Cursor *pCsr;
+  for(pCsr=pTab->pGlobal->pCsr; pCsr; pCsr=pCsr->pNext){
+    if( pCsr->base.pVtab==(sqlite3_vtab*)pTab ) return SQLITE_OK;
+  }
+  return sqlite3Fts5StorageReset(pTab->pStorage);
+}
+
 /*
 ** Implementation of xOpen method.
 */
 static int fts5OpenMethod(sqlite3_vtab *pVTab, sqlite3_vtab_cursor **ppCsr){
  Fts5Table *pTab = (Fts5Table*)pVTab;
  Fts5Config *pConfig = pTab->pConfig;
-  Fts5Cursor *pCsr;               /* New cursor object */
+  Fts5Cursor *pCsr = 0;           /* New cursor object */
  int nByte;                      /* Bytes of space to allocate */
-  int rc = SQLITE_OK;             /* Return code */
+  int rc;                         /* Return code */

-  nByte = sizeof(Fts5Cursor) + pConfig->nCol * sizeof(int);
-  pCsr = (Fts5Cursor*)sqlite3_malloc(nByte);
-  if( pCsr ){
-    Fts5Global *pGlobal = pTab->pGlobal;
-    memset(pCsr, 0, nByte);
-    pCsr->aColumnSize = (int*)&pCsr[1];
-    pCsr->pNext = pGlobal->pCsr;
-    pGlobal->pCsr = pCsr;
-    pCsr->iCsrId = ++pGlobal->iNextId;
-  }else{
-    rc = SQLITE_NOMEM;
+  rc = fts5NewTransaction(pTab);
+  if( rc==SQLITE_OK ){
+    nByte = sizeof(Fts5Cursor) + pConfig->nCol * sizeof(int);
+    pCsr = (Fts5Cursor*)sqlite3_malloc(nByte);
+    if( pCsr ){
+      Fts5Global *pGlobal = pTab->pGlobal;
+      memset(pCsr, 0, nByte);
+      pCsr->aColumnSize = (int*)&pCsr[1];
+      pCsr->pNext = pGlobal->pCsr;
+      pGlobal->pCsr = pCsr;
+      pCsr->iCsrId = ++pGlobal->iNextId;
+    }else{
+      rc = SQLITE_NOMEM;
+    }
  }
  *ppCsr = (sqlite3_vtab_cursor*)pCsr;
  return rc;
@@ -1578,8 +1589,8 @@ static int fts5SyncMethod(sqlite3_vtab *pVtab){
 ** Implementation of xBegin() method. 
 */
 static int fts5BeginMethod(sqlite3_vtab *pVtab){
-  UNUSED_PARAM(pVtab);  /* Call below is a no-op for NDEBUG builds */
  fts5CheckTransactionState((Fts5Table*)pVtab, FTS5_BEGIN, 0);
+  fts5NewTransaction((Fts5Table*)pVtab);
  return SQLITE_OK;
 }

@@ -145,6 +145,7 @@ static int fts5StorageGetStmt(
  }

  *ppStmt = p->aStmt[eStmt];
+  sqlite3_reset(*ppStmt);
  return rc;
 }

@@ -368,6 +369,7 @@ static int fts5StorageInsertCallback(
  Fts5InsertCtx *pCtx = (Fts5InsertCtx*)pContext;
  Fts5Index *pIdx = pCtx->pStorage->pIndex;
  UNUSED_PARAM2(iUnused1, iUnused2);
+  if( nToken>FTS5_MAX_TOKEN_SIZE ) nToken = FTS5_MAX_TOKEN_SIZE;
  if( (tflags & FTS5_TOKEN_COLOCATED)==0 || pCtx->szCol==0 ){
    pCtx->szCol++;
  }
@@ -639,6 +641,10 @@ int sqlite3Fts5StorageMerge(Fts5Storage *p, int nMerge){
  return sqlite3Fts5IndexMerge(p->pIndex, nMerge);
 }

+int sqlite3Fts5StorageReset(Fts5Storage *p){
+  return sqlite3Fts5IndexReset(p->pIndex);
+}
+
 /*
 ** Allocate a new rowid. This is used for "external content" tables when
 ** a NULL value is inserted into the rowid column. The new rowid is allocated
@@ -810,6 +816,7 @@ static int fts5StorageIntegrityCallback(
  int iCol;

  UNUSED_PARAM2(iUnused1, iUnused2);
+  if( nToken>FTS5_MAX_TOKEN_SIZE ) nToken = FTS5_MAX_TOKEN_SIZE;

  if( (tflags & FTS5_TOKEN_COLOCATED)==0 || pCtx->szCol==0 ){
    pCtx->szCol++;
@@ -1121,5 +1128,3 @@ int sqlite3Fts5StorageConfigValue(
  }
  return rc;
 }
-
-
@@ -432,9 +432,16 @@ proc funk {} {
 }
 db func funk funk

+# This test case corrupts the structure record within the first invocation
+# of function funk(). Which used to cause the bm25() function to throw an
+# exception. But since bm25() can now used the cached structure record,
+# it never sees the corruption introduced by funk() and so the following 
+# statement no longer fails.
+#
 do_catchsql_test 16.2 {
  SELECT funk(), bm25(n1), funk() FROM n1 WHERE n1 MATCH 'a+b+c+d'
-} {1 {SQL logic error or missing database}}
+} {0 {{} -1e-06 {}}}
+# {1 {SQL logic error or missing database}}

 #-------------------------------------------------------------------------
 #
@@ -179,6 +179,10 @@ for {set i 1} {1} {incr i} {
  if {$end<=$i} break
  lset var end [expr $end - $i]
  set struct [binary format c* $var]
+
+  db close
+  sqlite3 db test.db
+
  db eval {
    BEGIN;
    UPDATE t1_data SET block = $struct WHERE id=10;
@@ -0,0 +1,67 @@
+# 2016 March 21
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#*************************************************************************
+# This file implements regression tests for SQLite library.  The
+# focus of this script is testing the FTS5 module.
+#
+# Specifically, that the fts5 module is deterministic. At one point, when
+# segment ids were allocated using sqlite3_randomness(), this was not the
+# case.
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+set testprefix fts5aa
+return_if_no_fts5 
+
+proc do_determin_test {tn} {
+  uplevel [list
+    do_execsql_test $tn {
+      SELECT (SELECT md5sum(id, block) FROM t1_data)==
+             (SELECT md5sum(id, block) FROM t2_data),
+             (SELECT md5sum(id, block) FROM t1_data)==
+             (SELECT md5sum(id, block) FROM t3_data)
+    } {1 1}
+  ]
+}
+
+foreach_detail_mode $::testprefix {
+  do_execsql_test 1.0 {
+    CREATE VIRTUAL TABLE t1 USING fts5(a, b, prefix="1 2", detail=%DETAIL%);
+    CREATE VIRTUAL TABLE t2 USING fts5(a, b, prefix="1 2", detail=%DETAIL%);
+    CREATE VIRTUAL TABLE t3 USING fts5(a, b, prefix="1 2", detail=%DETAIL%);
+  }
+
+  do_test 1.1 {
+    foreach t {t1 t2 t3} {
+      execsql [string map [list TBL $t] {
+        INSERT INTO TBL VALUES('a b c', 'd e f');
+        INSERT INTO TBL VALUES('c1 c2 c3', 'c1 c2 c3');
+        INSERT INTO TBL VALUES('xyzxyzxyz', 'xyzxyzxyz');
+      }]
+    }
+  } {}
+
+  do_determin_test 1.2
+
+  do_test 1.3 {
+    foreach t {t1 t2 t3} {
+      execsql [string map [list TBL $t] {
+        INSERT INTO TBL(TBL) VALUES('optimize');
+      }]
+    }
+  } {}
+
+  do_determin_test 1.4
+}
+
+
+finish_test
+
+
@@ -178,7 +178,7 @@ do_execsql_test 3.2 {
  ORDER BY rowid DESC;
 } {16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1}

-do_execsql_test 3.2 {
+do_execsql_test 3.3 {
  INSERT INTO abc(abc) VALUES('integrity-check');
  INSERT INTO abc(abc) VALUES('optimize');
  INSERT INTO abc(abc) VALUES('integrity-check');
@@ -187,7 +187,7 @@ do_execsql_test 3.2 {
 set v [lindex $vocab 0]
 set i 0
 foreach v $vocab {
-  do_execsql_test 3.3.[incr i] {
+  do_execsql_test 3.4.[incr i] {
    SELECT rowid FROM abc WHERE abc MATCH $v
  } {1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16}
 }
@@ -86,7 +86,7 @@ set ::res [db eval {SELECT rowid, x1 FROM x1 WHERE x1 MATCH '*reads'}]
 do_faultsim_test 4 -faults oom-* -body {
  db eval {SELECT rowid, x, x1 FROM x1 WHERE x1 MATCH '*reads'}
 } -test {
-  faultsim_test_result {0 {0 {} 4}}
+  faultsim_test_result {0 {0 {} 3}}
 }

 #-------------------------------------------------------------------------
@@ -0,0 +1,48 @@
+# 2016 March 17
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#*************************************************************************
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+source $testdir/lock_common.tcl
+
+set testprefix fts5multiclient
+return_if_no_fts5
+
+foreach_detail_mode $testprefix {
+
+do_multiclient_test tn {
+
+  do_test 1.$tn.1 {
+    sql1 { CREATE VIRTUAL TABLE t1 USING fts5(x, detail=%DETAIL%) }
+    sql1 { INSERT INTO t1 VALUES('a b c') }
+    sql2 { SELECT rowid FROM t1('b') }
+  } {1}
+
+  do_test 1.$tn.2 {
+    sql2 { INSERT INTO t1 VALUES('a b c') }
+    sql1 { SELECT rowid FROM t1('b') }
+  } {1 2}
+
+  do_test 1.$tn.3 {
+    sql2 { INSERT INTO t1 VALUES('a b c') }
+    sql1 { SELECT rowid FROM t1('b') }
+  } {1 2 3}
+
+  do_test 1.$tn.4 {
+    sql2 { INSERT INTO t1 VALUES('a b c') }
+    sql1 { INSERT INTO t1 VALUES('a b c') }
+    sql3 { INSERT INTO t1(t1) VALUES('integrity-check') }
+  } {}
+
+};# do_multiclient_test
+};# foreach_detail_mode
+finish_test
+
@@ -340,7 +340,7 @@ do_test 14.2 {

 #-------------------------------------------------------------------------
 db func rnddoc fts5_rnddoc
-do_execsql_test 4.0 {
+do_execsql_test 14.3 {
  CREATE VIRTUAL TABLE x1 USING fts5(x);
  INSERT INTO x1(x1, rank) VALUES('pgsz', 32);

@@ -348,9 +348,9 @@ do_execsql_test 4.0 {
  INSERT INTO x1 SELECT rnddoc(5) FROM ii;
 }

-do_execsql_test 4.1 {
+do_execsql_test 14.4 {
  SELECT rowid, x, x1 FROM x1 WHERE x1 MATCH '*reads'
-} {0 {} 4}
+} {0 {} 3}

 #-------------------------------------------------------------------------
 reset_db
@@ -448,4 +448,24 @@ do_execsql_test 20.2 {
  INSERT INTO x1(x1) VALUES('integrity-check');
 } {}

+#-------------------------------------------------------------------------
+reset_db
+set doc "a b [string repeat x 100000]"
+do_execsql_test 21.0 {
+  CREATE VIRTUAL TABLE x1 USING fts5(x);
+  INSERT INTO x1(rowid, x) VALUES(11111, $doc);
+  INSERT INTO x1(rowid, x) VALUES(11112, $doc);
+}
+do_execsql_test 21.1 {
+  INSERT INTO x1(x1) VALUES('integrity-check');
+}
+do_execsql_test 21.2 {
+  SELECT rowid FROM x1($doc);
+} {11111 11112}
+do_execsql_test 21.3 {
+  DELETE FROM x1 WHERE rowid=11111;
+  INSERT INTO x1(x1) VALUES('integrity-check');
+  SELECT rowid FROM x1($doc);
+} {11112}
+
 finish_test
@@ -17,6 +17,7 @@ proc process_cmdline {} {
    {detail    "full"     "Fts5 detail mode to use"}
    {repeat    1          "Load each file this many times"}
    {prefix    ""         "Fts prefix= option"}
+    {trans     1          "True to use a transaction"}
    database
    file...
  } {
@@ -214,7 +215,7 @@ foreach c [lrange $cols 1 end] {
 }
 append sql ")"

-db eval BEGIN
+if {$A(trans)} { db eval BEGIN }
  while {$i < $N} {
    foreach c $cols s $A(colsize) {
      set R($c) [lrange $tokens $i [expr $i+$s-1]]
@@ -222,7 +223,7 @@ db eval BEGIN
    }
    db eval $sql
  }
-db eval COMMIT
+if {$A(trans)} { db eval COMMIT }



@@ -611,7 +611,7 @@ foreach {tn3 create_vfs destroy_vfs} {
       9 {
         CREATE TABLE t1(a, b PRIMARY KEY) WITHOUT ROWID;
         CREATE TABLE rbu.data_t1(a, b, rbu_control);
-         INSERT INTO rbu.data_t1 VALUES(1, 2, 2);
+         INSERT INTO rbu.data_t1 VALUES(1, 2, 3);
       } {SQLITE_ERROR - invalid rbu_control value}

       10 {
@@ -18,16 +18,7 @@ if {![info exists testdir]} {
 source $testdir/tester.tcl
 set testprefix rbudiff

-if {$tcl_platform(platform)=="windows"} {
-  set PROG "sqldiff.exe"
-} else {
-  set PROG "./sqldiff"
-}
-if {![file exe $PROG]} {
-  puts "rbudiff.test cannot run because $PROG is not available"
-  finish_test
-  return
-}
+set PROG [test_find_sqldiff]
 db close

 proc get_rbudiff_sql {db1 db2} {
@@ -45,6 +36,7 @@ proc step_rbu {target rbu} {
 }

 proc apply_rbudiff {sql target} {
+  test_rbucount $sql
  forcedelete rbu.db
  sqlite3 rbudb rbu.db
  rbudb eval $sql
@@ -52,6 +44,27 @@ proc apply_rbudiff {sql target} {
  step_rbu $target rbu.db
 }

+# The only argument is the output of an [sqldiff -rbu] run. This command
+# tests that the contents of the rbu_count table is correct. An exception
+# is thrown if it is not.
+#
+proc test_rbucount {sql} {
+  sqlite3 tmpdb ""
+  tmpdb eval $sql
+  tmpdb eval {
+    SELECT name FROM sqlite_master WHERE name LIKE 'data%' AND type='table'
+  } {
+    set a [tmpdb eval "SELECT count(*) FROM $name"]
+    set b [tmpdb eval {SELECT cnt FROM rbu_count WHERE tbl = $name}]
+    if {$a != $b} { 
+      tmpdb close
+      error "rbu_count error - tbl = $name" 
+    }
+  }
+  tmpdb close
+  return ""
+}
+
 proc rbudiff_cksum {db1} {
  set txt ""

@@ -0,0 +1,415 @@
+# 2016 March 18
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#***********************************************************************
+#
+
+source [file join [file dirname [info script]] rbu_common.tcl]
+set ::testprefix rbuprogress
+
+
+proc create_db_file {filename sql} {
+  forcedelete $filename
+  sqlite3 tmpdb $filename  
+  tmpdb eval $sql
+  tmpdb close
+}
+
+# Create a simple RBU database. That expects to write to a table:
+#
+#   CREATE TABLE t1(a INTEGER PRIMARY KEY, b, c);
+#
+proc create_rbu1 {filename} {
+  create_db_file $filename {
+    CREATE TABLE data_t1(a, b, c, rbu_control);
+    INSERT INTO data_t1 VALUES(1, 2, 3, 0);
+    INSERT INTO data_t1 VALUES(2, 'two', 'three', 0);
+    INSERT INTO data_t1 VALUES(3, NULL, 8.2, 0);
+
+    CREATE TABLE rbu_count(tbl, cnt);
+    INSERT INTO rbu_count VALUES('data_t1', 3);
+  }
+  return $filename
+}
+
+
+do_execsql_test 1.0 {
+  CREATE TABLE t1(a INTEGER PRIMARY KEY, b, c);
+}
+
+do_test 1.1 {
+  create_rbu1 rbu.db
+  sqlite3rbu rbu test.db rbu.db
+  rbu bp_progress
+} {0 0}
+do_test 1.2 { rbu step ; rbu bp_progress } {3333 0}
+do_test 1.3 { rbu step ; rbu bp_progress } {6666 0}
+do_test 1.4 { rbu step ; rbu bp_progress } {10000 0}
+do_test 1.5 { rbu step ; rbu bp_progress } {10000 0}
+do_test 1.6 { rbu step ; rbu bp_progress } {10000 0}
+do_test 1.7 { rbu step ; rbu bp_progress } {10000 5000}
+do_test 1.8 { rbu step ; rbu bp_progress } {10000 10000}
+do_test 1.9 { rbu step ; rbu bp_progress } {10000 10000}
+
+do_test 1.10 {
+  rbu close
+} {SQLITE_DONE}
+
+#-------------------------------------------------------------------------
+#
+proc do_sp_test {tn bReopen target rbu reslist} {
+  uplevel [list do_test $tn [subst -nocommands {
+    if {$bReopen==0} { sqlite3rbu rbu $target $rbu }
+    set res [list]
+    while 1 {
+      if {$bReopen} { sqlite3rbu rbu $target $rbu }
+      set rc [rbu step]
+      if {[set rc] != "SQLITE_OK"} { rbu close ; error "error 1" }
+      lappend res [lindex [rbu bp_progress] 0]
+      if {[lindex [set res] end]==10000} break
+      if {$bReopen} { rbu close }
+    }
+    if {[set res] != [list $reslist]} {
+      rbu close
+      error "1. reslist incorrect (expect=$reslist got=[set res])"
+    }
+
+    # One step to clean up the temporary tables used to update the only
+    # target table in the rbu database. And one more to move the *-oal 
+    # file to *-wal. After each of these steps, the progress remains
+    # at "10000 0".
+    #
+    if {[lindex [list $reslist] 0]!=-1} {
+      rbu step
+      set res [rbu bp_progress]
+      if {[set res] != [list 10000 0]} {
+        rbu close
+        error "2. reslist incorrect (expect=10000 0 got=[set res])"
+      }
+    }
+
+    rbu step
+    set res [rbu bp_progress]
+    if {[set res] != [list 10000 0]} {
+      rbu close
+      error "3. reslist incorrect (expect=10000 0 got=[set res])"
+    }
+
+    # Do the checkpoint.
+    while {[rbu step]=="SQLITE_OK"} { 
+      foreach {a b} [rbu bp_progress] {}
+      if {[set a]!=10000 || [set b]<=0 || [set b]>10000} {
+        rbu close
+        error "4. reslist incorrect (expect=10000 1..10000 got=[set a] [set b])"
+      }
+    }
+
+    set res [rbu bp_progress]
+    if {[set res] != [list 10000 10000]} {
+      rbu close
+      error "5. reslist is incorrect (expect=10000 10000 got=[set res])"
+    }
+
+    rbu close
+  }] {SQLITE_DONE}]
+}
+
+foreach {bReopen} { 0 1 } {
+  reset_db
+  do_test 2.$bReopen.1.0 {
+    execsql {
+      CREATE TABLE t1(a INTEGER PRIMARY KEY, b, c);
+    }
+    create_db_file rbu.db {
+      CREATE TABLE data_t1(a, b, c, rbu_control);
+      INSERT INTO data_t1 VALUES(4, 4, 4, 0);
+      INSERT INTO data_t1 VALUES(5, 5, 5, 0);
+  
+      CREATE TABLE rbu_count(tbl, cnt);
+      INSERT INTO rbu_count VALUES('data_t1', 2);
+    }
+  } {}
+  do_sp_test 2.$bReopen.1.1 $bReopen test.db rbu.db {5000 10000}
+  
+  reset_db
+  do_test 2.$bReopen.2.0 {
+    execsql { CREATE TABLE t1(a INTEGER PRIMARY KEY, b, c) }
+    create_rbu1 rbu.db
+  } {rbu.db}
+  do_sp_test 2.$bReopen.2.1 $bReopen test.db rbu.db {3333 6666 10000}
+  
+  reset_db
+  do_test 2.$bReopen.3.0 {
+    execsql { 
+      CREATE TABLE t1(a INTEGER PRIMARY KEY, b, c);
+      CREATE INDEX i1 ON t1(b);
+      INSERT INTO t1 VALUES(1, 1, 1);
+      INSERT INTO t1 VALUES(2, 2, 2);
+      INSERT INTO t1 VALUES(3, 3, 3);
+    }
+    create_db_file rbu.db {
+      CREATE TABLE data_t1(a, b, c, rbu_control);
+      INSERT INTO data_t1 VALUES(4, 4, 4, 0);
+      INSERT INTO data_t1 VALUES(2, NULL, NULL, 1);
+      INSERT INTO data_t1 VALUES(5, NULL, NULL, 1);
+  
+      CREATE TABLE rbu_count(tbl, cnt);
+      INSERT INTO rbu_count VALUES('data_t1', 3);
+    }
+  } {}
+  do_sp_test 2.$bReopen.3.1 $bReopen test.db rbu.db {1666 3333 6000 8000 10000}
+  
+  reset_db
+  do_test 2.$bReopen.4.0 {
+    execsql { 
+      CREATE TABLE t1(a INTEGER PRIMARY KEY, b, c);
+      CREATE INDEX i1 ON t1(b);
+      INSERT INTO t1 VALUES(1, 1, 1);
+      INSERT INTO t1 VALUES(2, 2, 2);
+      INSERT INTO t1 VALUES(3, 3, 3);
+    }
+    create_db_file rbu.db {
+      CREATE TABLE data_t1(a, b, c, rbu_control);
+      INSERT INTO data_t1 VALUES(2, 4, 4, '.xx');
+  
+      CREATE TABLE rbu_count(tbl, cnt);
+      INSERT INTO rbu_count VALUES('data_t1', 1);
+    }
+  } {}
+  do_sp_test 2.$bReopen.4.1 $bReopen test.db rbu.db {3333 6666 10000}
+  
+  reset_db
+  do_test 2.$bReopen.5.0 {
+    execsql { 
+      CREATE TABLE t1(a INTEGER PRIMARY KEY, b, c);
+      CREATE INDEX i1 ON t1(b);
+      INSERT INTO t1 VALUES(1, 1, 1);
+      INSERT INTO t1 VALUES(2, 2, 2);
+      INSERT INTO t1 VALUES(3, 3, 3);
+    }
+    create_db_file rbu.db {
+      CREATE TABLE data_t1(a, b, c, rbu_control);
+      INSERT INTO data_t1 VALUES(4, NULL, 4, '.xx');
+  
+      CREATE TABLE rbu_count(tbl, cnt);
+      INSERT INTO rbu_count VALUES('data_t1', 1);
+    }
+  } {}
+  do_sp_test 2.$bReopen.5.1 $bReopen test.db rbu.db {10000}
+
+  reset_db
+  do_test 2.$bReopen.6.0 {
+    execsql { 
+      CREATE TABLE t1(a INTEGER PRIMARY KEY, b, c);
+      CREATE INDEX i1 ON t1(b);
+      INSERT INTO t1 VALUES(1, 1, 1);
+      INSERT INTO t1 VALUES(2, 2, 2);
+      INSERT INTO t1 VALUES(3, 3, 3);
+    }
+    create_db_file rbu.db {
+      CREATE TABLE data_t1(a, b, c, rbu_control);
+      INSERT INTO data_t1 VALUES(4, 4, 4, 0);
+      INSERT INTO data_t1 VALUES(2, NULL, NULL, 1);
+      INSERT INTO data_t1 VALUES(5, NULL, NULL, 1);
+    }
+  } {}
+  do_sp_test 2.$bReopen.6.1 $bReopen test.db rbu.db {-1 -1 -1 -1 -1 10000}
+}
+
+#-------------------------------------------------------------------------
+# The following tests verify that the API works when resuming an update
+# during the incremental checkpoint stage.
+#
+proc do_phase2_test {tn bReopen target rbu nStep} {
+  uplevel [list do_test $tn [subst -nocommands {
+
+    # Build the OAL/WAL file:
+    sqlite3rbu rbu $target $rbu
+    while {[lindex [rbu bp_progress] 0]<10000} { 
+      set rc [rbu step]
+      if {"SQLITE_OK" != [set rc]} { rbu close }
+    }
+
+    # Clean up the temp tables and move the *-oal file to *-wal.
+    rbu step
+    rbu step
+
+    for {set i 0} {[set i] < $nStep} {incr i} {
+      if {$bReopen} {
+        rbu close
+        sqlite3rbu rbu $target $rbu
+      }
+      rbu step
+      set res [rbu bp_progress]
+      set expect [expr (1 + [set i]) * 10000 / $nStep]
+      if {[lindex [set res] 1] != [set expect]} {
+        error "Have [set res], expected 10000 [set expect]"
+      }
+    }
+
+    set rc [rbu step]
+    if {[set rc] != "SQLITE_DONE"} {
+      error "Have [set rc], expected SQLITE_DONE" 
+    }
+
+    rbu close
+  }] {SQLITE_DONE}]
+}
+
+foreach bReopen {0 1} {
+  do_test 3.$bReopen.1.0 {
+    reset_db
+    execsql {
+      CREATE TABLE t1(a INTEGER PRIMARY KEY, b);
+      CREATE TABLE t2(a INTEGER PRIMARY KEY, b);
+      CREATE TABLE t3(a INTEGER PRIMARY KEY, b);
+      CREATE TABLE t4(a INTEGER PRIMARY KEY, b);
+    }
+    create_db_file rbu.db {
+      CREATE TABLE data_t1(a, b, rbu_control);
+      CREATE TABLE data_t2(a, b, rbu_control);
+      CREATE TABLE data_t3(a, b, rbu_control);
+      CREATE TABLE data_t4(a, b, rbu_control);
+      INSERT INTO data_t1 VALUES(1, 2, 0);
+      INSERT INTO data_t2 VALUES(1, 2, 0);
+      INSERT INTO data_t3 VALUES(1, 2, 0);
+      INSERT INTO data_t4 VALUES(1, 2, 0);
+  
+      CREATE TABLE rbu_count(tbl, cnt);
+      INSERT INTO rbu_count VALUES('data_t1', 1);
+      INSERT INTO rbu_count VALUES('data_t2', 1);
+      INSERT INTO rbu_count VALUES('data_t3', 1);
+      INSERT INTO rbu_count VALUES('data_t4', 1);
+    }
+  } {}
+  do_phase2_test 3.$bReopen.1.1 $bReopen test.db rbu.db 5
+}
+
+
+foreach {bReopen} { 0 1 } {
+  foreach {tn tbl} {
+    ipk { CREATE TABLE t1(a INTEGER PRIMARY KEY, b, c) }
+    wr  { CREATE TABLE t1(a INT PRIMARY KEY, b, c) WITHOUT ROWID }
+    pk  { CREATE TABLE t1(a INT PRIMARY KEY, b, c) }
+  } {
+
+    foreach {tn2 rbusql r1 r3} {
+      1 {
+        CREATE TABLE data0_t1(a, b, c, rbu_control);
+        INSERT INTO data0_t1 VALUES(15, 15, 15, 0);
+        INSERT INTO data0_t1 VALUES(20, 20, 20, 0);
+        CREATE TABLE rbu_count(tbl, cnt);
+        INSERT INTO rbu_count VALUES('data0_t1', 2); 
+      } 
+      {2500 5000 7500 10000}
+      {1666 3333 5000 6666 8333 10000}
+
+      2 {
+        CREATE TABLE data0_t1(a, b, c, rbu_control);
+        INSERT INTO data0_t1 VALUES(10, 10, 10, 2);
+        CREATE TABLE rbu_count(tbl, cnt);
+        INSERT INTO rbu_count VALUES('data0_t1', 1); 
+      } 
+      {3333 6666 10000}
+      {2000 4000 6000 8000 10000}
+
+      3 {
+        CREATE TABLE data0_t1(a, b, c, rbu_control);
+        INSERT INTO data0_t1 VALUES(7, 7, 7, 2);
+        INSERT INTO data0_t1 VALUES(10, 10, 10, 2);
+        CREATE TABLE rbu_count(tbl, cnt);
+        INSERT INTO rbu_count VALUES('data0_t1', 2); 
+      } 
+      {2500 4000 6000 8000 10000}
+      {1666 2500 3750 5000 6250 7500 8750 10000}
+
+    } {
+
+      reset_db ; execsql $tbl
+      do_test 4.$tn.$bReopen.$tn2.0 {
+        execsql {
+          CREATE INDEX t1c ON t1(c);
+          INSERT INTO t1 VALUES(1, 1, 1);
+          INSERT INTO t1 VALUES(5, 5, 5);
+          INSERT INTO t1 VALUES(10, 10, 10);
+        }
+        create_db_file rbu.db $rbusql
+      } {}
+
+      set R(ipk) $r1
+      set R(wr) $r1
+      set R(pk) $r3
+      do_sp_test 4.$tn.$bReopen.$tn2.1 $bReopen test.db rbu.db $R($tn)
+    }
+  }
+}
+
+foreach {bReopen} { 0 1 } {
+  foreach {tn tbl} {
+    nopk { 
+      CREATE TABLE t1(a, b, c);
+      CREATE INDEX t1c ON t1(c);
+    }
+    vtab { 
+      CREATE VIRTUAL TABLE t1 USING fts5(a, b, c);
+    }
+  } {
+
+    foreach {tn2 rbusql r1 r2} {
+      1 {
+        CREATE TABLE data0_t1(a, b, c, rbu_rowid, rbu_control);
+        INSERT INTO data0_t1 VALUES(15, 15, 15, 4, 0);
+        INSERT INTO data0_t1 VALUES(20, 20, 20, 5, 0);
+        CREATE TABLE rbu_count(tbl, cnt);
+        INSERT INTO rbu_count VALUES('data0_t1', 2); 
+      } 
+      {2500 5000 7500 10000}
+      {5000 10000}
+
+      2 {
+        CREATE TABLE data0_t1(rbu_rowid, a, b, c, rbu_control);
+        INSERT INTO data0_t1 VALUES(0, 7, 7, 7, 2);
+        INSERT INTO data0_t1 VALUES(2, 10, 10, 10, 2);
+        CREATE TABLE rbu_count(tbl, cnt);
+        INSERT INTO rbu_count VALUES('data0_t1', 2); 
+      } 
+      {2500 4000 6000 8000 10000}
+      {5000 10000}
+
+      3 {
+        CREATE TABLE data0_t1(rbu_rowid, a, b, c, rbu_control);
+        INSERT INTO data0_t1 VALUES(1, NULL, NULL, NULL, 1);
+        INSERT INTO data0_t1 VALUES(2, NULL, NULL, 7, '..x');
+        CREATE TABLE rbu_count(tbl, cnt);
+        INSERT INTO rbu_count VALUES('data0_t1', 2); 
+      } 
+      {2500 4000 6000 8000 10000}
+      {5000 10000}
+    } {
+
+      reset_db ; execsql $tbl
+      do_test 5.$tn.$bReopen.$tn2.0 {
+        execsql {
+          INSERT INTO t1 VALUES(1, 1, 1);
+          INSERT INTO t1 VALUES(5, 5, 5);
+          INSERT INTO t1 VALUES(10, 10, 10);
+        }
+        create_db_file rbu.db $rbusql
+      } {}
+
+      set R(nopk) $r1
+      set R(vtab) $r2
+      do_sp_test 5.$tn.$bReopen.$tn2.1 $bReopen test.db rbu.db $R($tn)
+    }
+  }
+}
+
+
+finish_test
+
@@ -147,14 +147,15 @@
 ** RBU_STATE_OALSZ:
 **   Valid if STAGE==1. The size in bytes of the *-oal file.
 */
-#define RBU_STATE_STAGE       1
-#define RBU_STATE_TBL         2
-#define RBU_STATE_IDX         3
-#define RBU_STATE_ROW         4
-#define RBU_STATE_PROGRESS    5
-#define RBU_STATE_CKPT        6
-#define RBU_STATE_COOKIE      7
-#define RBU_STATE_OALSZ       8
+#define RBU_STATE_STAGE        1
+#define RBU_STATE_TBL          2
+#define RBU_STATE_IDX          3
+#define RBU_STATE_ROW          4
+#define RBU_STATE_PROGRESS     5
+#define RBU_STATE_CKPT         6
+#define RBU_STATE_COOKIE       7
+#define RBU_STATE_OALSZ        8
+#define RBU_STATE_PHASEONESTEP 9

 #define RBU_STAGE_OAL         1
 #define RBU_STAGE_MOVE        2
@@ -200,6 +201,7 @@ struct RbuState {
  i64 nProgress;
  u32 iCookie;
  i64 iOalSz;
+  i64 nPhaseOneStep;
 };

 struct RbuUpdateStmt {
@@ -244,6 +246,7 @@ struct RbuObjIter {
  int iTnum;                      /* Root page of current object */
  int iPkTnum;                    /* If eType==EXTERNAL, root of PK index */
  int bUnique;                    /* Current index is unique */
+  int nIndex;                     /* Number of aux. indexes on table zTbl */

  /* Statements created by rbuObjIterPrepareAll() */
  int nCol;                       /* Number of columns in current object */
@@ -297,6 +300,43 @@ struct RbuFrame {

 /*
 ** RBU handle.
+**
+** nPhaseOneStep:
+**   If the RBU database contains an rbu_count table, this value is set to
+**   a running estimate of the number of b-tree operations required to 
+**   finish populating the *-oal file. This allows the sqlite3_bp_progress()
+**   API to calculate the permyriadage progress of populating the *-oal file
+**   using the formula:
+**
+**     permyriadage = (10000 * nProgress) / nPhaseOneStep
+**
+**   nPhaseOneStep is initialized to the sum of:
+**
+**     nRow * (nIndex + 1)
+**
+**   for all source tables in the RBU database, where nRow is the number
+**   of rows in the source table and nIndex the number of indexes on the
+**   corresponding target database table.
+**
+**   This estimate is accurate if the RBU update consists entirely of
+**   INSERT operations. However, it is inaccurate if:
+**
+**     * the RBU update contains any UPDATE operations. If the PK specified
+**       for an UPDATE operation does not exist in the target table, then
+**       no b-tree operations are required on index b-trees. Or if the 
+**       specified PK does exist, then (nIndex*2) such operations are
+**       required (one delete and one insert on each index b-tree).
+**
+**     * the RBU update contains any DELETE operations for which the specified
+**       PK does not exist. In this case no operations are required on index
+**       b-trees.
+**
+**     * the RBU update contains REPLACE operations. These are similar to
+**       UPDATE operations.
+**
+**   nPhaseOneStep is updated to account for the conditions above during the
+**   first pass of each source table. The updated nPhaseOneStep value is
+**   stored in the rbu_state table if the RBU update is suspended.
 */
 struct sqlite3rbu {
  int eStage;                     /* Value of RBU_STATE_STAGE field */
@@ -314,6 +354,7 @@ struct sqlite3rbu {
  const char *zVfsName;           /* Name of automatically created rbu vfs */
  rbu_file *pTargetFd;            /* File handle open on target db */
  i64 iOalSz;
+  i64 nPhaseOneStep;

  /* The following state variables are used as part of the incremental
  ** checkpoint stage (eStage==RBU_STAGE_CKPT). See comments surrounding
@@ -1144,6 +1185,7 @@ static void rbuObjIterCacheIndexedCols(sqlite3rbu *p, RbuObjIter *pIter){
    );
  }

+  pIter->nIndex = 0;
  while( p->rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pList) ){
    const char *zIdx = (const char*)sqlite3_column_text(pList, 1);
    sqlite3_stmt *pXInfo = 0;
@@ -1157,6 +1199,12 @@ static void rbuObjIterCacheIndexedCols(sqlite3rbu *p, RbuObjIter *pIter){
    }
    rbuFinalize(p, pXInfo);
    bIndex = 1;
+    pIter->nIndex++;
+  }
+
+  if( pIter->eType==RBU_PK_WITHOUT_ROWID ){
+    /* "PRAGMA index_list" includes the main PK b-tree */
+    pIter->nIndex--;
  }

  rbuFinalize(p, pList);
@@ -1270,6 +1318,7 @@ static int rbuObjIterCacheTableInfo(sqlite3rbu *p, RbuObjIter *pIter){
    rbuFinalize(p, pStmt);
    rbuObjIterCacheIndexedCols(p, pIter);
    assert( pIter->eType!=RBU_PK_VTAB || pIter->abIndexed==0 );
+    assert( pIter->eType!=RBU_PK_VTAB || pIter->nIndex==0 );
  }

  return p->rc;
@@ -1823,6 +1872,14 @@ static void rbuTmpInsertFunc(
  int rc = SQLITE_OK;
  int i;

+  assert( sqlite3_value_int(apVal[0])!=0
+      || p->objiter.eType==RBU_PK_EXTERNAL 
+      || p->objiter.eType==RBU_PK_NONE 
+  );
+  if( sqlite3_value_int(apVal[0])!=0 ){
+    p->nPhaseOneStep += p->objiter.nIndex;
+  }
+
  for(i=0; rc==SQLITE_OK && i<nVal; i++){
    rc = sqlite3_bind_value(p->objiter.pTmpInsert, i+1, apVal[i]);
  }
@@ -2567,6 +2624,17 @@ static void rbuStepOneOp(sqlite3rbu *p, int eType){

  assert( p->rc==SQLITE_OK );
  assert( eType!=RBU_DELETE || pIter->zIdx==0 );
+  assert( eType==RBU_DELETE || eType==RBU_IDX_DELETE
+       || eType==RBU_INSERT || eType==RBU_IDX_INSERT
+  );
+
+  /* If this is a delete, decrement nPhaseOneStep by nIndex. If the DELETE
+  ** statement below does actually delete a row, nPhaseOneStep will be
+  ** incremented by the same amount when SQL function rbu_tmp_insert()
+  ** is invoked by the trigger.  */
+  if( eType==RBU_DELETE ){
+    p->nPhaseOneStep -= p->objiter.nIndex;
+  }

  if( eType==RBU_IDX_DELETE || eType==RBU_DELETE ){
    pWriter = pIter->pDelete;
@@ -2638,11 +2706,14 @@ static int rbuStep(sqlite3rbu *p){
    );
    assert( eType!=RBU_UPDATE || pIter->zIdx==0 );

-    if( pIter->zIdx==0 && eType==RBU_IDX_DELETE ){
+    if( pIter->zIdx==0 && (eType==RBU_IDX_DELETE || eType==RBU_IDX_INSERT) ){
      rbuBadControlError(p);
    }
    else if( eType==RBU_REPLACE ){
-      if( pIter->zIdx==0 ) rbuStepOneOp(p, RBU_DELETE);
+      if( pIter->zIdx==0 ){
+        p->nPhaseOneStep += p->objiter.nIndex;
+        rbuStepOneOp(p, RBU_DELETE);
+      }
      if( p->rc==SQLITE_OK ) rbuStepOneOp(p, RBU_INSERT);
    }
    else if( eType!=RBU_UPDATE ){
@@ -2652,6 +2723,7 @@ static int rbuStep(sqlite3rbu *p){
      sqlite3_value *pVal;
      sqlite3_stmt *pUpdate = 0;
      assert( eType==RBU_UPDATE );
+      p->nPhaseOneStep -= p->objiter.nIndex;
      rbuGetUpdateStmt(p, pIter, zMask, &pUpdate);
      if( pUpdate ){
        int i;
@@ -2729,6 +2801,7 @@ static void rbuSaveState(sqlite3rbu *p, int eStage){
          "(%d, %d), "
          "(%d, %lld), "
          "(%d, %lld), "
+          "(%d, %lld), "
          "(%d, %lld) ",
          p->zStateDb,
          RBU_STATE_STAGE, eStage,
@@ -2738,7 +2811,8 @@ static void rbuSaveState(sqlite3rbu *p, int eStage){
          RBU_STATE_PROGRESS, p->nProgress,
          RBU_STATE_CKPT, p->iWalCksum,
          RBU_STATE_COOKIE, (i64)p->pTargetFd->iCookie,
-          RBU_STATE_OALSZ, p->iOalSz
+          RBU_STATE_OALSZ, p->iOalSz,
+          RBU_STATE_PHASEONESTEP, p->nPhaseOneStep
      )
    );
    assert( pInsert==0 || rc==SQLITE_OK );
@@ -2925,6 +2999,10 @@ static RbuState *rbuLoadState(sqlite3rbu *p){
        pRet->iOalSz = (u32)sqlite3_column_int64(pStmt, 1);
        break;

+      case RBU_STATE_PHASEONESTEP:
+        pRet->nPhaseOneStep = sqlite3_column_int64(pStmt, 1);
+        break;
+
      default:
        rc = SQLITE_CORRUPT;
        break;
@@ -3032,6 +3110,100 @@ static void rbuDeleteVfs(sqlite3rbu *p){
  }
 }

+/*
+** This user-defined SQL function is invoked with a single argument - the
+** name of a table expected to appear in the target database. It returns
+** the number of auxilliary indexes on the table.
+*/
+static void rbuIndexCntFunc(
+  sqlite3_context *pCtx, 
+  int nVal,
+  sqlite3_value **apVal
+){
+  sqlite3rbu *p = (sqlite3rbu*)sqlite3_user_data(pCtx);
+  sqlite3_stmt *pStmt = 0;
+  char *zErrmsg = 0;
+  int rc;
+
+  assert( nVal==1 );
+  
+  rc = prepareFreeAndCollectError(p->dbMain, &pStmt, &zErrmsg, 
+      sqlite3_mprintf("SELECT count(*) FROM sqlite_master "
+        "WHERE type='index' AND tbl_name = %Q", sqlite3_value_text(apVal[0]))
+  );
+  if( rc!=SQLITE_OK ){
+    sqlite3_result_error(pCtx, zErrmsg, -1);
+  }else{
+    int nIndex = 0;
+    if( SQLITE_ROW==sqlite3_step(pStmt) ){
+      nIndex = sqlite3_column_int(pStmt, 0);
+    }
+    rc = sqlite3_finalize(pStmt);
+    if( rc==SQLITE_OK ){
+      sqlite3_result_int(pCtx, nIndex);
+    }else{
+      sqlite3_result_error(pCtx, sqlite3_errmsg(p->dbMain), -1);
+    }
+  }
+
+  sqlite3_free(zErrmsg);
+}
+
+/*
+** If the RBU database contains the rbu_count table, use it to initialize
+** the sqlite3rbu.nPhaseOneStep variable. The schema of the rbu_count table
+** is assumed to contain the same columns as:
+**
+**   CREATE TABLE rbu_count(tbl TEXT PRIMARY KEY, cnt INTEGER) WITHOUT ROWID;
+**
+** There should be one row in the table for each data_xxx table in the
+** database. The 'tbl' column should contain the name of a data_xxx table,
+** and the cnt column the number of rows it contains.
+**
+** sqlite3rbu.nPhaseOneStep is initialized to the sum of (1 + nIndex) * cnt
+** for all rows in the rbu_count table, where nIndex is the number of 
+** indexes on the corresponding target database table.
+*/
+static void rbuInitPhaseOneSteps(sqlite3rbu *p){
+  if( p->rc==SQLITE_OK ){
+    sqlite3_stmt *pStmt = 0;
+    int bExists = 0;                /* True if rbu_count exists */
+
+    p->nPhaseOneStep = -1;
+
+    p->rc = sqlite3_create_function(p->dbRbu, 
+        "rbu_index_cnt", 1, SQLITE_UTF8, (void*)p, rbuIndexCntFunc, 0, 0
+    );
+  
+    /* Check for the rbu_count table. If it does not exist, or if an error
+    ** occurs, nPhaseOneStep will be left set to -1. */
+    if( p->rc==SQLITE_OK ){
+      p->rc = prepareAndCollectError(p->dbRbu, &pStmt, &p->zErrmsg,
+          "SELECT 1 FROM sqlite_master WHERE tbl_name = 'rbu_count'"
+      );
+    }
+    if( p->rc==SQLITE_OK ){
+      if( SQLITE_ROW==sqlite3_step(pStmt) ){
+        bExists = 1;
+      }
+      p->rc = sqlite3_finalize(pStmt);
+    }
+  
+    if( p->rc==SQLITE_OK && bExists ){
+      p->rc = prepareAndCollectError(p->dbRbu, &pStmt, &p->zErrmsg,
+          "SELECT sum(cnt * (1 + rbu_index_cnt(rbu_target_name(tbl))))"
+          "FROM rbu_count"
+      );
+      if( p->rc==SQLITE_OK ){
+        if( SQLITE_ROW==sqlite3_step(pStmt) ){
+          p->nPhaseOneStep = sqlite3_column_int64(pStmt, 0);
+        }
+        p->rc = sqlite3_finalize(pStmt);
+      }
+    }
+  }
+}
+
 /*
 ** Open and return a new RBU handle. 
 */
@@ -3077,9 +3249,11 @@ sqlite3rbu *sqlite3rbu_open(

        if( pState->eStage==0 ){ 
          rbuDeleteOalFile(p);
+          rbuInitPhaseOneSteps(p);
          p->eStage = RBU_STAGE_OAL;
        }else{
          p->eStage = pState->eStage;
+          p->nPhaseOneStep = pState->nPhaseOneStep;
        }
        p->nProgress = pState->nProgress;
        p->iOalSz = pState->iOalSz;
@@ -3243,6 +3417,42 @@ sqlite3_int64 sqlite3rbu_progress(sqlite3rbu *pRbu){
  return pRbu->nProgress;
 }

+/*
+** Return permyriadage progress indications for the two main stages of
+** an RBU update.
+*/
+void sqlite3rbu_bp_progress(sqlite3rbu *p, int *pnOne, int *pnTwo){
+  const int MAX_PROGRESS = 10000;
+  switch( p->eStage ){
+    case RBU_STAGE_OAL:
+      if( p->nPhaseOneStep>0 ){
+        *pnOne = (int)(MAX_PROGRESS * (i64)p->nProgress/(i64)p->nPhaseOneStep);
+      }else{
+        *pnOne = -1;
+      }
+      *pnTwo = 0;
+      break;
+
+    case RBU_STAGE_MOVE:
+      *pnOne = MAX_PROGRESS;
+      *pnTwo = 0;
+      break;
+
+    case RBU_STAGE_CKPT:
+      *pnOne = MAX_PROGRESS;
+      *pnTwo = (int)(MAX_PROGRESS * (i64)p->nStep / (i64)p->nFrame);
+      break;
+
+    case RBU_STAGE_DONE:
+      *pnOne = MAX_PROGRESS;
+      *pnTwo = MAX_PROGRESS;
+      break;
+
+    default:
+      assert( 0 );
+  }
+}
+
 int sqlite3rbu_savestate(sqlite3rbu *p){
  int rc = p->rc;
  
@@ -400,6 +400,48 @@ int sqlite3rbu_close(sqlite3rbu *pRbu, char **pzErrmsg);
 */
 sqlite3_int64 sqlite3rbu_progress(sqlite3rbu *pRbu);

+/*
+** Obtain permyriadage (permyriadage is to 10000 as percentage is to 100) 
+** progress indications for the two stages of an RBU update. This API may
+** be useful for driving GUI progress indicators and similar.
+**
+** An RBU update is divided into two stages:
+**
+**   * Stage 1, in which changes are accumulated in an oal/wal file, and
+**   * Stage 2, in which the contents of the wal file are copied into the
+**     main database.
+**
+** The update is visible to non-RBU clients during stage 2. During stage 1
+** non-RBU reader clients may see the original database.
+**
+** If this API is called during stage 2 of the update, output variable 
+** (*pnOne) is set to 10000 to indicate that stage 1 has finished and (*pnTwo)
+** to a value between 0 and 10000 to indicate the permyriadage progress of
+** stage 2. A value of 5000 indicates that stage 2 is half finished, 
+** 9000 indicates that it is 90% finished, and so on.
+**
+** If this API is called during stage 1 of the update, output variable 
+** (*pnTwo) is set to 0 to indicate that stage 2 has not yet started. The
+** value to which (*pnOne) is set depends on whether or not the RBU 
+** database contains an "rbu_count" table. The rbu_count table, if it 
+** exists, must contain the same columns as the following:
+**
+**   CREATE TABLE rbu_count(tbl TEXT PRIMARY KEY, cnt INTEGER) WITHOUT ROWID;
+**
+** There must be one row in the table for each source (data_xxx) table within
+** the RBU database. The 'tbl' column should contain the name of the source
+** table. The 'cnt' column should contain the number of rows within the
+** source table.
+**
+** If the rbu_count table is present and populated correctly and this
+** API is called during stage 1, the *pnOne output variable is set to the
+** permyriadage progress of the same stage. If the rbu_count table does
+** not exist, then (*pnOne) is set to -1 during stage 1. If the rbu_count
+** table exists but is not correctly populated, the value of the *pnOne
+** output variable during stage 1 is undefined.
+*/
+void sqlite3rbu_bp_progress(sqlite3rbu *pRbu, int *pnOne, int *pnTwo);
+
 /*
 ** Create an RBU VFS named zName that accesses the underlying file-system
 ** via existing VFS zParent. Or, if the zParent parameter is passed NULL, 
@@ -66,6 +66,7 @@ static int test_sqlite3rbu_cmd(
    {"create_rbu_delta", 2, ""},  /* 2 */
    {"savestate", 2, ""},         /* 3 */
    {"dbMain_eval", 3, "SQL"},    /* 4 */
+    {"bp_progress", 2, ""},    /* 5 */
    {0,0,0}
  };
  int iCmd;
@@ -136,6 +137,18 @@ static int test_sqlite3rbu_cmd(
      break;
    }

+    case 5: /* bp_progress */ {
+      int one, two;
+      Tcl_Obj *pObj;
+      sqlite3rbu_bp_progress(pRbu, &one, &two);
+
+      pObj = Tcl_NewObj();
+      Tcl_ListObjAppendElement(interp, pObj, Tcl_NewIntObj(one));
+      Tcl_ListObjAppendElement(interp, pObj, Tcl_NewIntObj(two));
+      Tcl_SetObjResult(interp, pObj);
+      break;
+    }
+
    default: /* seems unlikely */
      assert( !"cannot happen" );
      break;
@@ -1,5 +1,5 @@
-C An\salternative\smethod\sof\sencoding\sthe\swildcard\sin\s"SELECT\s*".\s\sThis\sis\san\nexperiment.
-D 2016-03-17T18:41:38.733
+C Version\s3.12.0
+D 2016-03-29T10:14:15.762
 F Makefile.in f53429fb2f313c099283659d0df6f20f932c861f
 F Makefile.linux-gcc 7bc79876b875010e8c8f9502eb935ca92aa3c434
 F Makefile.msc df0bf9ff7f8b3f4dd9fb4cc43f92fe58f6ec5c66
@@ -33,7 +33,7 @@ F config.sub 9ebe4c3b3dab6431ece34f16828b594fb420da55
 F configure c01a159fdf7ea0171ad01ce2937283f2c7972bde x
 F configure.ac 89e4e02a83d8a1528011f8395621b8c3186b4089
 F contrib/sqlitecon.tcl 210a913ad63f9f991070821e599d600bd913e0ad
-F doc/lemon.html c30255bea0fd87a81f082d17a72c9dffbc3f6dd9
+F doc/lemon.html e2118945e5f07ed146b45c9cd2b2dd6eabb8ebf2
 F doc/pager-invariants.txt 27fed9a70ddad2088750c4a2b493b63853da2710
 F doc/vfs-shm.txt e101f27ea02a8387ce46a05be2b1a902a021d37a
 F ext/README.txt 913a7bd3f4837ab14d7e063304181787658b14e1
@@ -98,15 +98,15 @@ F ext/fts3/unicode/mkunicode.tcl 2debed3f582d77b3fdd0b8830880250021571fd8
 F ext/fts3/unicode/parseunicode.tcl da577d1384810fb4e2b209bf3313074353193e95
 F ext/fts5/extract_api_docs.tcl a36e54ec777172ddd3f9a88daf593b00848368e0
 F ext/fts5/fts5.h ff9c2782e8ed890b0de2f697a8d63971939e70c7
-F ext/fts5/fts5Int.h 4060504b7979601d99e1385c2b5713036854979a
+F ext/fts5/fts5Int.h 3677076aecbf645a7f2a019115c6a4ec3272dd78
 F ext/fts5/fts5_aux.c daa57fb45216491814520bbb587e97bf81ced458
 F ext/fts5/fts5_buffer.c 4c1502d4c956cd092c89ce4480867f9d8bf325cd
 F ext/fts5/fts5_config.c 5af9c360e99669d29f06492c370892394aba0857
-F ext/fts5/fts5_expr.c 35e9d92c89e7c7ea0759b73d24da1ecb7630a24b
+F ext/fts5/fts5_expr.c 5ca4bafe29aa3d27683c90e836192e4aefd20a3f
 F ext/fts5/fts5_hash.c f3a7217c86eb8f272871be5f6aa1b6798960a337
-F ext/fts5/fts5_index.c d4f0c12e4f04bbc3a06b6da052039f2ce3e45438
-F ext/fts5/fts5_main.c b8501e1a6a11591c53b18ce7aea7e5386cfb0421
-F ext/fts5/fts5_storage.c f8343db90d8c95a4d4b52f6676e354b4649ffd6e
+F ext/fts5/fts5_index.c fdd82bb421a5d1e64d004acb43f4dd9970c8d2b3
+F ext/fts5/fts5_main.c b4a0fc5bf17f2f1f056ee76cdd7d2af08b360f55
+F ext/fts5/fts5_storage.c 3309c6a8e34b974513016fd1ef47c83f5898f94c
 F ext/fts5/fts5_tcl.c f8731e0508299bd43f1a2eff7dbeaac870768966
 F ext/fts5/fts5_test_mi.c 783b86697ebf773c18fc109992426c0173a055bc
 F ext/fts5/fts5_test_tok.c db08af63673c3a7d39f053b36fd6e065017706be
@@ -117,7 +117,7 @@ F ext/fts5/fts5_vocab.c dba72ca393d71c2588548b51380387f6b44c77a8
 F ext/fts5/fts5parse.y fcc5e92e570d38cab38488b2109cbf67468923b2
 F ext/fts5/mkportersteps.tcl 5acf962d2e0074f701620bb5308155fa1e4a63ba
 F ext/fts5/test/fts5_common.tcl b01c584144b5064f30e6c648145a2dd6bc440841
-F ext/fts5/test/fts5aa.test 7e814df4a0e6c22a6fe2d84f210fdc0b5068a084
+F ext/fts5/test/fts5aa.test bd2d88182b9f7f30d300044048ad14683306b745
 F ext/fts5/test/fts5ab.test 30325a89453280160106be411bba3acf138e6d1b
 F ext/fts5/test/fts5ac.test 55cad4275a1f5acabfe14d8442a8046b47e49e5f
 F ext/fts5/test/fts5ad.test 36995f0586f30f5602074e012b9224c71ec5171c
@@ -141,16 +141,17 @@ F ext/fts5/test/fts5conflict.test 26f4e46c4d31e16221794832a990dc4e30e18de5
 F ext/fts5/test/fts5content.test 9a952c95518a14182dc3b59e3c8fa71cda82a4e1
 F ext/fts5/test/fts5corrupt.test c2ad090192708150d50d961278df10ae7a4b8b62
 F ext/fts5/test/fts5corrupt2.test 26c0a39dd9ff73207e6229f83b50b21d37c7658c
-F ext/fts5/test/fts5corrupt3.test b9558d5b0ca44a8b6247fbb5d4a47592a8976892
+F ext/fts5/test/fts5corrupt3.test f77f65e386231daf62902466b40ff998b2c8ce4f
 F ext/fts5/test/fts5detail.test ef5c690535a797413acaf5ad9b8ab5d49972df69
-F ext/fts5/test/fts5dlidx.test 13871a14641017ae42f6f1055a8067bafd44cb3d
+F ext/fts5/test/fts5determin.test 10648edb75ef1e196b10978fd21a9be0c31e09c3
+F ext/fts5/test/fts5dlidx.test 007e9390c94638760797dbec2990c97c3fa08dfe
 F ext/fts5/test/fts5doclist.test 8edb5b57e5f144030ed74ec00ef6fa4294fed79b
 F ext/fts5/test/fts5ea.test b01e3a18cdfabbff8104a96a5242a06a68a998a0
 F ext/fts5/test/fts5eb.test c516ae0c934be6fd29ec95ea8b5f11f461311535
 F ext/fts5/test/fts5fault1.test e09040d3e17b8c0837101e8c79c8a874c4376fb7
 F ext/fts5/test/fts5fault2.test d8c6c7f916ccbdfc10b2c69530e9dd3bc8313232
 F ext/fts5/test/fts5fault3.test d6e9577d4312e331a913c72931bf131704efc8f3
-F ext/fts5/test/fts5fault4.test 532b6dacb963016cbf7003196bd87fb366540277
+F ext/fts5/test/fts5fault4.test dcbe3043c5611edd350191ea03a8daa190f0de5a
 F ext/fts5/test/fts5fault5.test 10c13a783de3f42a21e3e53e123b62ed0c3a1618
 F ext/fts5/test/fts5fault6.test 9682664d679643ac6736e90c225526cc84073cda
 F ext/fts5/test/fts5fault7.test cb14ea3c1f42394f06f2284abc58eecee6ff8080
@@ -165,6 +166,7 @@ F ext/fts5/test/fts5integrity.test f5e4f8d284385875068ad0f3e894ce43e9de835d
 F ext/fts5/test/fts5matchinfo.test f7dde99697bcb310ea8faa8eb2714d9f4dfc0e1b
 F ext/fts5/test/fts5merge.test 9f65f090d214ff865c56bef4f864aaa1182af6e3
 F ext/fts5/test/fts5merge2.test a6da3c16d694235938d1939f503cfa53f0943d75
+F ext/fts5/test/fts5multiclient.test 5bd54a6bc8ff818be233a50a8dec93da99098f2a
 F ext/fts5/test/fts5near.test b214cddb1c1f1bddf45c75af768f20145f7e71cc
 F ext/fts5/test/fts5onepass.test 7ed9608e258132cb8d55e7c479b08676ad68810c
 F ext/fts5/test/fts5optimize.test 9d3ac53bb9cae58cb070d795db86bcb2f9fec105
@@ -178,7 +180,7 @@ F ext/fts5/test/fts5rank.test 7e9e64eac7245637f6f2033aec4b292aaf611aab
 F ext/fts5/test/fts5rebuild.test 03935f617ace91ed23a6099c7c74d905227ff29b
 F ext/fts5/test/fts5restart.test c17728fdea26e7d0f617d22ad5b4b2862b994c17
 F ext/fts5/test/fts5rowid.test 16908a99d6efc9ba21081b4f2b86b3fc699839a6
-F ext/fts5/test/fts5simple.test ac5006cc3d0d08b3538e1e76c7300de9f24fbed1
+F ext/fts5/test/fts5simple.test cd23d4072ea095d652c9b6db12284cc642e49c98
 F ext/fts5/test/fts5simple2.test 98377ae1ff7749a42c21fe1a139c1ed312522c46
 F ext/fts5/test/fts5simple3.test 8e71733b3d1b0e695011d02c68ebc5ca40b6124e
 F ext/fts5/test/fts5synonym.test 6475d189c2e20d60795808f83e36bf9318708d48
@@ -194,7 +196,7 @@ F ext/fts5/test/fts5update.test 57c7012a7919889048947addae10e0613df45529
 F ext/fts5/test/fts5version.test 978f59541d8cef7e8591f8be2115ec5ccb863e2e
 F ext/fts5/test/fts5vocab.test 480d780aa6b699816c5066225fbd86f3a0239477
 F ext/fts5/tool/fts5speed.tcl b0056f91a55b2d1a3684ec05729de92b042e2f85
-F ext/fts5/tool/fts5txt2db.tcl 1343745b89ca2a1e975c23f836d0cee410052975
+F ext/fts5/tool/fts5txt2db.tcl 526a9979c963f1c54fd50976a05a502e533a4c59
 F ext/fts5/tool/loadfts5.tcl 95b03429ee6b138645703c6ca192c3ac96eaf093
 F ext/fts5/tool/mkfts5c.tcl d1c2a9ab8e0ec690a52316f33dd9b1d379942f45
 F ext/fts5/tool/showfts5.tcl d54da0e067306663e2d5d523965ca487698e722c
@@ -221,7 +223,7 @@ F ext/misc/vfslog.c fe40fab5c077a40477f7e5eba994309ecac6cc95
 F ext/misc/vtshim.c babb0dc2bf116029e3e7c9a618b8a1377045303e
 F ext/misc/wholenumber.c 784b12543d60702ebdd47da936e278aa03076212
 F ext/rbu/rbu.c ba3983dceffa0938532e79142f391737513de023
-F ext/rbu/rbu1.test 57601977588603e82700a43c279bd55282ffa482
+F ext/rbu/rbu1.test 42bd835e019eff789ec241017965277baeb658b1
 F ext/rbu/rbu10.test 046b0980041d30700464a800bbf6733ed2df515d
 F ext/rbu/rbu11.test 9bc68c2d3dbeb1720153626e3bd0466dcc017702
 F ext/rbu/rbu12.test bde22ed0004dd5d1888c72a84ae407e574aeae16
@@ -238,14 +240,15 @@ F ext/rbu/rbuB.test c25bc325b8072a766e56bb76c001866b405925c2
 F ext/rbu/rbuC.test efe47db508a0269b683cb2a1913a425ffd39a831
 F ext/rbu/rbu_common.tcl 0398545fed614f807d5f0ba55a85a51f08ba8f1a
 F ext/rbu/rbucrash.test 8d2ed5d4b05fef6c00c2a6b5f7ead71fa172a695
-F ext/rbu/rbudiff.test 6cc806dc36389292f2a8f5842d0103721df4a07d
+F ext/rbu/rbudiff.test 2df0a8a7d998ecf81764c21eeda3cde5611c5091
 F ext/rbu/rbufault.test cc0be8d5d392d98b0c2d6a51be377ea989250a89
 F ext/rbu/rbufault2.test 9a7f19edd6ea35c4c9f807d8a3db0a03a5670c06
 F ext/rbu/rbufts.test 828cd689da825f0a7b7c53ffc1f6f7fdb6fa5bda
+F ext/rbu/rbuprogress.test 2023a7df2c523e3df1cb532eff811cda385a789a
 F ext/rbu/rbusave.test 0f43b6686084f426ddd040b878426452fd2c2f48
-F ext/rbu/sqlite3rbu.c 5956f8bee63b5ab2b04e65c1801ea0f5920dac92
-F ext/rbu/sqlite3rbu.h 0bdeb3be211aaba7d85445fa36f4701a25a3dbde
-F ext/rbu/test_rbu.c 4a4cdcef4ef9379fc2a21f008805c80b27bcf573
+F ext/rbu/sqlite3rbu.c 007fc4db8c0b95c7ef10162b5864921ef5cc8106
+F ext/rbu/sqlite3rbu.h d7cc99350c10134f358fe1a8997d9225b3f712b2
+F ext/rbu/test_rbu.c 3505641a78b723589b8780d5f9b2faeeb73e037d
 F ext/rtree/README 6315c0d73ebf0ec40dedb5aa0e942bc8b54e3761
 F ext/rtree/rtree.c 0b870ccb7b58b734a2a8e1e2755a7c0ded070920
 F ext/rtree/rtree.h 834dbcb82dc85b2481cde6a07cdadfddc99e9b9e
@@ -294,17 +297,17 @@ F src/auth.c b56c78ebe40a2110fd361379f7e8162d23f92240
 F src/backup.c f60f0aa55d25d853ffde53d0b0370a7bb7ee41ce
 F src/bitvec.c 3ee4c8b2c94ed3a7377256e18199e6ff5cf33f63
 F src/btmutex.c bc87dd3b062cc26edfe79918de2200ccb8d41e73
-F src/btree.c 6eee126fe9d1f57118de9be2be840a4c6e691828
+F src/btree.c 577fb5674e2f0aa0a38246afc19e1885a0b8c9b0
 F src/btree.h a5008b9afe56e8e54ade6c436a910f112defcca9
 F src/btreeInt.h c18b7d2a3494695133e4e60ee36061d37f45d9a5
-F src/build.c 213cbf84e99dd834e6ea46615633656d7ef79321
+F src/build.c 39faaaeecb77eb3936d4bd5024e865e3836ca323
 F src/callback.c 2e76147783386374bf01b227f752c81ec872d730
 F src/complete.c a3634ab1e687055cd002e11b8f43eb75c17da23e
 F src/ctime.c 60e135af364d777a9ab41c97e5e89cd224da6198
 F src/date.c 0b73e681c11fca867fec554750c07fe0d4e417c1
 F src/dbstat.c c845548d4346e606e2f2b7d2e714ace2b8a7dd1b
 F src/delete.c 48802aa3ee6339f576d074336d3ae1b5f40e240f
-F src/expr.c ee11e39874c8ea8f56eb86eb561b7b592c51605b
+F src/expr.c 289ffac5240b60fee0a824d3d5ab2d7bd2630c94
 F src/fault.c 160a0c015b6c2629d3899ed2daf63d75754a32bb
 F src/fkey.c 5cb42d9a59e2a590776fd3fc8ff6f61d40df3c6e
 F src/func.c 552d300265aed09eea21f68ac742a440550c0062
@@ -314,15 +317,15 @@ F src/hash.h c8f3c31722cf3277d03713909761e152a5b81094
 F src/hwtime.h d32741c8f4df852c7d959236615444e2b1063b08
 F src/insert.c 723d5d708cdb61bdd47c00b9f07c75be45aefc09
 F src/legacy.c 75d3023be8f0d2b99d60f905090341a03358c58e
-F src/loadext.c 9e2a41adcaff16ebc1ebff1f336cbf33de55396f
-F src/main.c 29ea8ebb23ceb5159da167e18d5c807fbb1545c4
+F src/loadext.c e70f8f9e97624a232870ea5486e682c813ac3002
+F src/main.c 63791c66321f07b5828bb9161b477d5e0b511d7e
 F src/malloc.c 1443d1ad95d67c21d77af7ae3f44678252f0efec
 F src/mem0.c 6a55ebe57c46ca1a7d98da93aaa07f99f1059645
 F src/mem1.c 6919bcf12f221868ea066eec27e579fed95ce98b
 F src/mem2.c f1940d9e91948dd6a908fbb9ce3835c36b5d83c3
 F src/mem3.c 8768ac94694f31ffaf8b4d0ea5dc08af7010a35a
 F src/mem5.c 9bf955937b07f8c32541c8a9991f33ce3173d944
-F src/memjournal.c 6423a0817ffd8c7a04ef9e5fb974b6b9dd71f8b6
+F src/memjournal.c 5253fd4335a8d9c64e5df25cb9da6329af5242c7
 F src/msvc.h d9ba56c6851227ab44b3f228a35f3f5772296495
 F src/mutex.c 8e45800ee78e0cd1f1f3fe8e398853307f4a085c
 F src/mutex.h 779d588e3b7756ec3ecf7d78cde1d84aba414f85
@@ -330,37 +333,37 @@ F src/mutex_noop.c 9d4309c075ba9cc7249e19412d3d62f7f94839c4
 F src/mutex_unix.c 27bb6cc49485ee46711a6580ab7b3f1402211d23
 F src/mutex_w32.c 5e6fe1c298fb5a8a15aaed4161d5759311431c17
 F src/notify.c 9711a7575036f0d3040ba61bc6e217f13a9888e7
-F src/os.c f89e3ca1c2e3d5015b847aec60371c474acbac82
-F src/os.h 3e57a24e2794a94d3cf2342c6d9a884888cd96bf
+F src/os.c ca10edb445ad2c5fdc7285b49d72bcdf261fa23e
+F src/os.h 91ff889115ecd01f436d3611f7f5ea4dc12d92f1
 F src/os_common.h b2f4707a603e36811d9b1a13278bffd757857b85
 F src/os_setup.h c9d4553b5aaa6f73391448b265b89bed0b890faa
-F src/os_unix.c 4bde11921a2bebcf2167efc8540ddabc814189dc
-F src/os_win.c cbf8c442a0d818d05bcf40b093cb3ebad435b9be
+F src/os_unix.c b1ccb273771f41dbdbe0ba7c1ad63c38ad5972ec
+F src/os_win.c ff870d89f4cb088a04cbf5ea0cbd9ff1b089ff4a
 F src/os_win.h eb7a47aa17b26b77eb97e4823f20a00b8bda12ca
 F src/pager.c 38718a019ca762ba4f6795425d5a54db70d1790d
 F src/pager.h e1d38a2f14849e219df0f91f8323504d134c8a56
-F src/parse.y fc4d11e4f57edc617d1eba062c2dd6604d63aed2
+F src/parse.y 5ea8c81c5c41b27887f41b4a7e1c58470d7d3821
 F src/pcache.c 647bb53a86b7bbcf55ad88089b3ea5a9170b90df
 F src/pcache.h 4d0ccaad264d360981ec5e6a2b596d6e85242545
-F src/pcache1.c 72f644dc9e1468c72922eff5904048427b817051
-F src/pragma.c f0670909e915179fec47e17f72f14660995b8022
+F src/pcache1.c c40cdb93586e21b5dd826b5e671240bd91c26b05
+F src/pragma.c faf42922bb7ab2f6672cb550356c1967abae3c84
 F src/pragma.h 64c78a648751b9f4f297276c4eb7507b14b4628c
 F src/prepare.c 22df6171aec1d86904ed2ad30c2348a5748aa04e
 F src/printf.c 63e6fb12bbe702dd664dc3703776c090383a5a26
 F src/random.c ba2679f80ec82c4190062d756f22d0c358180696
 F src/resolve.c b8f7174e5f8c33c44ded3a25a973d0bb89228c20
 F src/rowset.c 9fe4b3ad7cc00944386bb600233d8f523de07a6e
-F src/select.c 41467e0c63d7282831c3ad829e01630da9576320
+F src/select.c 7849cee0a01952a9c93cd28989daedfa57731143
 F src/shell.c 5e0ab1e708dc294330ccd8230536e1801f60822e
-F src/sqlite.h.in 0235586b3fb639e85998d495c90f007657fd82af
+F src/sqlite.h.in e877f141b15ef68ef28f84714e69d7234f9a071e
 F src/sqlite3.rc 5121c9e10c3964d5755191c80dd1180c122fc3a8
-F src/sqlite3ext.h dfbe62ffd95b99afe2140d8c35b180d11924072d
-F src/sqliteInt.h 8435a8b21e2d7c89a67a25b7a203afc27c5d42d7
-F src/sqliteLimit.h 7b28cf72cbd52f178bfc97ea266445e351f2cd24
+F src/sqlite3ext.h 98f72cbfe00169c39089115427d06ea05fe4b4a2
+F src/sqliteInt.h 3a68b48967a7d62fcb38572d27fd21ab4e4bcca3
+F src/sqliteLimit.h c0373387c287c8d0932510b5547ecde31b5da247
 F src/status.c 70912d7be68e9e2dbc4010c93d344af61d4c59ba
 F src/table.c 5226df15ab9179b9ed558d89575ea0ce37b03fc9
 F src/tclsqlite.c 4bf3bea9b03aeac176ac114700f35f76a1de4c8a
-F src/test1.c 52965bd684ddcd7f22328ebd7d50fd0b6e51f0d4
+F src/test1.c 7187b7e924bfc97780e6fd2a40dad94a32bddca0
 F src/test2.c 5586f43fcd9a1be0830793cf9d354082c261b25b
 F src/test3.c a8887dabbbee3059af338f20d290084a63ed1b0f
 F src/test4.c d168f83cc78d02e8d35567bb5630e40dcd85ac1e
@@ -375,7 +378,7 @@ F src/test_backup.c 2e6e6a081870150f20c526a2e9d0d29cda47d803
 F src/test_bestindex.c 29af3cc3b963ffe5760c85d142b9b3e5302c1e3d
 F src/test_blob.c b2551a9b5573232db5f66f292307c37067937239
 F src/test_btree.c 2e9978eca99a9a4bfa8cae949efb00886860a64f
-F src/test_config.c 0dee90328e3dedf8ba002ee94b6a7e7ea7726fe4
+F src/test_config.c 0fb2571777ef43a8442be4f9c62a25f530872328
 F src/test_demovfs.c 0de72c2c89551629f58486fde5734b7d90758852
 F src/test_devsym.c e7498904e72ba7491d142d5c83b476c4e76993bc
 F src/test_fs.c f10f840ca4f8c72e4837908bd8347ac4bcab074b
@@ -387,7 +390,7 @@ F src/test_intarray.h 9dc57417fb65bc7835cc18548852cc08cc062202
 F src/test_journal.c 5360fbe1d1e4416ca36290562fd5a2e3f70f32aa
 F src/test_loadext.c 337056bae59f80b9eb00ba82088b39d0f4fe6dfd
 F src/test_malloc.c 96df9381a1ff1f6d3805ff7231b9baf1386aaabf
-F src/test_multiplex.c 6a088d8d9d4aad4bec45dd8878af11b15900702d
+F src/test_multiplex.c 6a9de820fcaaf506c59aa14bc8693822333cea48
 F src/test_multiplex.h c08e4e8f8651f0c5e0509b138ff4d5b43ed1f5d3
 F src/test_mutex.c dbdfaff8580071f2212a0deae3325a93a737819c
 F src/test_onefile.c 38f7cbe79d5bafe95bde683cc3a53b8ca16daf10
@@ -414,18 +417,18 @@ F src/treeview.c e4b41a37530a191579d3c53142cc44ee2eb99373
 F src/trigger.c e14840ee0c3e549e758ec9bf3e4146e166002280
 F src/update.c 56b3db7edff0110360a12b76af97c39ebe3ea8b8
 F src/utf.c 699001c79f28e48e9bcdf8a463da029ea660540c
-F src/util.c 34ef7be420f82415ec48131404995ddb6ee7502f
+F src/util.c 8873d696c9ccc4206058c402e09e101f1b81561a
 F src/vacuum.c feb1eabb20987983d9350cad98299b21fa811f52
-F src/vdbe.c 8cf45bb8da77d39f55d108e759d15a57acd0255c
+F src/vdbe.c cd990451b504507b455cc649d46dad51f2835a67
 F src/vdbe.h 6f44193e7be52fd5f7c308175a936555b1e6b101
 F src/vdbeInt.h f88d3115e9bde33b01d81f0dd26d8dd51f995991
 F src/vdbeapi.c 95b1f8e527240a18a9aea41a655b013bf07a7009
-F src/vdbeaux.c a930f913d40e4ca6f6caaef6a7b5906a369fa2b1
+F src/vdbeaux.c c8dd3e4e932bede6363b380519d05c0557ad27ce
 F src/vdbeblob.c 3b570b730109e8f653d9d2081649f6e7015113db
-F src/vdbemem.c 9b0cb32cc267ef026515f15a3594d5ff91fe4dfc
+F src/vdbemem.c fe76c1f866de362d9b8332e59d74aa44f6560d69
 F src/vdbesort.c 307460bfa4de4d1c3901fcd42089159131e34062
 F src/vdbetrace.c f75c5455d8cf389ef86a8bfdfd3177e0e3692484
-F src/vtab.c fd69fd398e23e57ea4ea377d8a44b6998fc569c7
+F src/vtab.c 23b6cdfa996152d43b390504ed4a942c8caf3a00
 F src/vxworks.h d2988f4e5a61a4dfe82c6524dd3d6e4f2ce3cdb9
 F src/wal.c 10deb6b43887662691e5f53d10b3c171c401169b
 F src/wal.h 2f7c831cf3b071fa548bf2d5cac640846a7ff19c
@@ -520,9 +523,9 @@ F test/busy.test 76b4887f8b9160ba903c1ac22e8ff406ad6ae2f0
 F test/cache.test 13bc046b26210471ca6f2889aceb1ea52dc717de
 F test/cacheflush.test af25bb1509df04c1da10e38d8f322d66eceedf61
 F test/capi2.test 011c16da245fdc0106a2785035de6b242c05e738
-F test/capi3.test bf6f0308bbbba1e770dac13aa08e5c2ac61c7324
+F test/capi3.test f0c66919e43d42e1572a69be039e4527a931b00f
 F test/capi3b.test efb2b9cfd127efa84433cd7a2d72ce0454ae0dc4
-F test/capi3c.test 06f6261f9e9b4ef6f76afcd9900f3665408af1c8
+F test/capi3c.test 0b9edb0c2156a964b9271cd5ea7ae56736cc2fcb
 F test/capi3d.test 485048dc5cd07bc68011e4917ad035ad6047ab82
 F test/capi3e.test 3d49c01ef2a1a55f41d73cba2b23b5059ec460fe
 F test/cast.test 4c275cbdc8202d6f9c54a3596701719868ac7dc3
@@ -629,7 +632,7 @@ F test/enc4.test c8f1ce3618508fd0909945beb8b8831feef2c020
 F test/eqp.test 3fe051af50921284189d1970eb653f9fcf5117d2
 F test/errmsg.test f31592a594b44ee121371d25ddd5d63497bb3401
 F test/eval.test a64c9105d6ff163df7cf09d6ac29cdad5922078c
-F test/exclusive.test f48243eaf40e0957215501a12f510a8644d13a02
+F test/exclusive.test 9a57bd66e39144b888ca75c309914fcdefb4e3f9
 F test/exclusive2.test 32798111aae78a5deec980eee383213f189df308
 F test/exec.test e949714dc127eaa5ecc7d723efec1ec27118fdd7
 F test/exists.test 8f7b27b61c2fbe5822f0a1f899c715d14e416e30
@@ -770,7 +773,7 @@ F test/fuzz2.test 76dc35b32b6d6f965259508508abce75a6c4d7e1
 F test/fuzz3.test b47377143f0c80f91ed29d722861077ff34415d5
 F test/fuzz_common.tcl a87dfbb88c2a6b08a38e9a070dabd129e617b45b
 F test/fuzz_malloc.test 328f70aaca63adf29b4c6f06505ed0cf57ca7c26
-F test/fuzzcheck.c 93bb9d309888634615e21ef98d1c30d51483e942
+F test/fuzzcheck.c f01d432d001ba29e7916df8411be7d4e7cddc574
 F test/fuzzdata1.db 7ee3227bad0e7ccdeb08a9e6822916777073c664
 F test/fuzzdata2.db f03a420d3b822cc82e4f894ca957618fbe9c4973
 F test/fuzzdata3.db c6586d3e3cef0fbc18108f9bb649aa77bfc38aba
@@ -889,7 +892,7 @@ F test/manydb.test 28385ae2087967aa05c38624cec7d96ec74feb3e
 F test/mem5.test c6460fba403c5703141348cd90de1c294188c68f
 F test/memdb.test c1f2a343ad14398d5d6debda6ea33e80d0dafcc7
 F test/memleak.test 10b9c6c57e19fc68c32941495e9ba1c50123f6e2
-F test/memsubsys1.test 0311bfb4edd2615e3aa56c7a9cf44574e4229077
+F test/memsubsys1.test 6d268d0ae90f8d61a2356a1838665654d83de518
 F test/memsubsys2.test 3e4a8d0c05fd3e5fa92017c64666730a520c7e08
 F test/minmax.test 42fbad0e81afaa6e0de41c960329f2b2c3526efd
 F test/minmax2.test b44bae787fc7b227597b01b0ca5575c7cb54d3bc
@@ -913,10 +916,10 @@ F test/multiplex.test efd015ca0b5b4a57dc9535b8feb1273eebeadb60
 F test/multiplex2.test 580ca5817c7edbe4cc68fa150609c9473393003a
 F test/multiplex3.test d228f59eac91839a977eac19f21d053f03e4d101
 F test/multiplex4.test e8ae4c4bd70606a5727743241f13b5701990abe4
-F test/mutex1.test e0a44072d98189003deae4b091106f085d94bea8
+F test/mutex1.test ea2cc74d97f077b9e74c84cbd024f14d79a8126f
 F test/mutex2.test bfeaeac2e73095b2ac32285d2756e3a65e681660
 F test/nan.test dacc57f80859c06a433d30839336fe227d2038b3
-F test/nolock.test 96e922d2d3db71c2dd6557c98e8027a28277b415
+F test/nolock.test f196cf8b8fbea4e2ca345140a2b3f3b0da45c76e
 F test/notify1.test 669b2b743618efdc18ca4b02f45423d5d2304abf
 F test/notify2.test 2ecabaa1305083856b7c39cf32816b612740c161
 F test/notify3.test 10ff25cde502e72a92053a2f215d64bece4ef934
@@ -971,7 +974,7 @@ F test/rdonly.test 64e2696c322e3538df0b1ed624e21f9a23ed9ff8
 F test/regexp1.test 497ea812f264d12b6198d6e50a76be4a1973a9d8
 F test/regexp2.test aa7ffcc21350007a78361b82bcf3b74d12227144
 F test/reindex.test 44edd3966b474468b823d481eafef0c305022254
-F test/releasetest.tcl a246ecb14ed594bf44bf77bd21df873971d779bf
+F test/releasetest.tcl f2519836bebefcc7db36b4946e8028d4885baa59
 F test/resolver01.test f4022acafda7f4d40eca94dbf16bc5fc4ac30ceb
 F test/rollback.test 458fe73eb3ffdfdf9f6ba3e9b7350a6220414dea
 F test/rollback2.test 8435d6ff0f13f51d2a4181c232e706005fa90fc5
@@ -1025,9 +1028,9 @@ F test/sharedA.test 0cdf1a76dfa00e6beee66af5b534b1e8df2720f5
 F test/sharedB.test 16cc7178e20965d75278f410943109b77b2e645e
 F test/shared_err.test 2f2aee20db294b9924e81f6ccbe60f19e21e8506
 F test/sharedlock.test 5ede3c37439067c43b0198f580fd374ebf15d304
-F test/shell1.test e7dac9830b7d80432be9cebfae06eff9c18675f5
-F test/shell2.test 2e7a32c9ee03c6398478637d72416d5b9ebd9777
-F test/shell3.test c39453d3012a39ffec944566eca8a6bda10a2284
+F test/shell1.test dff5b20ad989770aface6d714491121172dfe8b0
+F test/shell2.test e242a9912f44f4c23c3d1d802a83e934e84c853b
+F test/shell3.test da513d522ef6f01cee8475dcf8332bff8982b3dd
 F test/shell4.test 69995ee1cc278eb149aa8746ce1f935f4eaf98b9
 F test/shell5.test 50a732c1c2158b1cd62cf53975ce1ea7ce6b9dc9
 F test/shortread1.test bb591ef20f0fd9ed26d0d12e80eee6d7ac8897a3
@@ -1039,8 +1042,8 @@ F test/skipscan2.test d1d1450952b7275f0b0a3a981f0230532743951a
 F test/skipscan3.test ec5bab3f81c7038b43450e7b3062e04a198bdbb5
 F test/skipscan5.test 67817a4b6857c47e0e33ba3e506da6f23ef68de2
 F test/skipscan6.test 5866039d03a56f5bd0b3d172a012074a1d90a15b
-F test/snapshot.test c03eb5ba1602df33c1edc1d12393d6ca7a282a46
-F test/snapshot_fault.test 25973aeb1b86a280800e0bcf1eb5ce70e9ef57ab
+F test/snapshot.test 9ed24c792fb05382814258daf68b2256f23de57f
+F test/snapshot_fault.test 062ff0438a074978d45e9f9a92e7ad459b74ee73
 F test/soak.test 0b5b6375c9f4110c828070b826b3b4b0bb65cd5f
 F test/softheap1.test 843cd84db9891b2d01b9ab64cef3e9020f98d087
 F test/sort.test c2adc635c2564241fefec0b3a68391ef6868fd3b
@@ -1061,7 +1064,7 @@ F test/speedtest1.c 1478cb3fb64ad30f291ddca87ca9dbd72ff552aa
 F test/spellfix.test f9c1f431e2c096c8775fec032952320c0e4700db
 F test/spellfix2.test dfc8f519a3fc204cb2dfa8b4f29821ae90f6f8c3
 F test/spellfix3.test 0f9efaaa502a0e0a09848028518a6fb096c8ad33
-F test/sqldiff1.test 8f6bc7c6a5b3585d350d779c6078869ba402f8f5
+F test/sqldiff1.test 28cd737cf1b0078b1ec1bbf425e674c47785835e
 F test/sqllimits1.test a74ee2a3740b9f9c2437c246d8fb77354862a142
 F test/sqllog.test 6af6cb0b09f4e44e1917e06ce85be7670302517a
 F test/stat.test b65bad7120c52583b8f0054d99eff80718119a77
@@ -1085,7 +1088,7 @@ F test/tclsqlite.test c6d9f546f79d15d0134c1e06583fb3ee0c3afad3
 F test/tempdb.test bd92eba8f20e16a9136e434e20b280794de3cdb6
 F test/temptable.test d2c9b87a54147161bcd1822e30c1d1cd891e5b30
 F test/temptrigger.test 8ec228b0db5d7ebc4ee9b458fc28cb9e7873f5e1
-F test/tester.tcl 859a7ccbe5bd65f0f8b524cd51c318d3edcd3008
+F test/tester.tcl f2b99e912d4da1e5755969b8614febd883885c8b
 F test/thread001.test 9f22fd3525a307ff42a326b6bc7b0465be1745a5
 F test/thread002.test e630504f8a06c00bf8bbe68528774dd96aeb2e58
 F test/thread003.test ee4c9efc3b86a6a2767516a37bd64251272560a7
@@ -1304,8 +1307,8 @@ F test/vtab_alter.test 9e374885248f69e251bdaacf480b04a197f125e5
 F test/vtab_err.test 0d4d8eb4def1d053ac7c5050df3024fd47a3fbd8
 F test/vtab_shared.test 5253bff2355a9a3f014c15337da7e177ab0ef8ad
 F test/wal.test 613efec03e517e1775d86b993a54877d2e29a477
-F test/wal2.test 25ae059e900dbb584e0775627e45415ba5940df1
-F test/wal3.test 5dd734147f1f8f958c5261a1f2775d346d7013ce
+F test/wal2.test 56b0bc95b8693a0be294f8d210c49025dd094bd7
+F test/wal3.test 2a93004bc0fb2b5c29888964024695bade278ab2
 F test/wal4.test 4744e155cd6299c6bd99d3eab1c82f77db9cdb3c
 F test/wal5.test 9c11da7aeccd83a46d79a556ad11a18d3cb15aa9
 F test/wal6.test a9d6aa635b9d63607dabdc11406f5f96ca986635
@@ -1423,7 +1426,7 @@ F tool/speedtest2.tcl ee2149167303ba8e95af97873c575c3e0fab58ff
 F tool/speedtest8.c 2902c46588c40b55661e471d7a86e4dd71a18224
 F tool/speedtest8inst1.c 7ce07da76b5e745783e703a834417d725b7d45fd
 F tool/split-sqlite3c.tcl d9be87f1c340285a3e081eb19b4a247981ed290c
-F tool/sqldiff.c 5a26205111e6fa856d9b1535b1637744dcdb930b
+F tool/sqldiff.c ca315aca4e2d24233e8f2000edea5880c53d1875
 F tool/srcck1.c 4f65e1a6748e42f24c0ea629dddc934d821c729a
 F tool/stack_usage.tcl f8e71b92cdb099a147dad572375595eae55eca43
 F tool/symbols-mingw.sh 4dbcea7e74768305384c9fd2ed2b41bbf9f0414d
@@ -1456,10 +1459,10 @@ F vsixtest/vsixtest.tcl 6a9a6ab600c25a91a7acc6293828957a386a8a93
 F vsixtest/vsixtest.vcxproj.data 2ed517e100c66dc455b492e1a33350c1b20fbcdc
 F vsixtest/vsixtest.vcxproj.filters 37e51ffedcdb064aad6ff33b6148725226cd608e
 F vsixtest/vsixtest_TemporaryKey.pfx e5b1b036facdb453873e7084e1cae9102ccc67a0
-P 97b0e88cc7c3d677217d0bfab4cb4a34a4abb238
-R 7cc43be281eed43b468d8c4ff2e01351
-T *branch * select-wildcard
-T *sym-select-wildcard *
-T -sym-trunk *
+P f6a88cccbc0c62a0b453f4711298c9d5e1882b18
+R 345c74a0cb9568a53ff6b8825c4a812b
+T +bgcolor * #d0c0ff
+T +sym-relaese *
+T +sym-version-3.12.0 *
 U drh
-Z 21f570cb00d593253e312795bb662288
+Z 9c957a50c25c5de44ce9010b47906f30
@@ -1 +1 @@
-41335d8800eaf36c8d3e96ac855c9aef11d49460
+e9bb4cf40f4971974a74468ef922bdee481c988b
@@ -2360,12 +2360,12 @@ int sqlite3BtreeOpen(
    for(i=0; i<db->nDb; i++){
      if( (pSib = db->aDb[i].pBt)!=0 && pSib->sharable ){
        while( pSib->pPrev ){ pSib = pSib->pPrev; }
-        if( p->pBt<pSib->pBt ){
+        if( (uptr)p->pBt<(uptr)pSib->pBt ){
          p->pNext = pSib;
          p->pPrev = 0;
          pSib->pPrev = p;
        }else{
-          while( pSib->pNext && pSib->pNext->pBt<p->pBt ){
+          while( pSib->pNext && (uptr)pSib->pNext->pBt<(uptr)p->pBt ){
            pSib = pSib->pNext;
          }
          p->pNext = pSib->pNext;
@@ -4472,8 +4472,13 @@ static int accessPayload(
 #endif
  assert( offset+amt <= pCur->info.nPayload );

-  if( &aPayload[pCur->info.nLocal] > &pPage->aData[pBt->usableSize] ){
-    /* Trying to read or write past the end of the data is an error */
+  assert( aPayload > pPage->aData );
+  if( (uptr)(aPayload - pPage->aData) > (pBt->usableSize - pCur->info.nLocal) ){
+    /* Trying to read or write past the end of the data is an error.  The
+    ** conditional above is really:
+    **    &aPayload[pCur->info.nLocal] > &pPage->aData[pBt->usableSize]
+    ** but is recast into its current form to avoid integer overflow problems
+    */
    return SQLITE_CORRUPT_BKPT;
  }

@@ -1056,9 +1056,6 @@ void sqlite3AddColumn(Parse *pParse, Token *pName, Token *pType){
  memcpy(z, pName->z, pName->n);
  z[pName->n] = 0;
  sqlite3Dequote(z);
-  zType = z + sqlite3Strlen30(z) + 1;
-  memcpy(zType, pType->z, pType->n);
-  zType[pType->n] = 0;
  for(i=0; i<p->nCol; i++){
    if( sqlite3_stricmp(z, p->aCol[i].zName)==0 ){
      sqlite3ErrorMsg(pParse, "duplicate column name: %s", z);
@@ -1086,7 +1083,11 @@ void sqlite3AddColumn(Parse *pParse, Token *pName, Token *pType){
    pCol->affinity = SQLITE_AFF_BLOB;
    pCol->szEst = 1;
  }else{
+    zType = z + sqlite3Strlen30(z) + 1;
+    memcpy(zType, pType->z, pType->n);
+    zType[pType->n] = 0;
    pCol->affinity = sqlite3AffinityType(zType, &pCol->szEst);
+    pCol->colFlags |= COLFLAG_HASTYPE;
  }
  p->nCol++;
  pParse->constraintName.n = 0;
@@ -1282,7 +1283,7 @@ void sqlite3AddPrimaryKey(
  int sortOrder     /* SQLITE_SO_ASC or SQLITE_SO_DESC */
 ){
  Table *pTab = pParse->pNewTable;
-  const char *zName = 0;
+  Column *pCol = 0;
  int iCol = -1, i;
  int nTerm;
  if( pTab==0 || IN_DECLARE_VTAB ) goto primary_key_exit;
@@ -1294,8 +1295,8 @@ void sqlite3AddPrimaryKey(
  pTab->tabFlags |= TF_HasPrimaryKey;
  if( pList==0 ){
    iCol = pTab->nCol - 1;
-    pTab->aCol[iCol].colFlags |= COLFLAG_PRIMKEY;
-    zName = pTab->aCol[iCol].zName;
+    pCol = &pTab->aCol[iCol];
+    pCol->colFlags |= COLFLAG_PRIMKEY;
    nTerm = 1;
  }else{
    nTerm = pList->nExpr;
@@ -1307,8 +1308,8 @@ void sqlite3AddPrimaryKey(
        const char *zCName = pCExpr->u.zToken;
        for(iCol=0; iCol<pTab->nCol; iCol++){
          if( sqlite3StrICmp(zCName, pTab->aCol[iCol].zName)==0 ){
-            pTab->aCol[iCol].colFlags |= COLFLAG_PRIMKEY;
-            zName = pTab->aCol[iCol].zName;
+            pCol = &pTab->aCol[iCol];
+            pCol->colFlags |= COLFLAG_PRIMKEY;
            break;
          }
        }
@@ -1316,8 +1317,8 @@ void sqlite3AddPrimaryKey(
    }
  }
  if( nTerm==1
-   && zName
-   && sqlite3StrICmp(sqlite3StrNext(zName), "INTEGER")==0
+   && pCol
+   && sqlite3StrICmp(sqlite3ColumnType(pCol,""), "INTEGER")==0
   && sortOrder!=SQLITE_SO_DESC
  ){
    pTab->iPKey = iCol;
@@ -3549,6 +3549,13 @@ void sqlite3ExprIfTrue(Parse *pParse, Expr *pExpr, int dest, int jumpIfNull){
      sqlite3ExprIfFalse(pParse, pExpr->pLeft, dest, jumpIfNull);
      break;
    }
+    case TK_IS:
+    case TK_ISNOT:
+      testcase( op==TK_IS );
+      testcase( op==TK_ISNOT );
+      op = (op==TK_IS) ? TK_EQ : TK_NE;
+      jumpIfNull = SQLITE_NULLEQ;
+      /* Fall thru */
    case TK_LT:
    case TK_LE:
    case TK_GT:
@@ -3564,23 +3571,12 @@ void sqlite3ExprIfTrue(Parse *pParse, Expr *pExpr, int dest, int jumpIfNull){
      assert(TK_LE==OP_Le); testcase(op==OP_Le); VdbeCoverageIf(v,op==OP_Le);
      assert(TK_GT==OP_Gt); testcase(op==OP_Gt); VdbeCoverageIf(v,op==OP_Gt);
      assert(TK_GE==OP_Ge); testcase(op==OP_Ge); VdbeCoverageIf(v,op==OP_Ge);
-      assert(TK_EQ==OP_Eq); testcase(op==OP_Eq); VdbeCoverageIf(v,op==OP_Eq);
-      assert(TK_NE==OP_Ne); testcase(op==OP_Ne); VdbeCoverageIf(v,op==OP_Ne);
-      testcase( regFree1==0 );
-      testcase( regFree2==0 );
-      break;
-    }
-    case TK_IS:
-    case TK_ISNOT: {
-      testcase( op==TK_IS );
-      testcase( op==TK_ISNOT );
-      r1 = sqlite3ExprCodeTemp(pParse, pExpr->pLeft, &regFree1);
-      r2 = sqlite3ExprCodeTemp(pParse, pExpr->pRight, &regFree2);
-      op = (op==TK_IS) ? TK_EQ : TK_NE;
-      codeCompare(pParse, pExpr->pLeft, pExpr->pRight, op,
-                  r1, r2, dest, SQLITE_NULLEQ);
-      VdbeCoverageIf(v, op==TK_EQ);
-      VdbeCoverageIf(v, op==TK_NE);
+      assert(TK_EQ==OP_Eq); testcase(op==OP_Eq);
+      VdbeCoverageIf(v, op==OP_Eq && jumpIfNull==SQLITE_NULLEQ);
+      VdbeCoverageIf(v, op==OP_Eq && jumpIfNull!=SQLITE_NULLEQ);
+      assert(TK_NE==OP_Ne); testcase(op==OP_Ne);
+      VdbeCoverageIf(v, op==OP_Ne && jumpIfNull==SQLITE_NULLEQ);
+      VdbeCoverageIf(v, op==OP_Ne && jumpIfNull!=SQLITE_NULLEQ);
      testcase( regFree1==0 );
      testcase( regFree2==0 );
      break;
@@ -3705,6 +3701,13 @@ void sqlite3ExprIfFalse(Parse *pParse, Expr *pExpr, int dest, int jumpIfNull){
      sqlite3ExprIfTrue(pParse, pExpr->pLeft, dest, jumpIfNull);
      break;
    }
+    case TK_IS:
+    case TK_ISNOT:
+      testcase( pExpr->op==TK_IS );
+      testcase( pExpr->op==TK_ISNOT );
+      op = (pExpr->op==TK_IS) ? TK_NE : TK_EQ;
+      jumpIfNull = SQLITE_NULLEQ;
+      /* Fall thru */
    case TK_LT:
    case TK_LE:
    case TK_GT:
@@ -3720,23 +3723,12 @@ void sqlite3ExprIfFalse(Parse *pParse, Expr *pExpr, int dest, int jumpIfNull){
      assert(TK_LE==OP_Le); testcase(op==OP_Le); VdbeCoverageIf(v,op==OP_Le);
      assert(TK_GT==OP_Gt); testcase(op==OP_Gt); VdbeCoverageIf(v,op==OP_Gt);
      assert(TK_GE==OP_Ge); testcase(op==OP_Ge); VdbeCoverageIf(v,op==OP_Ge);
-      assert(TK_EQ==OP_Eq); testcase(op==OP_Eq); VdbeCoverageIf(v,op==OP_Eq);
-      assert(TK_NE==OP_Ne); testcase(op==OP_Ne); VdbeCoverageIf(v,op==OP_Ne);
-      testcase( regFree1==0 );
-      testcase( regFree2==0 );
-      break;
-    }
-    case TK_IS:
-    case TK_ISNOT: {
-      testcase( pExpr->op==TK_IS );
-      testcase( pExpr->op==TK_ISNOT );
-      r1 = sqlite3ExprCodeTemp(pParse, pExpr->pLeft, &regFree1);
-      r2 = sqlite3ExprCodeTemp(pParse, pExpr->pRight, &regFree2);
-      op = (pExpr->op==TK_IS) ? TK_NE : TK_EQ;
-      codeCompare(pParse, pExpr->pLeft, pExpr->pRight, op,
-                  r1, r2, dest, SQLITE_NULLEQ);
-      VdbeCoverageIf(v, op==TK_EQ);
-      VdbeCoverageIf(v, op==TK_NE);
+      assert(TK_EQ==OP_Eq); testcase(op==OP_Eq);
+      VdbeCoverageIf(v, op==OP_Eq && jumpIfNull!=SQLITE_NULLEQ);
+      VdbeCoverageIf(v, op==OP_Eq && jumpIfNull==SQLITE_NULLEQ);
+      assert(TK_NE==OP_Ne); testcase(op==OP_Ne);
+      VdbeCoverageIf(v, op==OP_Ne && jumpIfNull!=SQLITE_NULLEQ);
+      VdbeCoverageIf(v, op==OP_Ne && jumpIfNull==SQLITE_NULLEQ);
      testcase( regFree1==0 );
      testcase( regFree2==0 );
      break;
@@ -4259,3 +4251,26 @@ void sqlite3ClearTempRegCache(Parse *pParse){
  pParse->nTempReg = 0;
  pParse->nRangeReg = 0;
 }
+
+/*
+** Validate that no temporary register falls within the range of
+** iFirst..iLast, inclusive.  This routine is only call from within assert()
+** statements.
+*/
+#ifdef SQLITE_DEBUG
+int sqlite3NoTempsInRange(Parse *pParse, int iFirst, int iLast){
+  int i;
+  if( pParse->nRangeReg>0
+   && pParse->iRangeReg+pParse->nRangeReg<iLast
+   && pParse->iRangeReg>=iFirst
+  ){
+     return 0;
+  }
+  for(i=0; i<pParse->nTempReg; i++){
+    if( pParse->aTempReg[i]>=iFirst && pParse->aTempReg[i]<=iLast ){
+      return 0;
+    }
+  }
+  return 1;
+}
+#endif /* SQLITE_DEBUG */
@@ -414,7 +414,9 @@ static const sqlite3_api_routines sqlite3Apis = {
  /* Version 3.10.0 and later */
  sqlite3_status64,
  sqlite3_strlike,
-  sqlite3_db_cacheflush
+  sqlite3_db_cacheflush,
+  /* Version 3.12.0 and later */
+  sqlite3_system_errno
 };

 /*
@@ -2240,6 +2240,9 @@ int sqlite3_extended_errcode(sqlite3 *db){
  }
  return db->errCode;
 }
+int sqlite3_system_errno(sqlite3 *db){
+  return db ? db->iSysErrno : 0;
+}  

 /*
 ** Return a string that describes the kind of error specified in the
@@ -3340,8 +3343,7 @@ int sqlite3_table_column_metadata(
  **        explicitly declared column. Copy meta information from *pCol.
  */ 
  if( pCol ){
-    zDataType = sqlite3StrNext(pCol->zName);
-    if( zDataType[0]==0 ) zDataType = 0;
+    zDataType = sqlite3ColumnType(pCol,0);
    zCollSeq = pCol->zColl;
    notnull = pCol->notNull!=0;
    primarykey  = (pCol->colFlags & COLFLAG_PRIMKEY)!=0;
@@ -403,5 +403,5 @@ int sqlite3JournalIsInMemory(sqlite3_file *p){
 ** pVfs to create the underlying on-disk files.
 */
 int sqlite3JournalSize(sqlite3_vfs *pVfs){
-  return MAX(pVfs->szOsFile, sizeof(MemJournal));
+  return MAX(pVfs->szOsFile, (int)sizeof(MemJournal));
 }
@@ -262,6 +262,9 @@ int sqlite3OsRandomness(sqlite3_vfs *pVfs, int nByte, char *zBufOut){
 int sqlite3OsSleep(sqlite3_vfs *pVfs, int nMicro){
  return pVfs->xSleep(pVfs, nMicro);
 }
+int sqlite3OsGetLastError(sqlite3_vfs *pVfs){
+  return pVfs->xGetLastError ? pVfs->xGetLastError(pVfs, 0, 0) : 0;
+}
 int sqlite3OsCurrentTimeInt64(sqlite3_vfs *pVfs, sqlite3_int64 *pTimeOut){
  int rc;
  /* IMPLEMENTATION-OF: R-49045-42493 SQLite will use the xCurrentTimeInt64()
@@ -197,6 +197,7 @@ void sqlite3OsDlClose(sqlite3_vfs *, void *);
 #endif /* SQLITE_OMIT_LOAD_EXTENSION */
 int sqlite3OsRandomness(sqlite3_vfs *, int, char *);
 int sqlite3OsSleep(sqlite3_vfs *, int);
+int sqlite3OsGetLastError(sqlite3_vfs*);
 int sqlite3OsCurrentTimeInt64(sqlite3_vfs *, sqlite3_int64*);

 /*
@@ -1366,7 +1366,7 @@ static void verifyDbFile(unixFile *pFile){
    sqlite3_log(SQLITE_WARNING, "cannot fstat db file %s", pFile->zPath);
    return;
  }
-  if( buf.st_nlink==0 && (pFile->ctrlFlags & UNIXFILE_DELETE)==0 ){
+  if( buf.st_nlink==0 ){
    sqlite3_log(SQLITE_WARNING, "file unlinked while open: %s", pFile->zPath);
    return;
  }
@@ -6264,23 +6264,18 @@ static int unixCurrentTime(sqlite3_vfs *NotUsed, double *prNow){
 # define unixCurrentTime 0
 #endif

-#ifndef SQLITE_OMIT_DEPRECATED
 /*
-** We added the xGetLastError() method with the intention of providing
-** better low-level error messages when operating-system problems come up
-** during SQLite operation.  But so far, none of that has been implemented
-** in the core.  So this routine is never called.  For now, it is merely
-** a place-holder.
+** The xGetLastError() method is designed to return a better
+** low-level error message when operating-system problems come up
+** during SQLite operation.  Only the integer return code is currently
+** used.
 */
 static int unixGetLastError(sqlite3_vfs *NotUsed, int NotUsed2, char *NotUsed3){
  UNUSED_PARAMETER(NotUsed);
  UNUSED_PARAMETER(NotUsed2);
  UNUSED_PARAMETER(NotUsed3);
-  return 0;
+  return errno;
 }
-#else
-# define unixGetLastError 0
-#endif


 /*
@@ -344,11 +344,23 @@ struct winFile {
 #  define SQLITE_WIN32_HEAP_CREATE    (TRUE)
 #endif

+/*
+ * This is cache size used in the calculation of the initial size of the
+ * Win32-specific heap.  It cannot be negative.
+ */
+#ifndef SQLITE_WIN32_CACHE_SIZE
+#  if SQLITE_DEFAULT_CACHE_SIZE>=0
+#    define SQLITE_WIN32_CACHE_SIZE (SQLITE_DEFAULT_CACHE_SIZE)
+#  else
+#    define SQLITE_WIN32_CACHE_SIZE (-(SQLITE_DEFAULT_CACHE_SIZE))
+#  endif
+#endif
+
 /*
 * The initial size of the Win32-specific heap.  This value may be zero.
 */
 #ifndef SQLITE_WIN32_HEAP_INIT_SIZE
-#  define SQLITE_WIN32_HEAP_INIT_SIZE ((SQLITE_DEFAULT_CACHE_SIZE) * \
+#  define SQLITE_WIN32_HEAP_INIT_SIZE ((SQLITE_WIN32_CACHE_SIZE) * \
                                       (SQLITE_DEFAULT_PAGE_SIZE) + 4194304)
 #endif

@@ -5424,8 +5436,8 @@ static int winRandomness(sqlite3_vfs *pVfs, int nBuf, char *zBuf){
  EntropyGatherer e;
  UNUSED_PARAMETER(pVfs);
  memset(zBuf, 0, nBuf);
-#if defined(_MSC_VER) && _MSC_VER>=1400
-  rand_s((int*)zBuf); /* rand_s() is not available with MinGW */
+#if defined(_MSC_VER) && _MSC_VER>=1400 && !SQLITE_OS_WINCE
+  rand_s((unsigned int*)zBuf); /* rand_s() is not available with MinGW */
 #endif /* defined(_MSC_VER) && _MSC_VER>=1400 */
  e.a = (unsigned char*)zBuf;
  e.na = nBuf;
@@ -5584,8 +5596,10 @@ static int winCurrentTime(sqlite3_vfs *pVfs, double *prNow){
 ** sqlite3_errmsg(), possibly making IO errors easier to debug.
 */
 static int winGetLastError(sqlite3_vfs *pVfs, int nBuf, char *zBuf){
+  DWORD e = osGetLastError();
  UNUSED_PARAMETER(pVfs);
-  return winGetLastErrorMsg(osGetLastError(), nBuf, zBuf);
+  if( nBuf>0 ) winGetLastErrorMsg(e, nBuf, zBuf);
+  return e;
 }

 /*
@@ -550,13 +550,13 @@ selcollist(A) ::= sclp(A) expr(X) as(Y).     {
   sqlite3ExprListSetSpan(pParse,A,&X);
 }
 selcollist(A) ::= sclp(A) STAR. {
-  A = sqlite3WildcardResultSet(pParse,A);
+  Expr *p = sqlite3Expr(pParse->db, TK_ASTERISK, 0);
+  A = sqlite3ExprListAppend(pParse, A, p);
 }
 selcollist(A) ::= sclp(A) nm(X) DOT STAR(Y). {
  Expr *pRight = sqlite3PExpr(pParse, TK_ASTERISK, 0, 0, &Y);
  Expr *pLeft = sqlite3PExpr(pParse, TK_ID, 0, 0, &X);
  Expr *pDot = sqlite3PExpr(pParse, TK_DOT, pLeft, pRight, 0);
-  if( pDot ) ExprSetProperty(pDot, EP_Wildcard);
  A = sqlite3ExprListAppend(pParse,A, pDot);
 }

@@ -348,7 +348,6 @@ static void *pcache1Alloc(int nByte){
 ** Free an allocated buffer obtained from pcache1Alloc().
 */
 static void pcache1Free(void *p){
-  int nFreed = 0;
  if( p==0 ) return;
  if( SQLITE_WITHIN(p, pcache1.pStart, pcache1.pEnd) ){
    PgFreeslot *pSlot;
@@ -365,10 +364,13 @@ static void pcache1Free(void *p){
    assert( sqlite3MemdebugHasType(p, MEMTYPE_PCACHE) );
    sqlite3MemdebugSetType(p, MEMTYPE_HEAP);
 #ifndef SQLITE_DISABLE_PAGECACHE_OVERFLOW_STATS
-    nFreed = sqlite3MallocSize(p);
-    sqlite3_mutex_enter(pcache1.mutex);
-    sqlite3StatusDown(SQLITE_STATUS_PAGECACHE_OVERFLOW, nFreed);
-    sqlite3_mutex_leave(pcache1.mutex);
+    {
+      int nFreed = 0;
+      nFreed = sqlite3MallocSize(p);
+      sqlite3_mutex_enter(pcache1.mutex);
+      sqlite3StatusDown(SQLITE_STATUS_PAGECACHE_OVERFLOW, nFreed);
+      sqlite3_mutex_leave(pcache1.mutex);
+    }
 #endif
    sqlite3_free(p);
  }
@@ -1066,7 +1066,6 @@ void sqlite3Pragma(
      setAllColumnNames(v, 6, azCol); assert( 6==ArraySize(azCol) );
      sqlite3ViewGetColumnNames(pParse, pTab);
      for(i=0, pCol=pTab->aCol; i<pTab->nCol; i++, pCol++){
-        const char *zName;
        if( IsHiddenColumn(pCol) ){
          nHidden++;
          continue;
@@ -1079,11 +1078,10 @@ void sqlite3Pragma(
          for(k=1; k<=pTab->nCol && pPk->aiColumn[k-1]!=i; k++){}
        }
        assert( pCol->pDflt==0 || pCol->pDflt->op==TK_SPAN );
-        zName = pCol->zName;
        sqlite3VdbeMultiLoad(v, 1, "issisi",
               i-nHidden,
-               zName,
-               sqlite3StrNext(zName),
+               pCol->zName,
+               sqlite3ColumnType(pCol,""),
               pCol->notNull ? 1 : 0,
               pCol->pDflt ? pCol->pDflt->u.zToken : 0,
               k);
@@ -1441,6 +1439,8 @@ void sqlite3Pragma(
      Hash *pTbls;
      int *aRoot;
      int cnt = 0;
+      int mxIdx = 0;
+      int nIdx;

      if( OMIT_TEMPDB && i==1 ) continue;
      if( iDb>=0 && i!=iDb ) continue;
@@ -1462,7 +1462,8 @@ void sqlite3Pragma(
        Table *pTab = sqliteHashData(x);
        Index *pIdx;
        if( HasRowid(pTab) ) cnt++;
-        for(pIdx=pTab->pIndex; pIdx; pIdx=pIdx->pNext){ cnt++; }
+        for(nIdx=0, pIdx=pTab->pIndex; pIdx; pIdx=pIdx->pNext, nIdx++){ cnt++; }
+        if( nIdx>mxIdx ) mxIdx = nIdx;
      }
      aRoot = sqlite3DbMallocRawNN(db, sizeof(int)*(cnt+1));
      if( aRoot==0 ) break;
@@ -1477,7 +1478,7 @@ void sqlite3Pragma(
      aRoot[cnt] = 0;

      /* Make sure sufficient number of registers have been allocated */
-      pParse->nMem = MAX( pParse->nMem, 14 );
+      pParse->nMem = MAX( pParse->nMem, 8+mxIdx );

      /* Do the b-tree integrity checks */
      sqlite3VdbeAddOp4(v, OP_IntegrityCk, 2, cnt, 1, (char*)aRoot,P4_INTARRAY);
@@ -1514,7 +1515,8 @@ void sqlite3Pragma(
        for(j=0, pIdx=pTab->pIndex; pIdx; pIdx=pIdx->pNext, j++){
          sqlite3VdbeAddOp2(v, OP_Integer, 0, 8+j); /* index entries counter */
        }
-        pParse->nMem = MAX(pParse->nMem, 8+j);
+        assert( pParse->nMem>=8+j );
+        assert( sqlite3NoTempsInRange(pParse,1,7+j) );
        sqlite3VdbeAddOp2(v, OP_Rewind, iDataCur, 0); VdbeCoverage(v);
        loopTop = sqlite3VdbeAddOp2(v, OP_AddImm, 7, 1);
        /* Verify that all NOT NULL columns really are NOT NULL */
@@ -92,16 +92,6 @@ void sqlite3SelectDestInit(SelectDest *pDest, int eDest, int iParm){
  pDest->nSdst = 0;
 }

-/*
-** Return an expression list that represents the wildcard result set
-** of a SELECT.
-*/
-ExprList *sqlite3WildcardResultSet(Parse *pParse, ExprList *pPrior){
-  Expr *p = sqlite3Expr(pParse->db, TK_ASTERISK, 0);
-  if( p ) ExprSetProperty(p, EP_Wildcard);
-  return sqlite3ExprListAppend(pParse, pPrior, p);
-}
-

 /*
 ** Allocate a new Select structure and return a pointer to that
@@ -127,7 +117,9 @@ Select *sqlite3SelectNew(
    assert( db->mallocFailed );
    pNew = &standin;
  }
-  if( pEList==0 ) pEList = sqlite3WildcardResultSet(pParse,0);
+  if( pEList==0 ){
+    pEList = sqlite3ExprListAppend(pParse, 0, sqlite3Expr(db,TK_ASTERISK,0));
+  }
  pNew->pEList = pEList;
  pNew->op = TK_SELECT;
  pNew->selFlags = selFlags;
@@ -1438,7 +1430,7 @@ static const char *columnTypeImpl(
          zOrigCol = "rowid";
        }else{
          zOrigCol = pTab->aCol[iCol].zName;
-          zType = sqlite3StrNext(zOrigCol);
+          zType = sqlite3ColumnType(&pTab->aCol[iCol],0);
          estWidth = pTab->aCol[iCol].szEst;
        }
        zOrigTab = pTab->zName;
@@ -1450,7 +1442,7 @@ static const char *columnTypeImpl(
        if( iCol<0 ){
          zType = "INTEGER";
        }else{
-          zType = sqlite3StrNext(pTab->aCol[iCol].zName);
+          zType = sqlite3ColumnType(&pTab->aCol[iCol],0);
          estWidth = pTab->aCol[iCol].szEst;
        }
 #endif
@@ -3960,7 +3952,7 @@ static int convertCompoundSelectToSubquery(Walker *pWalker, Select *p){
  if( pNewSrc==0 ) return WRC_Abort;
  *pNew = *p;
  p->pSrc = pNewSrc;
-  p->pEList = sqlite3WildcardResultSet(pParse,0);
+  p->pEList = sqlite3ExprListAppend(pParse, 0, sqlite3Expr(db, TK_ASTERISK, 0));
  p->op = TK_SELECT;
  p->pWhere = 0;
  pNew->pGroupBy = 0;
@@ -4318,8 +4310,18 @@ static int selectExpander(Walker *pWalker, Select *p){
  ** list.  The following code just has to locate the TK_ASTERISK
  ** expressions and expand each one to the list of all columns in
  ** all tables.
+  **
+  ** The first loop just checks to see if there are any "*" operators
+  ** that need expanding.
  */
-  if( sqlite3ExprListFlags(pEList) & EP_Wildcard ){
+  for(k=0; k<pEList->nExpr; k++){
+    pE = pEList->a[k].pExpr;
+    if( pE->op==TK_ASTERISK ) break;
+    assert( pE->op!=TK_DOT || pE->pRight!=0 );
+    assert( pE->op!=TK_DOT || (pE->pLeft!=0 && pE->pLeft->op==TK_ID) );
+    if( pE->op==TK_DOT && pE->pRight->op==TK_ASTERISK ) break;
+  }
+  if( k<pEList->nExpr ){
    /*
    ** If we get here it means the result set contains one or more "*"
    ** operators that need to be expanded.  Loop through each expression
@@ -4335,7 +4337,9 @@ static int selectExpander(Walker *pWalker, Select *p){
      pE = a[k].pExpr;
      pRight = pE->pRight;
      assert( pE->op!=TK_DOT || pRight!=0 );
-      if( !ExprHasProperty(pE, EP_Wildcard) ){
+      if( pE->op!=TK_ASTERISK
+       && (pE->op!=TK_DOT || pRight->op!=TK_ASTERISK)
+      ){
        /* This particular expression does not need to be expanded.
        */
        pNew = sqlite3ExprListAppend(pParse, pNew, a[k].pExpr);
@@ -7915,6 +7915,18 @@ void sqlite3_stmt_scanstatus_reset(sqlite3_stmt*);
 */
 int sqlite3_db_cacheflush(sqlite3*);

+/*
+** CAPI3REF: Low-level system error code
+**
+** ^Attempt to return the underlying operating system error code or error
+** number that caused the most reason I/O error or failure to open a file.
+** The return value is OS-dependent.  For example, on unix systems, after
+** [sqlite3_open_v2()] returns [SQLITE_CANTOPEN], this interface could be
+** called to get back the underlying "errno" that caused the problem, such
+** as ENOSPC, EAUTH, EISDIR, and so forth.  
+*/
+int sqlite3_system_errno(sqlite3*);
+
 /*
 ** CAPI3REF: Database Snapshot
 ** KEYWORDS: {snapshot}
@@ -279,6 +279,8 @@ struct sqlite3_api_routines {
  int (*status64)(int,sqlite3_int64*,sqlite3_int64*,int);
  int (*strlike)(const char*,const char*,unsigned int);
  int (*db_cacheflush)(sqlite3*);
+  /* Version 3.12.0 and later */
+  int (*system_errno)(sqlite3*);
 };

 /*
@@ -522,6 +524,8 @@ struct sqlite3_api_routines {
 #define sqlite3_status64               sqlite3_api->status64
 #define sqlite3_strlike                sqlite3_api->strlike
 #define sqlite3_db_cacheflush          sqlite3_api->db_cacheflush
+/* Version 3.12.0 and later */
+#define sqlite3_system_errno           sqlite3_api->system_errno
 #endif /* !defined(SQLITE_CORE) && !defined(SQLITE_OMIT_LOAD_EXTENSION) */

 #if !defined(SQLITE_CORE) && !defined(SQLITE_OMIT_LOAD_EXTENSION)
@@ -181,21 +181,6 @@
 # define SQLITE_PTR_TO_INT(X)  ((int)(X))
 #endif

-/*
-** The SQLITE_WITHIN(P,S,E) macro checks to see if pointer P points to
-** something between S (inclusive) and E (exclusive).
-**
-** In other words, S is a buffer and E is a pointer to the first byte after
-** the end of buffer S.  This macro returns true if P points to something
-** contained within the buffer S.
-*/
-#if defined(HAVE_STDINT_H)
-# define SQLITE_WITHIN(P,S,E) \
-    ((uintptr_t)(P)>=(uintptr_t)(S) && (uintptr_t)(P)<(uintptr_t)(E))
-#else
-# define SQLITE_WITHIN(P,S,E) ((P)>=(S) && (P)<(E))
-#endif
-
 /*
 ** A macro to hint to the compiler that a function should not be
 ** inlined.
@@ -583,8 +568,12 @@
 /*
 ** Macros to compute minimum and maximum of two numbers.
 */
-#define MIN(A,B) ((A)<(B)?(A):(B))
-#define MAX(A,B) ((A)>(B)?(A):(B))
+#ifndef MIN
+# define MIN(A,B) ((A)<(B)?(A):(B))
+#endif
+#ifndef MAX
+# define MAX(A,B) ((A)>(B)?(A):(B))
+#endif

 /*
 ** Swap two objects of type TYPE.
@@ -713,6 +702,27 @@ typedef INT16_TYPE LogEst;
 # endif
 #endif

+/* The uptr type is an unsigned integer large enough to hold a pointer
+*/
+#if defined(HAVE_STDINT_H)
+  typedef uintptr_t uptr;
+#elif SQLITE_PTRSIZE==4
+  typedef u32 uptr;
+#else
+  typedef u64 uptr;
+#endif
+
+/*
+** The SQLITE_WITHIN(P,S,E) macro checks to see if pointer P points to
+** something between S (inclusive) and E (exclusive).
+**
+** In other words, S is a buffer and E is a pointer to the first byte after
+** the end of buffer S.  This macro returns true if P points to something
+** contained within the buffer S.
+*/
+#define SQLITE_WITHIN(P,S,E) (((uptr)(P)>=(uptr)(S))&&((uptr)(P)<(uptr)(E)))
+
+
 /*
 ** Macros to determine whether the machine is big or little endian,
 ** and whether or not that determination is run-time or compile-time.
@@ -1220,6 +1230,7 @@ struct sqlite3 {
  unsigned int openFlags;       /* Flags passed to sqlite3_vfs.xOpen() */
  int errCode;                  /* Most recent error code (SQLITE_*) */
  int errMask;                  /* & result codes with this before returning */
+  int iSysErrno;                /* Errno value from last system error */
  u16 dbOptFlags;               /* Flags to enable/disable optimizations */
  u8 enc;                       /* Text encoding */
  u8 autoCommit;                /* The auto-commit flag. */
@@ -1593,6 +1604,7 @@ struct Column {
 */
 #define COLFLAG_PRIMKEY  0x0001    /* Column is part of the primary key */
 #define COLFLAG_HIDDEN   0x0002    /* A hidden column in a virtual table */
+#define COLFLAG_HASTYPE  0x0004    /* Type name follows column name */

 /*
 ** A "Collating Sequence" is defined by an instance of the following
@@ -2257,7 +2269,6 @@ struct Expr {
 #define EP_CanBeNull 0x100000 /* Can be null despite NOT NULL constraint */
 #define EP_Subquery  0x200000 /* Tree contains a TK_SELECT operator */
 #define EP_Alias     0x400000 /* Is an alias for a result set column */
-#define EP_Wildcard  0x800000 /* The "*" or "TABLE.*" of a SELECT result */

 /*
 ** Combinations of two or more EP_* flags
@@ -3297,7 +3308,7 @@ int sqlite3IsIdChar(u8);
 */
 int sqlite3StrICmp(const char*,const char*);
 int sqlite3Strlen30(const char*);
-const char *sqlite3StrNext(const char*);
+char *sqlite3ColumnType(Column*,char*);
 #define sqlite3StrNICmp sqlite3_strnicmp

 int sqlite3MallocInit(void);
@@ -3420,6 +3431,9 @@ void sqlite3ReleaseTempReg(Parse*,int);
 int sqlite3GetTempRange(Parse*,int);
 void sqlite3ReleaseTempRange(Parse*,int,int);
 void sqlite3ClearTempRegCache(Parse*);
+#ifdef SQLITE_DEBUG
+int sqlite3NoTempsInRange(Parse*,int,int);
+#endif
 Expr *sqlite3ExprAlloc(sqlite3*,int,const Token*,int);
 Expr *sqlite3Expr(sqlite3*,int,const char*);
 void sqlite3ExprAttachSubtrees(sqlite3*,Expr*,Expr*,Expr*);
@@ -3528,7 +3542,6 @@ Index *sqlite3AllocateIndexObject(sqlite3*,i16,int,char**);
 Index *sqlite3CreateIndex(Parse*,Token*,Token*,SrcList*,ExprList*,int,Token*,
                          Expr*, int, int);
 void sqlite3DropIndex(Parse*, SrcList*, int);
-ExprList *sqlite3WildcardResultSet(Parse*,ExprList*);
 int sqlite3Select(Parse*, Select*, SelectDest*);
 Select *sqlite3SelectNew(Parse*,ExprList*,SrcList*,Expr*,ExprList*,
                         Expr*,ExprList*,u32,Expr*,Expr*);
@@ -3765,6 +3778,7 @@ int sqlite3Atoi64(const char*, i64*, int, u8);
 int sqlite3DecOrHexToI64(const char*, i64*);
 void sqlite3ErrorWithMsg(sqlite3*, int, const char*,...);
 void sqlite3Error(sqlite3*,int);
+void sqlite3SystemError(sqlite3*,int);
 void *sqlite3HexToBlob(sqlite3*, const char *z, int n);
 u8 sqlite3HexToInt(int h);
 int sqlite3TwoPartName(Parse *, Token *, Token *, Token **);
@@ -101,8 +101,8 @@
 ** The suggested maximum number of in-memory pages to use for
 ** the main database table and for temporary tables.
 **
-** IMPLEMENTATION-OF: R-31093-59126 The default suggested cache size
-** is 2000*1024 bytes.
+** IMPLEMENTATION-OF: R-30185-15359 The default suggested cache size is -2000,
+** which means the cache size is limited to 2048000 bytes of memory.
 ** IMPLEMENTATION-OF: R-48205-43578 The default suggested cache size can be
 ** altered using the SQLITE_DEFAULT_CACHE_SIZE compile-time options.
 */
@@ -1931,6 +1931,8 @@ static int test_load_extension(
 #ifdef SQLITE_OMIT_LOAD_EXTENSION
  rc = SQLITE_ERROR;
  zErr = sqlite3_mprintf("this build omits sqlite3_load_extension()");
+  (void)zProc;
+  (void)zFile;
 #else
  rc = sqlite3_load_extension(db, zFile, zProc, &zErr);
 #endif
@@ -4847,6 +4849,29 @@ static int test_db_cacheflush(
  return TCL_OK;
 }

+/*
+** Usage:  sqlite3_system_errno DB
+**
+** Return the low-level system errno value.
+*/
+static int test_system_errno(
+  void * clientData,
+  Tcl_Interp *interp,
+  int objc,
+  Tcl_Obj *CONST objv[]
+){
+  sqlite3 *db;
+  int iErrno;
+  if( objc!=2 ){
+    Tcl_WrongNumArgs(interp, 1, objv, "DB");
+    return TCL_ERROR;
+  }
+  if( getDbPointer(interp, Tcl_GetString(objv[1]), &db) ) return TCL_ERROR;
+  iErrno = sqlite3_system_errno(db);
+  Tcl_SetObjResult(interp, Tcl_NewIntObj(iErrno));
+  return TCL_OK;
+}
+
 /*
 ** Usage:  sqlite3_db_filename DB DBNAME
 **
@@ -7084,6 +7109,7 @@ int Sqlitetest1_Init(Tcl_Interp *interp){
     { "sqlite3_release_memory",        test_release_memory,     0},
     { "sqlite3_db_release_memory",     test_db_release_memory,  0},
     { "sqlite3_db_cacheflush",         test_db_cacheflush,      0},
+     { "sqlite3_system_errno",          test_system_errno,       0},
     { "sqlite3_db_filename",           test_db_filename,        0},
     { "sqlite3_db_readonly",           test_db_readonly,        0},
     { "sqlite3_soft_heap_limit",       test_soft_heap_limit,    0},
@@ -79,6 +79,13 @@ static void set_options(Tcl_Interp *interp){
  Tcl_SetVar2(interp, "sqlite_options", "debug", "0", TCL_GLOBAL_ONLY);
 #endif

+#ifdef SQLITE_DEFAULT_CKPTFULLFSYNC
+  Tcl_SetVar2(interp, "sqlite_options", "default_ckptfullfsync", 
+              SQLITE_DEFAULT_CKPTFULLFSYNC ? "1" : "0", TCL_GLOBAL_ONLY);
+#else
+  Tcl_SetVar2(interp, "sqlite_options", "default_ckptfullfsync", "0", TCL_GLOBAL_ONLY);
+#endif
+
 #ifdef SQLITE_DIRECT_OVERFLOW_READ
  Tcl_SetVar2(interp, "sqlite_options", "direct_read", "1", TCL_GLOBAL_ONLY);
 #else
@@ -97,6 +104,12 @@ static void set_options(Tcl_Interp *interp){
  Tcl_SetVar2(interp, "sqlite_options", "lfs", "1", TCL_GLOBAL_ONLY);
 #endif

+#ifdef SQLITE_DISABLE_PAGECACHE_OVERFLOW_STATS
+  Tcl_SetVar2(interp, "sqlite_options", "pagecache_overflow_stats","0",TCL_GLOBAL_ONLY);
+#else
+  Tcl_SetVar2(interp, "sqlite_options", "pagecache_overflow_stats","1",TCL_GLOBAL_ONLY);
+#endif
+
 #if SQLITE_MAX_MMAP_SIZE>0
  Tcl_SetVar2(interp, "sqlite_options", "mmap", "1", TCL_GLOBAL_ONLY);
 #else
@@ -576,7 +589,11 @@ Tcl_SetVar2(interp, "sqlite_options", "mergesort", "1", TCL_GLOBAL_ONLY);
 #endif

  Tcl_SetVar2(interp, "sqlite_options", "threadsafe", 
-      STRINGVALUE(SQLITE_THREADSAFE), TCL_GLOBAL_ONLY);
+      SQLITE_THREADSAFE ? "1" : "0", TCL_GLOBAL_ONLY);
+  Tcl_SetVar2(interp, "sqlite_options", "threadsafe1", 
+      SQLITE_THREADSAFE==1 ? "1" : "0", TCL_GLOBAL_ONLY);
+  Tcl_SetVar2(interp, "sqlite_options", "threadsafe2", 
+      SQLITE_THREADSAFE==2 ? "1" : "0", TCL_GLOBAL_ONLY);
  assert( sqlite3_threadsafe()==SQLITE_THREADSAFE );

 #ifdef SQLITE_OMIT_TEMPDB
@@ -717,7 +717,11 @@ static int multiplexCurrentTime(sqlite3_vfs *a, double *b){
  return gMultiplex.pOrigVfs->xCurrentTime(gMultiplex.pOrigVfs, b);
 }
 static int multiplexGetLastError(sqlite3_vfs *a, int b, char *c){
-  return gMultiplex.pOrigVfs->xGetLastError(gMultiplex.pOrigVfs, b, c);
+  if( gMultiplex.pOrigVfs->xGetLastError ){
+    return gMultiplex.pOrigVfs->xGetLastError(gMultiplex.pOrigVfs, b, c);
+  }else{
+    return 0;
+  }
 }
 static int multiplexCurrentTimeInt64(sqlite3_vfs *a, sqlite3_int64 *b){
  return gMultiplex.pOrigVfs->xCurrentTimeInt64(gMultiplex.pOrigVfs, b);
@@ -110,20 +110,48 @@ int sqlite3Strlen30(const char *z){
 }

 /*
-** The string z[] is followed immediately by another string.  Return
-** a poiner to that other string.
+** Return the declared type of a column.  Or return zDflt if the column 
+** has no declared type.
+**
+** The column type is an extra string stored after the zero-terminator on
+** the column name if and only if the COLFLAG_HASTYPE flag is set.
 */
-const char *sqlite3StrNext(const char *z){
-  return z + strlen(z) + 1;
+char *sqlite3ColumnType(Column *pCol, char *zDflt){
+  if( (pCol->colFlags & COLFLAG_HASTYPE)==0 ) return zDflt;
+  return pCol->zName + strlen(pCol->zName) + 1;
+}
+
+/*
+** Helper function for sqlite3Error() - called rarely.  Broken out into
+** a separate routine to avoid unnecessary register saves on entry to
+** sqlite3Error().
+*/
+static SQLITE_NOINLINE void  sqlite3ErrorFinish(sqlite3 *db, int err_code){
+  if( db->pErr ) sqlite3ValueSetNull(db->pErr);
+  sqlite3SystemError(db, err_code);
 }

 /*
 ** Set the current error code to err_code and clear any prior error message.
+** Also set iSysErrno (by calling sqlite3System) if the err_code indicates
+** that would be appropriate.
 */
 void sqlite3Error(sqlite3 *db, int err_code){
  assert( db!=0 );
  db->errCode = err_code;
-  if( db->pErr ) sqlite3ValueSetNull(db->pErr);
+  if( err_code || db->pErr ) sqlite3ErrorFinish(db, err_code);
+}
+
+/*
+** Load the sqlite3.iSysErrno field if that is an appropriate thing
+** to do based on the SQLite error code in rc.
+*/
+void sqlite3SystemError(sqlite3 *db, int rc){
+  if( rc==SQLITE_IOERR_NOMEM ) return;
+  rc &= 0xff;
+  if( rc==SQLITE_CANTOPEN || rc==SQLITE_IOERR ){
+    db->iSysErrno = sqlite3OsGetLastError(db->pVfs);
+  }
 }

 /*
@@ -150,6 +178,7 @@ void sqlite3Error(sqlite3 *db, int err_code){
 void sqlite3ErrorWithMsg(sqlite3 *db, int err_code, const char *zFormat, ...){
  assert( db!=0 );
  db->errCode = err_code;
+  sqlite3SystemError(db, err_code);
  if( zFormat==0 ){
    sqlite3Error(db, err_code);
  }else if( db->pErr || (db->pErr = sqlite3ValueNew(db))!=0 ){
@@ -192,11 +192,11 @@ static VdbeCursor *allocateCursor(
  **     be freed lazily via the sqlite3_release_memory() API. This
  **     minimizes the number of malloc calls made by the system.
  **
-  ** Memory cells for cursors are allocated at the top of the address
-  ** space. Memory cell (p->nMem) corresponds to cursor 0. Space for
-  ** cursor 1 is managed by memory cell (p->nMem-1), etc.
+  ** The memory cell for cursor 0 is aMem[0]. The rest are allocated from
+  ** the top of the register space.  Cursor 1 is at Mem[p->nMem-1].
+  ** Cursor 2 is at Mem[p->nMem-2]. And so forth.
  */
-  Mem *pMem = &p->aMem[p->nMem-iCur];
+  Mem *pMem = iCur>0 ? &p->aMem[p->nMem-iCur] : p->aMem;

  int nByte;
  VdbeCursor *pCx = 0;
@@ -204,7 +204,7 @@ static VdbeCursor *allocateCursor(
      ROUND8(sizeof(VdbeCursor)) + 2*sizeof(u32)*nField + 
      (eCurType==CURTYPE_BTREE?sqlite3BtreeCursorSize():0);

-  assert( iCur<p->nCursor );
+  assert( iCur>=0 && iCur<p->nCursor );
  if( p->apCsr[iCur] ){
    sqlite3VdbeFreeCursor(p, p->apCsr[iCur]);
    p->apCsr[iCur] = 0;
@@ -529,7 +529,7 @@ static SQLITE_NOINLINE Mem *out2PrereleaseWithClear(Mem *pOut){
 static Mem *out2Prerelease(Vdbe *p, VdbeOp *pOp){
  Mem *pOut;
  assert( pOp->p2>0 );
-  assert( pOp->p2<=(p->nMem-p->nCursor) );
+  assert( pOp->p2<=(p->nMem+1 - p->nCursor) );
  pOut = &p->aMem[pOp->p2];
  memAboutToChange(p, pOut);
  if( VdbeMemDynamic(pOut) ){
@@ -667,33 +667,33 @@ int sqlite3VdbeExec(
    assert( pOp->opflags==sqlite3OpcodeProperty[pOp->opcode] );
    if( (pOp->opflags & OPFLG_IN1)!=0 ){
      assert( pOp->p1>0 );
-      assert( pOp->p1<=(p->nMem-p->nCursor) );
+      assert( pOp->p1<=(p->nMem+1 - p->nCursor) );
      assert( memIsValid(&aMem[pOp->p1]) );
      assert( sqlite3VdbeCheckMemInvariants(&aMem[pOp->p1]) );
      REGISTER_TRACE(pOp->p1, &aMem[pOp->p1]);
    }
    if( (pOp->opflags & OPFLG_IN2)!=0 ){
      assert( pOp->p2>0 );
-      assert( pOp->p2<=(p->nMem-p->nCursor) );
+      assert( pOp->p2<=(p->nMem+1 - p->nCursor) );
      assert( memIsValid(&aMem[pOp->p2]) );
      assert( sqlite3VdbeCheckMemInvariants(&aMem[pOp->p2]) );
      REGISTER_TRACE(pOp->p2, &aMem[pOp->p2]);
    }
    if( (pOp->opflags & OPFLG_IN3)!=0 ){
      assert( pOp->p3>0 );
-      assert( pOp->p3<=(p->nMem-p->nCursor) );
+      assert( pOp->p3<=(p->nMem+1 - p->nCursor) );
      assert( memIsValid(&aMem[pOp->p3]) );
      assert( sqlite3VdbeCheckMemInvariants(&aMem[pOp->p3]) );
      REGISTER_TRACE(pOp->p3, &aMem[pOp->p3]);
    }
    if( (pOp->opflags & OPFLG_OUT2)!=0 ){
      assert( pOp->p2>0 );
-      assert( pOp->p2<=(p->nMem-p->nCursor) );
+      assert( pOp->p2<=(p->nMem+1 - p->nCursor) );
      memAboutToChange(p, &aMem[pOp->p2]);
    }
    if( (pOp->opflags & OPFLG_OUT3)!=0 ){
      assert( pOp->p3>0 );
-      assert( pOp->p3<=(p->nMem-p->nCursor) );
+      assert( pOp->p3<=(p->nMem+1 - p->nCursor) );
      memAboutToChange(p, &aMem[pOp->p3]);
    }
 #endif
@@ -792,7 +792,7 @@ check_for_interrupt:
 ** and then jump to address P2.
 */
 case OP_Gosub: {            /* jump */
-  assert( pOp->p1>0 && pOp->p1<=(p->nMem-p->nCursor) );
+  assert( pOp->p1>0 && pOp->p1<=(p->nMem+1 - p->nCursor) );
  pIn1 = &aMem[pOp->p1];
  assert( VdbeMemDynamic(pIn1)==0 );
  memAboutToChange(p, pIn1);
@@ -832,7 +832,7 @@ case OP_Return: {           /* in1 */
 ** See also: EndCoroutine
 */
 case OP_InitCoroutine: {     /* jump */
-  assert( pOp->p1>0 &&  pOp->p1<=(p->nMem-p->nCursor) );
+  assert( pOp->p1>0 &&  pOp->p1<=(p->nMem+1 - p->nCursor) );
  assert( pOp->p2>=0 && pOp->p2<p->nOp );
  assert( pOp->p3>=0 && pOp->p3<p->nOp );
  pOut = &aMem[pOp->p1];
@@ -1101,7 +1101,7 @@ case OP_String: {          /* out2 */
 #ifndef SQLITE_LIKE_DOESNT_MATCH_BLOBS
  if( pOp->p5 ){
    assert( pOp->p3>0 );
-    assert( pOp->p3<=(p->nMem-p->nCursor) );
+    assert( pOp->p3<=(p->nMem+1 - p->nCursor) );
    pIn3 = &aMem[pOp->p3];
    assert( pIn3->flags & MEM_Int );
    if( pIn3->u.i ) pOut->flags = MEM_Blob|MEM_Static|MEM_Term;
@@ -1127,7 +1127,7 @@ case OP_Null: {           /* out2 */
  u16 nullFlag;
  pOut = out2Prerelease(p, pOp);
  cnt = pOp->p3-pOp->p2;
-  assert( pOp->p3<=(p->nMem-p->nCursor) );
+  assert( pOp->p3<=(p->nMem+1 - p->nCursor) );
  pOut->flags = nullFlag = pOp->p1 ? (MEM_Null|MEM_Cleared) : MEM_Null;
  while( cnt>0 ){
    pOut++;
@@ -1148,7 +1148,7 @@ case OP_Null: {           /* out2 */
 ** previously copied using OP_SCopy, the copies will continue to be valid.
 */
 case OP_SoftNull: {
-  assert( pOp->p1>0 && pOp->p1<=(p->nMem-p->nCursor) );
+  assert( pOp->p1>0 && pOp->p1<=(p->nMem+1 - p->nCursor) );
  pOut = &aMem[pOp->p1];
  pOut->flags = (pOut->flags|MEM_Null)&~MEM_Undefined;
  break;
@@ -1215,8 +1215,8 @@ case OP_Move: {
  pIn1 = &aMem[p1];
  pOut = &aMem[p2];
  do{
-    assert( pOut<=&aMem[(p->nMem-p->nCursor)] );
-    assert( pIn1<=&aMem[(p->nMem-p->nCursor)] );
+    assert( pOut<=&aMem[(p->nMem+1 - p->nCursor)] );
+    assert( pIn1<=&aMem[(p->nMem+1 - p->nCursor)] );
    assert( memIsValid(pIn1) );
    memAboutToChange(p, pOut);
    sqlite3VdbeMemMove(pOut, pIn1);
@@ -1316,7 +1316,7 @@ case OP_ResultRow: {
  int i;
  assert( p->nResColumn==pOp->p2 );
  assert( pOp->p1>0 );
-  assert( pOp->p1+pOp->p2<=(p->nMem-p->nCursor)+1 );
+  assert( pOp->p1+pOp->p2<=(p->nMem+1 - p->nCursor)+1 );

 #ifndef SQLITE_OMIT_PROGRESS_CALLBACK
  /* Run the progress counter just before returning.
@@ -1628,8 +1628,8 @@ case OP_Function0: {

  assert( pOp->p4type==P4_FUNCDEF );
  n = pOp->p5;
-  assert( pOp->p3>0 && pOp->p3<=(p->nMem-p->nCursor) );
-  assert( n==0 || (pOp->p2>0 && pOp->p2+n<=(p->nMem-p->nCursor)+1) );
+  assert( pOp->p3>0 && pOp->p3<=(p->nMem+1 - p->nCursor) );
+  assert( n==0 || (pOp->p2>0 && pOp->p2+n<=(p->nMem+1 - p->nCursor)+1) );
  assert( pOp->p3<pOp->p2 || pOp->p3>=pOp->p2+n );
  pCtx = sqlite3DbMallocRawNN(db, sizeof(*pCtx) + (n-1)*sizeof(sqlite3_value*));
  if( pCtx==0 ) goto no_mem;
@@ -2129,11 +2129,11 @@ case OP_Compare: {
  if( aPermute ){
    int k, mx = 0;
    for(k=0; k<n; k++) if( aPermute[k]>mx ) mx = aPermute[k];
-    assert( p1>0 && p1+mx<=(p->nMem-p->nCursor)+1 );
-    assert( p2>0 && p2+mx<=(p->nMem-p->nCursor)+1 );
+    assert( p1>0 && p1+mx<=(p->nMem+1 - p->nCursor)+1 );
+    assert( p2>0 && p2+mx<=(p->nMem+1 - p->nCursor)+1 );
  }else{
-    assert( p1>0 && p1+n<=(p->nMem-p->nCursor)+1 );
-    assert( p2>0 && p2+n<=(p->nMem-p->nCursor)+1 );
+    assert( p1>0 && p1+n<=(p->nMem+1 - p->nCursor)+1 );
+    assert( p2>0 && p2+n<=(p->nMem+1 - p->nCursor)+1 );
  }
 #endif /* SQLITE_DEBUG */
  for(i=0; i<n; i++){
@@ -2395,7 +2395,7 @@ case OP_Column: {
  /* If the cursor cache is stale, bring it up-to-date */
  rc = sqlite3VdbeCursorMoveto(&pC, &p2);

-  assert( pOp->p3>0 && pOp->p3<=(p->nMem-p->nCursor) );
+  assert( pOp->p3>0 && pOp->p3<=(p->nMem+1 - p->nCursor) );
  pDest = &aMem[pOp->p3];
  memAboutToChange(p, pDest);
  assert( pOp->p1>=0 && pOp->p1<p->nCursor );
@@ -2524,7 +2524,6 @@ case OP_Column: {
      }while( i<=p2 && zHdr<zEndHdr );
      pC->nHdrParsed = i;
      pC->iHdrOffset = (u32)(zHdr - zData);
-      if( pC->aRow==0 ) sqlite3VdbeMemRelease(&sMem);
  
      /* The record is corrupt if any of the following are true:
      ** (1) the bytes of the header extend past the declared header size
@@ -2534,9 +2533,12 @@ case OP_Column: {
      if( (zHdr>=zEndHdr && (zHdr>zEndHdr || offset64!=pC->payloadSize))
       || (offset64 > pC->payloadSize)
      ){
+        if( pC->aRow==0 ) sqlite3VdbeMemRelease(&sMem);
        rc = SQLITE_CORRUPT_BKPT;
        goto abort_due_to_error;
      }
+      if( pC->aRow==0 ) sqlite3VdbeMemRelease(&sMem);
+
    }else{
      t = 0;
    }
@@ -2638,7 +2640,7 @@ case OP_Affinity: {
  assert( zAffinity[pOp->p2]==0 );
  pIn1 = &aMem[pOp->p1];
  while( (cAff = *(zAffinity++))!=0 ){
-    assert( pIn1 <= &p->aMem[(p->nMem-p->nCursor)] );
+    assert( pIn1 <= &p->aMem[(p->nMem+1 - p->nCursor)] );
    assert( memIsValid(pIn1) );
    applyAffinity(pIn1, cAff, encoding);
    pIn1++;
@@ -2700,7 +2702,7 @@ case OP_MakeRecord: {
  nZero = 0;         /* Number of zero bytes at the end of the record */
  nField = pOp->p1;
  zAffinity = pOp->p4.z;
-  assert( nField>0 && pOp->p2>0 && pOp->p2+nField<=(p->nMem-p->nCursor)+1 );
+  assert( nField>0 && pOp->p2>0 && pOp->p2+nField<=(p->nMem+1 - p->nCursor)+1 );
  pData0 = &aMem[nField];
  nField = pOp->p2;
  pLast = &pData0[nField-1];
@@ -2790,7 +2792,7 @@ case OP_MakeRecord: {
  assert( i==nHdr );
  assert( j==nByte );

-  assert( pOp->p3>0 && pOp->p3<=(p->nMem-p->nCursor) );
+  assert( pOp->p3>0 && pOp->p3<=(p->nMem+1 - p->nCursor) );
  pOut->n = (int)nByte;
  pOut->flags = MEM_Blob;
  if( nZero ){
@@ -3376,7 +3378,7 @@ case OP_OpenWrite:
  }
  if( pOp->p5 & OPFLAG_P2ISREG ){
    assert( p2>0 );
-    assert( p2<=(p->nMem-p->nCursor) );
+    assert( p2<=(p->nMem+1 - p->nCursor) );
    pIn2 = &aMem[p2];
    assert( memIsValid(pIn2) );
    assert( (pIn2->flags & MEM_Int)!=0 );
@@ -4171,7 +4173,7 @@ case OP_NewRowid: {           /* out2 */
        pMem = &pFrame->aMem[pOp->p3];
      }else{
        /* Assert that P3 is a valid memory cell. */
-        assert( pOp->p3<=(p->nMem-p->nCursor) );
+        assert( pOp->p3<=(p->nMem+1 - p->nCursor) );
        pMem = &aMem[pOp->p3];
        memAboutToChange(p, pMem);
      }
@@ -4947,7 +4949,7 @@ case OP_IdxDelete: {
  UnpackedRecord r;

  assert( pOp->p3>0 );
-  assert( pOp->p2>0 && pOp->p2+pOp->p3<=(p->nMem-p->nCursor)+1 );
+  assert( pOp->p2>0 && pOp->p2+pOp->p3<=(p->nMem+1 - p->nCursor)+1 );
  assert( pOp->p1>=0 && pOp->p1<p->nCursor );
  pC = p->apCsr[pOp->p1];
  assert( pC!=0 );
@@ -5453,7 +5455,7 @@ case OP_IntegrityCk: {
  aRoot = pOp->p4.ai;
  assert( nRoot>0 );
  assert( aRoot[nRoot]==0 );
-  assert( pOp->p3>0 && pOp->p3<=(p->nMem-p->nCursor) );
+  assert( pOp->p3>0 && pOp->p3<=(p->nMem+1 - p->nCursor) );
  pnErr = &aMem[pOp->p3];
  assert( (pnErr->flags & MEM_Int)!=0 );
  assert( (pnErr->flags & (MEM_Str|MEM_Blob))==0 );
@@ -5643,6 +5645,8 @@ case OP_Program: {        /* jump */
    ** variable nMem (and later, VdbeFrame.nChildMem) to this value.
    */
    nMem = pProgram->nMem + pProgram->nCsr;
+    assert( nMem>0 );
+    if( pProgram->nCsr==0 ) nMem++;
    nByte = ROUND8(sizeof(VdbeFrame))
              + nMem * sizeof(Mem)
              + pProgram->nCsr * sizeof(VdbeCursor *)
@@ -5679,7 +5683,8 @@ case OP_Program: {        /* jump */
    }
  }else{
    pFrame = pRt->u.pFrame;
-    assert( pProgram->nMem+pProgram->nCsr==pFrame->nChildMem );
+    assert( pProgram->nMem+pProgram->nCsr==pFrame->nChildMem 
+        || (pProgram->nCsr==0 && pProgram->nMem+1==pFrame->nChildMem) );
    assert( pProgram->nCsr==pFrame->nChildCsr );
    assert( (int)(pOp - aOp)==pFrame->pc );
  }
@@ -5694,10 +5699,10 @@ case OP_Program: {        /* jump */
  p->pAuxData = 0;
  p->nChange = 0;
  p->pFrame = pFrame;
-  p->aMem = aMem = &VdbeFrameMem(pFrame)[-1];
+  p->aMem = aMem = VdbeFrameMem(pFrame);
  p->nMem = pFrame->nChildMem;
  p->nCursor = (u16)pFrame->nChildCsr;
-  p->apCsr = (VdbeCursor **)&aMem[p->nMem+1];
+  p->apCsr = (VdbeCursor **)&aMem[p->nMem];
  p->aOp = aOp = pProgram->aOp;
  p->nOp = pProgram->nOp;
  p->aOnceFlag = (u8 *)&p->apCsr[p->nCursor];
@@ -5943,8 +5948,8 @@ case OP_AggStep0: {

  assert( pOp->p4type==P4_FUNCDEF );
  n = pOp->p5;
-  assert( pOp->p3>0 && pOp->p3<=(p->nMem-p->nCursor) );
-  assert( n==0 || (pOp->p2>0 && pOp->p2+n<=(p->nMem-p->nCursor)+1) );
+  assert( pOp->p3>0 && pOp->p3<=(p->nMem+1 - p->nCursor) );
+  assert( n==0 || (pOp->p2>0 && pOp->p2+n<=(p->nMem+1 - p->nCursor)+1) );
  assert( pOp->p3<pOp->p2 || pOp->p3>=pOp->p2+n );
  pCtx = sqlite3DbMallocRawNN(db, sizeof(*pCtx) + (n-1)*sizeof(sqlite3_value*));
  if( pCtx==0 ) goto no_mem;
@@ -6023,7 +6028,7 @@ case OP_AggStep: {
 */
 case OP_AggFinal: {
  Mem *pMem;
-  assert( pOp->p1>0 && pOp->p1<=(p->nMem-p->nCursor) );
+  assert( pOp->p1>0 && pOp->p1<=(p->nMem+1 - p->nCursor) );
  pMem = &aMem[pOp->p1];
  assert( (pMem->flags & ~(MEM_Null|MEM_Agg))==0 );
  rc = sqlite3VdbeMemFinalize(pMem, pOp->p4.pFunc);
@@ -6465,7 +6470,7 @@ case OP_VColumn: {

  VdbeCursor *pCur = p->apCsr[pOp->p1];
  assert( pCur->eCurType==CURTYPE_VTAB );
-  assert( pOp->p3>0 && pOp->p3<=(p->nMem-p->nCursor) );
+  assert( pOp->p3>0 && pOp->p3<=(p->nMem+1 - p->nCursor) );
  pDest = &aMem[pOp->p3];
  memAboutToChange(p, pDest);
  if( pCur->nullRow ){
@@ -6823,6 +6828,7 @@ abort_due_to_error:
    sqlite3VdbeError(p, "%s", sqlite3ErrStr(rc));
  }
  p->rc = rc;
+  sqlite3SystemError(db, rc);
  testcase( sqlite3GlobalConfig.xLog!=0 );
  sqlite3_log(rc, "statement aborts at %d: [%s] %s", 
                   (int)(pOp - aOp), p->zSql, p->zErrMsg);
@@ -1801,7 +1801,7 @@ void sqlite3VdbeRewind(Vdbe *p){
  p->magic = VDBE_MAGIC_RUN;

 #ifdef SQLITE_DEBUG
-  for(i=1; i<p->nMem; i++){
+  for(i=0; i<p->nMem; i++){
    assert( p->aMem[i].db==p->db );
  }
 #endif
@@ -1866,16 +1866,13 @@ void sqlite3VdbeMakeReady(
  nOnce = pParse->nOnce;
  if( nOnce==0 ) nOnce = 1; /* Ensure at least one byte in p->aOnceFlag[] */
  
-  /* For each cursor required, also allocate a memory cell. Memory
-  ** cells (nMem+1-nCursor)..nMem, inclusive, will never be used by
-  ** the vdbe program. Instead they are used to allocate memory for
-  ** VdbeCursor/BtCursor structures. The blob of memory associated with 
-  ** cursor 0 is stored in memory cell nMem. Memory cell (nMem-1)
-  ** stores the blob of memory associated with cursor 1, etc.
-  **
+  /* Each cursor uses a memory cell.  The first cursor (cursor 0) can
+  ** use aMem[0] which is not otherwise used by the VDBE program.  Allocate
+  ** space at the end of aMem[] for cursors 1 and greater.
  ** See also: allocateCursor().
  */
  nMem += nCursor;
+  if( nCursor==0 && nMem>0 ) nMem++;  /* Space for aMem[0] even if not used */

  /* Figure out how much reusable memory is available at the end of the
  ** opcode array.  This extra memory will be reallocated for other elements
@@ -1937,9 +1934,8 @@ void sqlite3VdbeMakeReady(
  pParse->nzVar =  0;
  pParse->azVar = 0;
  if( p->aMem ){
-    p->aMem--;                      /* aMem[] goes from 1..nMem */
-    p->nMem = nMem;                 /*       not from 0..nMem-1 */
-    for(n=1; n<=nMem; n++){
+    p->nMem = nMem;
+    for(n=0; n<nMem; n++){
      p->aMem[n].flags = MEM_Undefined;
      p->aMem[n].db = db;
    }
@@ -2049,7 +2045,7 @@ static void closeAllCursors(Vdbe *p){
  assert( p->nFrame==0 );
  closeCursorsInFrame(p);
  if( p->aMem ){
-    releaseMemArray(&p->aMem[1], p->nMem);
+    releaseMemArray(p->aMem, p->nMem);
  }
  while( p->pDelFrame ){
    VdbeFrame *pDel = p->pDelFrame;
@@ -2074,7 +2070,7 @@ static void Cleanup(Vdbe *p){
  int i;
  if( p->apCsr ) for(i=0; i<p->nCursor; i++) assert( p->apCsr[i]==0 );
  if( p->aMem ){
-    for(i=1; i<=p->nMem; i++) assert( p->aMem[i].flags==MEM_Undefined );
+    for(i=0; i<p->nMem; i++) assert( p->aMem[i].flags==MEM_Undefined );
  }
 #endif

@@ -761,7 +761,7 @@ int sqlite3VdbeMemTooBig(Mem *p){
 void sqlite3VdbeMemAboutToChange(Vdbe *pVdbe, Mem *pMem){
  int i;
  Mem *pX;
-  for(i=1, pX=&pVdbe->aMem[1]; i<=pVdbe->nMem; i++, pX++){
+  for(i=0, pX=pVdbe->aMem; i<pVdbe->nMem; i++, pX++){
    if( pX->pScopyFrom==pMem ){
      pX->flags |= MEM_Undefined;
      pX->pScopyFrom = 0;
@@ -564,22 +564,16 @@ static int vtabCallConstructor(
      pTab->pVTable = pVTable;

      for(iCol=0; iCol<pTab->nCol; iCol++){
-        char *zType = (char*)sqlite3StrNext(pTab->aCol[iCol].zName);
+        char *zType = sqlite3ColumnType(&pTab->aCol[iCol], "");
        int nType;
        int i = 0;
-        if( !zType[0] ){
-          pTab->tabFlags |= oooHidden;
-          continue;
-        }
        nType = sqlite3Strlen30(zType);
-        if( sqlite3StrNICmp("hidden", zType, 6)||(zType[6] && zType[6]!=' ') ){
-          for(i=0; i<nType; i++){
-            if( (0==sqlite3StrNICmp(" hidden", &zType[i], 7))
-             && (zType[i+7]=='\0' || zType[i+7]==' ')
-            ){
-              i++;
-              break;
-            }
+        for(i=0; i<nType; i++){
+          if( 0==sqlite3StrNICmp("hidden", &zType[i], 6)
+           && (i==0 || zType[i-1]==' ')
+           && (zType[i+6]=='\0' || zType[i+6]==' ')
+          ){
+            break;
          }
        }
        if( i<nType ){
@@ -172,14 +172,15 @@ do_test capi3-3.3 {
  catch {
    set db2 [sqlite3_open /bogus/path/test.db {}]
  }
-  sqlite3_extended_errcode $db2
-} {SQLITE_CANTOPEN}
+  set ::capi3_errno [sqlite3_system_errno $db2]
+  list [sqlite3_extended_errcode $db2] [expr {$::capi3_errno!=0}]
+} {SQLITE_CANTOPEN 1}
 do_test capi3-3.4 {
  sqlite3_errmsg $db2
 } {unable to open database file}
 do_test capi3-3.5 {
-  sqlite3_close $db2
-} {SQLITE_OK}
+  list [sqlite3_system_errno $db2] [sqlite3_close $db2]
+} [list $::capi3_errno SQLITE_OK]
 if {[clang_sanitize_address]==0} {
  do_test capi3-3.6.1-misuse {
    sqlite3_close $db2
@@ -925,19 +926,20 @@ do_test capi3-11.9.3 {
 do_test capi3-11.10 {
  sqlite3_step $STMT
 } {SQLITE_ROW}
-ifcapable !autoreset {
-  # If SQLITE_OMIT_AUTORESET is defined, then the statement must be
-  # reset() before it can be passed to step() again.
-  do_test capi3-11.11a { sqlite3_step $STMT } {SQLITE_MISUSE}
-  do_test capi3-11.11b { sqlite3_reset $STMT } {SQLITE_ABORT}
-}
 do_test capi3-11.11 {
  sqlite3_step $STMT
 } {SQLITE_DONE}
-do_test capi3-11.12 {
-  sqlite3_step $STMT
-  sqlite3_step $STMT
-} {SQLITE_ROW}
+ifcapable !autoreset {
+  do_test capi3-11.12armor {
+    sqlite3_step $STMT
+    sqlite3_step $STMT
+  } {SQLITE_MISUSE}
+} else {
+  do_test capi3-11.12 {
+    sqlite3_step $STMT
+    sqlite3_step $STMT
+  } {SQLITE_ROW}
+}
 do_test capi3-11.13 {
  sqlite3_finalize $STMT
 } {SQLITE_OK}
@@ -865,19 +865,20 @@ do_test capi3c-11.9.3 {
 do_test capi3c-11.10 {
  sqlite3_step $STMT
 } {SQLITE_ROW}
-ifcapable !autoreset {
-  # If SQLITE_OMIT_AUTORESET is defined, then the statement must be
-  # reset() before it can be passed to step() again.
-  do_test capi3-11.11a { sqlite3_step $STMT } {SQLITE_MISUSE}
-  do_test capi3-11.11b { sqlite3_reset $STMT } {SQLITE_ABORT}
-}
 do_test capi3c-11.11 {
  sqlite3_step $STMT
 } {SQLITE_DONE}
-do_test capi3c-11.12 {
-  sqlite3_step $STMT
-  sqlite3_step $STMT
-} {SQLITE_ROW}
+ifcapable !autoreset {
+  do_test capi3c-11.12armor {
+    sqlite3_step $STMT
+    sqlite3_step $STMT
+  } {SQLITE_MISUSE}
+} else {
+  do_test capi3c-11.12 {
+    sqlite3_step $STMT
+    sqlite3_step $STMT
+  } {SQLITE_ROW}
+}
 do_test capi3c-11.13 {
  sqlite3_finalize $STMT
 } {SQLITE_OK}
@@ -423,7 +423,7 @@ do_test exclusive-5.1 {
  # (2016-03-04) The statement-journal is now opened lazily
  set sqlite_open_file_count
  expr $sqlite_open_file_count-$extrafds
-} [expr 2 - ($TEMP_STORE>=2)]
+} {2}
 do_test exclusive-5.2 {
  execsql {
    COMMIT;
@@ -450,7 +450,7 @@ do_test exclusive-5.4 {
  # 2016-03-04: The statement-journal open is deferred
  set sqlite_open_file_count
  expr $sqlite_open_file_count-$extrafds
-} [expr 2 - ($TEMP_STORE>=2)]
+} {2}
 do_test exclusive-5.5 {
  execsql {
    COMMIT;
@@ -459,7 +459,7 @@ do_test exclusive-5.5 {
  # 2016-03-04: The statement-journal open is deferred
  set sqlite_open_file_count
  expr $sqlite_open_file_count-$extrafds
-} [expr 2 - ($TEMP_STORE>=2)]
+} {2}
 do_test exclusive-5.6 {
  execsql {
    PRAGMA locking_mode = normal;
@@ -189,7 +189,7 @@ static int progressHandler(void *pVdbeLimitFlag){
 ** Reallocate memory.  Show and error and quit if unable.
 */
 static void *safe_realloc(void *pOld, int szNew){
-  void *pNew = realloc(pOld, szNew);
+  void *pNew = realloc(pOld, szNew<=0 ? 1 : szNew);
  if( pNew==0 ) fatalError("unable to realloc for %d bytes", szNew);
  return pNew;
 }
@@ -100,10 +100,12 @@ reset_highwater_marks
 build_test_db memsubsys1-2 {PRAGMA page_size=1024; PRAGMA mmap_size=0}
 #show_memstats
 set MEMORY_MANAGEMENT $sqlite_options(memorymanage)
-ifcapable !malloc_usable_size {
- do_test memsubsys1-2.3 {
-    set pg_ovfl [lindex [sqlite3_status SQLITE_STATUS_PAGECACHE_OVERFLOW 0] 2]
-  } [expr ($TEMP_STORE>1 || $MEMORY_MANAGEMENT==0)*1024]
+ifcapable pagecache_overflow_stats {
+  ifcapable !malloc_usable_size {
+    do_test memsubsys1-2.3 {
+      set pg_ovfl [lindex [sqlite3_status SQLITE_STATUS_PAGECACHE_OVERFLOW 0] 2]
+    } [expr ($TEMP_STORE>1 || $MEMORY_MANAGEMENT==0)*1024]
+  }
 }
 do_test memsubsys1-2.4 {
  set pg_used [lindex [sqlite3_status SQLITE_STATUS_PAGECACHE_USED 0] 2]
@@ -255,7 +257,7 @@ do_test memsubsys1-7.4 {
 } 0
 do_test memsubsys1-7.5 {
  set maxreq [lindex [sqlite3_status SQLITE_STATUS_MALLOC_SIZE 0] 2]
-  expr {$maxreq<4100 + 4200*[nonzero_reserved_bytes]}
+  expr {$maxreq<(4100 + 8200*[nonzero_reserved_bytes])}
 } 1
 do_test memsubsys1-7.6 {
  set s_used [lindex [sqlite3_status SQLITE_STATUS_SCRATCH_USED 0] 2]
@@ -97,7 +97,7 @@ do_test mutex1-1.9 {
 #   * Multi-threaded mode,
 #   * Single-threaded mode.
 #
-ifcapable threadsafe&&shared_cache {
+ifcapable threadsafe1&&shared_cache {
  set enable_shared_cache [sqlite3_enable_shared_cache 1]
  foreach {mode mutexes} {
    singlethread {}
@@ -183,35 +183,38 @@ db2 close
 db close
 tvfs delete

-# 2016-03-11:  Make sure all works when transitioning to WAL mode under nolock.
-#
-do_test nolock-4.1 {
-  forcedelete test.db
-  sqlite3 db file:test.db?nolock=1 -uri 1
-  db eval {
-     PRAGMA journal_mode=WAL;
-     CREATE TABLE t1(x);
-     INSERT INTO t1 VALUES('youngling');
-     SELECT * FROM t1;
-  }
-} {delete youngling}
-db close
-
-do_test nolock-4.2 {
-  forcedelete test.db
-  sqlite3 db test.db
-  db eval {
-    PRAGMA journal_mode=WAL;
-    CREATE TABLE t1(x);
-    INSERT INTO t1 VALUES('catbird');
-    SELECT * FROM t1;
-  }
-} {wal catbird}
-do_test nolock-4.3 {
+if {[permutation]!="inmemory_journal"} {
+  # 2016-03-11:  Make sure all works when transitioning to WAL mode
+  # under nolock.
+  #
+  do_test nolock-4.1 {
+    forcedelete test.db
+    sqlite3 db file:test.db?nolock=1 -uri 1
+    db eval {
+       PRAGMA journal_mode=WAL;
+       CREATE TABLE t1(x);
+       INSERT INTO t1 VALUES('youngling');
+       SELECT * FROM t1;
+    }
+  } {delete youngling}
  db close
-  sqlite3 db file:test.db?nolock=1 -uri 1
-  set rc [catch {db eval {SELECT * FROM t1}} msg]
-  lappend rc $msg
-} {1 {unable to open database file}}
+  
+  do_test nolock-4.2 {
+    forcedelete test.db
+    sqlite3 db test.db
+    db eval {
+      PRAGMA journal_mode=WAL;
+      CREATE TABLE t1(x);
+      INSERT INTO t1 VALUES('catbird');
+      SELECT * FROM t1;
+    }
+  } {wal catbird}
+  do_test nolock-4.3 {
+    db close
+    sqlite3 db file:test.db?nolock=1 -uri 1
+    set rc [catch {db eval {SELECT * FROM t1}} msg]
+    lappend rc $msg
+  } {1 {unable to open database file}}
+}

 finish_test
@@ -174,24 +174,47 @@ array set ::Configs [strip_comments {
    -O2
    -DSQLITE_ENABLE_LOCKING_STYLE=1
  }
-  "OS-X" {
+  "Apple" {
    -O1   # Avoid a compiler bug in gcc 4.2.1 build 5658
-    -DSQLITE_OMIT_LOAD_EXTENSION=1
-    -DSQLITE_DEFAULT_MEMSTATUS=0
-    -DSQLITE_THREADSAFE=2
-    -DSQLITE_OS_UNIX=1
-    -DSQLITE_ENABLE_JSON1=1
-    -DSQLITE_ENABLE_LOCKING_STYLE=1
-    -DUSE_PREAD=1
-    -DSQLITE_ENABLE_RTREE=1
+    -DHAVE_GMTIME_R=1
+    -DHAVE_ISNAN=1
+    -DHAVE_LOCALTIME_R=1
+    -DHAVE_PREAD=1
+    -DHAVE_PWRITE=1
+    -DHAVE_USLEEP=1
+    -DHAVE_USLEEP=1
+    -DHAVE_UTIME=1
+    -DSQLITE_DEFAULT_CACHE_SIZE=1000
+    -DSQLITE_DEFAULT_CKPTFULLFSYNC=1
+    -DSQLITE_DEFAULT_MEMSTATUS=1
+    -DSQLITE_DEFAULT_PAGE_SIZE=1024
+    -DSQLITE_DISABLE_PAGECACHE_OVERFLOW_STATS=1
+    -DSQLITE_ENABLE_API_ARMOR=1
+    -DSQLITE_ENABLE_AUTO_PROFILE=1
+    -DSQLITE_ENABLE_FLOCKTIMEOUT=1
    -DSQLITE_ENABLE_FTS3=1
    -DSQLITE_ENABLE_FTS3_PARENTHESIS=1
-    -DSQLITE_DEFAULT_CACHE_SIZE=1000
+    -DSQLITE_ENABLE_FTS3_TOKENIZER=1
+    if:os=="Darwin" -DSQLITE_ENABLE_LOCKING_STYLE=1
+    -DSQLITE_ENABLE_PERSIST_WAL=1
+    -DSQLITE_ENABLE_PURGEABLE_PCACHE=1
+    -DSQLITE_ENABLE_RTREE=1
+    -DSQLITE_ENABLE_SNAPSHOT=1
+    # -DSQLITE_ENABLE_SQLLOG=1
+    -DSQLITE_ENABLE_UPDATE_DELETE_LIMIT=1
    -DSQLITE_MAX_LENGTH=2147483645
    -DSQLITE_MAX_VARIABLE_NUMBER=500000
-    -DSQLITE_DEBUG=1
+    # -DSQLITE_MEMDEBUG=1
+    -DSQLITE_NO_SYNC=1
+    -DSQLITE_OMIT_AUTORESET=1
+    -DSQLITE_OMIT_LOAD_EXTENSION=1
    -DSQLITE_PREFER_PROXY_LOCKING=1
-    -DSQLITE_ENABLE_API_ARMOR=1
+    -DSQLITE_SERIES_CONSTRAINT_VERIFY=1
+    -DSQLITE_THREADSAFE=2
+    -DSQLITE_USE_URI=1
+    -DSQLITE_WRITE_WALFRAME_PREBUFFERED=1
+    -DUSE_GUARDED_FD=1
+    -DUSE_PREAD=1
    --enable-json1 --enable-fts5
  }
  "Extra-Robustness" {
@@ -248,6 +271,7 @@ array set ::Platforms [strip_comments {
    "Device-Two"              test
    "No-lookaside"            test
    "Devkit"                  test
+    "Apple"                   test
    "Sanitize"                {QUICKTEST_OMIT=func4.test,nan.test test}
    "Device-One"              fulltest
    "Default"                 "threadtest fulltest"
@@ -264,12 +288,12 @@ array set ::Platforms [strip_comments {
  Darwin-i386 {
    "Locking-Style"           "mptest test"
    "Have-Not"                test
-    "OS-X"                    "threadtest fulltest"
+    "Apple"                   "threadtest fulltest"
  }
  Darwin-x86_64 {
    "Locking-Style"           "mptest test"
    "Have-Not"                test
-    "OS-X"                    "threadtest fulltest"
+    "Apple"                   "threadtest fulltest"
  }
  "Windows NT-intel" {
    "Have-Not"                test
@@ -588,13 +612,23 @@ proc add_test_suite {listvar name testtarget config} {
  set opts ""
  set title ${name}($testtarget)
  set configOpts $::WITHTCL
+  set skip 0

  regsub -all {#[^\n]*\n} $config \n config
  foreach arg $config {
+    if {$skip} {
+      set skip 0
+      continue
+    }
    if {[regexp {^-[UD]} $arg]} {
      lappend opts $arg
    } elseif {[regexp {^[A-Z]+=} $arg]} {
      lappend testtarget $arg
+    } elseif {[regexp {^if:([a-z]+)(.*)} $arg all key tail]} {
+      # Arguments of the form 'if:os=="Linux"' will cause the subsequent
+      # argument to be skipped if the $tcl_platform(os) is not "Linux", for
+      # example...
+      set skip [expr !(\$::tcl_platform($key)$tail)]
    } elseif {[regexp {^--(enable|disable)-} $arg]} {
      if {$::MSVC} {
        if {$arg eq "--disable-amalgamation"} {
@@ -45,7 +45,7 @@ do_test shell1-1.1.1b {
 } {1 1}
 # error on extra options
 do_test shell1-1.1.2 {
-  catchcmd "test.db \"select 3\" \"select 4\"" ""
+  catchcmd "test.db \"select+3\" \"select+4\"" ""
 } {0 {3
 4}}
 # error on extra options
@@ -276,11 +276,11 @@ do_test shell1-3.2.4 {
 # .databases             List names and files of attached databases
 do_test shell1-3.3.1 {
  catchcmd "-csv test.db" ".databases"
-} "/0 +.*main +[string map {/ .} [string range [get_pwd] 0 10]].*/"
+} "/0.+main.+[string map {/ ".{1,2}"} [string range [get_pwd] 0 10]].*/"
 do_test shell1-3.3.2 {
  # extra arguments ignored
  catchcmd "test.db" ".databases BAD"
-} "/0 +.*main +[string map {/ .} [string range [get_pwd] 0 10]].*/"
+} "/0.+main.+[string map {/ ".{1,2}"} [string range [get_pwd] 0 10]].*/"

 # .dump ?TABLE? ...      Dump the database in an SQL text format
 #                          If TABLE specified, only dump tables matching
@@ -43,7 +43,7 @@ do_test shell2-1.1.1 {
 # Shell silently ignores extra parameters.
 # Ticket [f5cb008a65].
 do_test shell2-1.2.1 {
-  set rc [catch { eval exec $CLI \":memory:\" \"select 3\" \"select 4\" } msg]
+  set rc [catch { eval exec $CLI \":memory:\" \"select+3\" \"select+4\" } msg]
  list $rc $msg
 } {0 {3
 4}}
@@ -26,6 +26,18 @@ db close
 forcedelete test.db test.db-journal test.db-wal
 sqlite3 db test.db

+# There are inconsistencies in command-line argument quoting on Windows.
+# In particular, individual applications are responsible for command-line
+# parsing in Windows, not the shell.  Depending on whether the sqlite3.exe
+# program is compiled with MinGW or MSVC, the command-line parsing is
+# different.  This causes problems for the tests below.  To avoid
+# issues, these tests are disabled for windows.
+#
+if {$::tcl_platform(platform)=="windows"} {
+  finish_test
+  return
+}
+
 #----------------------------------------------------------------------------
 #   shell3-1.*: Basic tests for running SQL statments from command line.
 #
@@ -17,6 +17,15 @@ source $testdir/tester.tcl
 ifcapable !snapshot {finish_test; return}
 set testprefix snapshot

+# This test does not work with the inmemory_journal permutation. The reason
+# is that each connection opened as part of this permutation executes
+# "PRAGMA journal_mode=memory", which fails if the database is in wal mode
+# and there are one or more existing connections.
+if {[permutation]=="inmemory_journal"} {
+  finish_test
+  return
+}
+
 #-------------------------------------------------------------------------
 # Check some error conditions in snapshot_get(). It is an error if:
 #
@@ -146,7 +146,7 @@ do_faultsim_test 3.0 -prep {
    error $msg
  }
 } -test {
-  faultsim_test_result {0 {}} {1 SQLITE_IOERR} \
+  faultsim_test_result {0 {}} {1 SQLITE_IOERR} {1 SQLITE_NOMEM} \
                              {1 SQLITE_IOERR_NOMEM} {1 SQLITE_IOERR_READ}
  if {$testrc==0} {
    set res [db eval { 
@@ -14,16 +14,8 @@
 set testdir [file dirname $argv0]
 source $testdir/tester.tcl

-if {$tcl_platform(platform)=="windows"} {
-  set PROG "sqldiff.exe"
-} else {
-  set PROG "./sqldiff"
-}
-if {![file exe $PROG]} {
-  puts "sqldiff cannot run because $PROG is not available"
-  finish_test
-  return
-}
+set PROG [test_find_sqldiff]
+
 db close
 forcedelete test.db test2.db
 sqlite3 db test.db
@@ -2136,24 +2136,41 @@ proc test_restore_config_pagecache {} {
  sqlite3 db test.db
 }

+proc test_find_binary {nm} {
+  if {$::tcl_platform(platform)=="windows"} {
+    set ret "$nm.exe"
+  } else {
+    set ret $nm
+  }
+  set ret [file normalize [file join $::cmdlinearg(TESTFIXTURE_HOME) $ret]]
+  if {![file executable $ret]} {
+    finish_test
+    return ""
+  }
+  return $ret
+}
+
 # Find the name of the 'shell' executable (e.g. "sqlite3.exe") to use for
 # the tests in shell[1-5].test. If no such executable can be found, invoke
 # [finish_test ; return] in the callers context.
 #
 proc test_find_cli {} {
-  if {$::tcl_platform(platform)=="windows"} {
-    set ret "sqlite3.exe"
-  } else {
-    set ret "sqlite3"
-  }
-  set ret [file normalize [file join $::cmdlinearg(TESTFIXTURE_HOME) $ret]]
-  if {![file executable $ret]} {
-    finish_test
-    return -code return
-  }
-  return $ret
+  set prog [test_find_binary sqlite3]
+  if {$prog==""} { return -code return }
+  return $prog
 }

+# Find the name of the 'sqldiff' executable (e.g. "sqlite3.exe") to use for
+# the tests in sqldiff tests. If no such executable can be found, invoke
+# [finish_test ; return] in the callers context.
+#
+proc test_find_sqldiff {} {
+  set prog [test_find_binary sqldiff]
+  if {$prog==""} { return -code return }
+  return $prog
+}
+
+
 # If the library is compiled with the SQLITE_DEFAULT_AUTOVACUUM macro set
 # to non-zero, then set the global variable $AUTOVACUUM to 1.
 set AUTOVACUUM $sqlite_options(default_autovacuum)
@@ -1194,6 +1194,9 @@ foreach {tn sql reslist} {
  2 { PRAGMA checkpoint_fullfsync = 1 } {10 4 4 2 6 2}
  3 { PRAGMA checkpoint_fullfsync = 0 } {10 0 4 0 6 0}
 } {
+  ifcapable default_ckptfullfsync {
+    if {[string trim $sql]==""} continue
+  }
  faultsim_delete_and_reopen

  execsql {PRAGMA auto_vacuum = 0; PRAGMA synchronous = FULL;}
@@ -220,6 +220,7 @@ foreach {tn syncmode synccount} {
    sqlite3 db test.db -vfs T
  
    execsql "PRAGMA synchronous = $syncmode"
+    execsql "PRAGMA checkpoint_fullfsync = 0"
    execsql { PRAGMA journal_mode = WAL }
    execsql { CREATE TABLE filler(a,b,c); }

@@ -1244,6 +1244,7 @@ static void rbudiff_one_table(const char *zTab, FILE *out){
  Str sql = {0, 0, 0};            /* Query to find differences */
  Str insert = {0, 0, 0};         /* First part of output INSERT statement */
  sqlite3_stmt *pStmt = 0;
+  int nRow = 0;                   /* Total rows in data_xxx table */

  /* --rbu mode must use real primary keys. */
  g.bSchemaPK = 1;
@@ -1289,6 +1290,7 @@ static void rbudiff_one_table(const char *zTab, FILE *out){

    /* Output the first part of the INSERT statement */
    fprintf(out, "%s", insert.z);
+    nRow++;

    if( sqlite3_column_type(pStmt, nCol)==SQLITE_INTEGER ){
      for(i=0; i<=nCol; i++){
@@ -1342,6 +1344,12 @@ static void rbudiff_one_table(const char *zTab, FILE *out){
  }

  sqlite3_finalize(pStmt);
+  if( nRow>0 ){
+    Str cnt = {0, 0, 0};
+    strPrintf(&cnt, "INSERT INTO rbu_count VALUES('data_%q', %d);", zTab, nRow);
+    fprintf(out, "%s\n", cnt.z);
+    strFree(&cnt);
+  }

  strFree(&ct);
  strFree(&sql);
@@ -1757,8 +1765,10 @@ int main(int argc, char **argv){
  char *zTab = 0;
  FILE *out = stdout;
  void (*xDiff)(const char*,FILE*) = diff_one_table;
+#ifndef SQLITE_OMIT_LOAD_EXTENSION
  int nExt = 0;
  char **azExt = 0;
+#endif
  int useTransaction = 0;
  int neverUseTransaction = 0;

@@ -1841,8 +1851,8 @@ int main(int argc, char **argv){
      cmdlineError("error loading %s: %s", azExt[i], zErrMsg);
    }
  }
-#endif
  free(azExt);
+#endif
  zSql = sqlite3_mprintf("ATTACH %Q as aux;", zDb2);
  rc = sqlite3_exec(g.db, zSql, 0, 0, &zErrMsg);
  if( rc || zErrMsg ){
@@ -1854,7 +1864,13 @@ int main(int argc, char **argv){
  }

  if( neverUseTransaction ) useTransaction = 0;
-  if( useTransaction ) printf("BEGIN TRANSACTION;\n");
+  if( useTransaction ) fprintf(out, "BEGIN TRANSACTION;\n");
+  if( xDiff==rbudiff_one_table ){
+    fprintf(out, "CREATE TABLE IF NOT EXISTS rbu_count"
+           "(tbl TEXT PRIMARY KEY COLLATE NOCASE, cnt INTEGER) "
+           "WITHOUT ROWID;\n"
+    );
+  }
  if( zTab ){
    xDiff(zTab, out);
  }else{
Author	SHA1	Message	Date
drh	10df23fe4f	Version 3.12.0 FossilOrigin-Name: e9bb4cf40f4971974a74468ef922bdee481c988b	2016-03-29 10:14:15 +00:00
drh	72cd360caf	Fix the multiplexor so that it does not assume that the xGetLastError method is non-NULL in the child VFS. FossilOrigin-Name: f6a88cccbc0c62a0b453f4711298c9d5e1882b18	2016-03-28 11:01:54 +00:00
drh	60c4249fcc	More changes to the shellN.test scripts to get them working on all variations of Windows. FossilOrigin-Name: 8213c2f58167243411d29cc58e303b4be656f756	2016-03-26 15:36:36 +00:00
drh	eaa544d44d	Remove the catchcmd_collapse_space procedure in tester.tcl - no longer needed. FossilOrigin-Name: 3bd499d3bdf4e80f83513966c2ee9dd11e67cbd1	2016-03-26 14:41:13 +00:00
drh	15707ac992	Update shell1.test tests to deal with backslash escaping in TCL on Windows-style pathnames. FossilOrigin-Name: 219a251e182bf43376fbc8ae1ce6f77f18a1b092	2016-03-26 13:26:35 +00:00
drh	4f69540b71	Do not run the shell3.test module under mingw because of mingw's dodgy command-line parsing. FossilOrigin-Name: a7c080a90a236fbc18b9a42b78dbed4dd8a25160	2016-03-25 20:10:20 +00:00
mistachkin	82452480ea	In the Win32 VFS, avoid trying to use rand_s() on Windows CE. FossilOrigin-Name: 183350fa7cc6a5ab8bd0a86cab231a6e4c915890	2016-03-25 17:19:57 +00:00
drh	5729c31daf	Add the catchcmd_collapse_space command and use it to help make shell testing more robust against TCL space escapes. FossilOrigin-Name: 93caabb66082f76ef161a51ac822b919517a7171	2016-03-25 17:09:16 +00:00
drh	ddb2b4a310	Fix two instances of undefined behavior in the C code - both harmless for all current compilers. FossilOrigin-Name: 99fd194c83dbcfcdcc582983b86678b85b1b9570	2016-03-25 12:10:32 +00:00
drh	cb620b4797	Make test cases in shell1.test robust against TCL deciding to quote strings using lots of backslashes. FossilOrigin-Name: 817e93f42c09eb876421e27eb8eceb7b077cb45d	2016-03-24 20:55:03 +00:00
mistachkin	eb2329bed3	Prevent negative values of SQLITE_DEFAULT_CACHE_SIZE from making SQLITE_WIN32_HEAP_INIT_SIZE negative. FossilOrigin-Name: e0737f5236ed3e85bd03203c880ee41b34619137	2016-03-24 20:36:47 +00:00
drh	d38a2e2511	Larger margins for one memsubsys1 range check when running with a non-zero reserved-bytes value. FossilOrigin-Name: 6db1d1fbc6c1acc896fa635dfdc5564800502c40	2016-03-24 15:32:19 +00:00
dan	cc47eac005	Fix some test scripts so that they work with various permutations. FossilOrigin-Name: f4d234b5013bf93b6eac7f8be6d4c074cf9f5293	2016-03-24 15:09:14 +00:00
drh	c5412d533c	Avoid calling realloc() with a zero size in fuzzcheck.c. FossilOrigin-Name: a1fd14694c1adc54e5c443ebfdef38e38637f5c5	2016-03-23 17:54:19 +00:00
dan	231ee68808	Fix "ifcapable" tests in capi3.test and capi3c.test so that the tests work with more build configurations. FossilOrigin-Name: 3fa88f68c3e1bbb3421cb0d2b82b9bb3fe7b9b14	2016-03-23 16:32:29 +00:00
drh	997de998a8	Remove an unused local variable from FTS5. FossilOrigin-Name: 0ed693c29f184223cde3b3d51f0e06273e586803	2016-03-23 15:53:45 +00:00
dan	848b190e40	Explicitly limit the size of fts5 tokens to 32768 bytes. FossilOrigin-Name: 70fc69eed9b09159899d7cbd1416a59d04210a63	2016-03-23 15:04:00 +00:00
drh	3767026113	Update a requirement mark. No changes to code. FossilOrigin-Name: 412984642af40578ec611d8c0b7c0508cb5cf9c9	2016-03-23 13:46:05 +00:00
drh	d7564865ad	The sqlite3_column_decltype() routine should return NULL, not an empty string, if the column has no declared type. FossilOrigin-Name: 605eba4a756e7185119088e2242f82691d078b01	2016-03-22 20:05:09 +00:00
mistachkin	527b0435fa	Fix harmless compiler warning for MSVC. FossilOrigin-Name: 142cd359d37f1d8d53de32e329523d9a93c7d6e5	2016-03-22 15:26:03 +00:00
drh	c5e7f94494	Fix a harmless compiler warning. FossilOrigin-Name: 5ace870d3ac3e9eb29cb4602c9036873adbcb99d	2016-03-22 15:25:16 +00:00
dan	0a2f2b546a	Fix some errors in fts5 test scripts. FossilOrigin-Name: e1ab2d376a72786098125a41c1ea8140fcbd15c6	2016-03-22 15:01:54 +00:00
drh	3bfa7e82b6	Create the "uptr" typedef (the same as uintptr_t when available) and use it to cast pointers before comparison. FossilOrigin-Name: 2484cc0c3ffc8834a155f89af1581bd07d453a90	2016-03-22 14:37:59 +00:00
drh	0b98207c5b	Avoid the possibility of integer overflow on a pointer comparison test for corruption in the database file. FossilOrigin-Name: ff1b1ac3313ba9d70414e928ef3dd82913298a1a	2016-03-22 14:10:45 +00:00
drh	13969f5af0	Fix harmless compiler warnings. FossilOrigin-Name: 25d776e4523aefeec007943fe29aa17c23ccb301	2016-03-21 22:28:51 +00:00
dan	6737aab5cf	Add tests to ensure that fts5 is now deterministic. FossilOrigin-Name: 3d076094ce411bffcfeb76c0284d15d0f83bd459	2016-03-21 16:25:34 +00:00
drh	4567beefac	Skip recently added test cases in nolock.test when running in the inmemory_journal permutation. FossilOrigin-Name: a1dcf7af42bb7394d46138318f9367fe9e8be560	2016-03-21 16:19:29 +00:00
drh	6369bc3fbb	Remove an unreachable branch from the unlink verification logic in the UNIX VFS. FossilOrigin-Name: 4dc30cce7fdb7dba9a9aad96bb8b499b965610b2	2016-03-21 16:06:42 +00:00
dan	f5268268b7	Have fts5 cache the structure of its index in main memory. Use "PRAGMA data_version" to figure out when this cache should be invalidated. FossilOrigin-Name: 902f221754f3cc6fb4ae049c135f85efce604ed3	2016-03-21 15:30:50 +00:00
dan	3c6707f72b	Rearrange code so that tests pass whether SQLITE_DEBUG is defined or not. FossilOrigin-Name: 89296a46c3b891ee0b637ff6f89d828d6e46e7b5	2016-03-21 15:18:13 +00:00
drh	d260dd3534	Add the sqlite3_system_errno() interface. FossilOrigin-Name: 4bd12b57ea177cfb1f44d54bfa7aedfb8a8c0c64	2016-03-21 14:46:37 +00:00
drh	f68521c4e6	Make sure system errors that occur durign sqlite3_step() are captured for use by sqlite3_system_errno(). FossilOrigin-Name: b4a1114f730c62e93623f889bc0e4fd8d0b31efa	2016-03-21 12:28:02 +00:00
drh	80fbee092e	Improved comments. No logical changes to code. FossilOrigin-Name: a6b6c6c466f3feb257b4fc08ef6b9a27a68ca073	2016-03-21 11:57:13 +00:00
drh	8d2f41ccd2	Do a better job of capturing all system errno values regardless of when they occur. FossilOrigin-Name: 7d49998d571d841a6d1b55f5f9889e613daaab2a	2016-03-21 11:38:01 +00:00
drh	b7c9659db3	Merge updates from trunk. FossilOrigin-Name: 86ab8643969bd2e51a257d80da9316c668437f7b	2016-03-21 10:49:49 +00:00
dan	ef480d37dc	Change the way fts5 internally allocates segment ids in order to eliminated non-determinism from the module. FossilOrigin-Name: d6e2637df16764aa9723a30ea2eb8a631d28cb2b	2016-03-21 09:56:19 +00:00
drh	544c6b8e1b	Change the Vdbe.aMem array so that it is zero-based instead of one-based. FossilOrigin-Name: c39081e878faccc8552141afa5732a2bf2f77570	2016-03-21 00:38:59 +00:00
drh	3cdce92c38	Remove an unreachable branch. Improvements to comments. FossilOrigin-Name: c5677ecd5cd2637d92a831ec6bd5b002f8d75626	2016-03-21 00:30:40 +00:00
drh	9f6168b6a3	Change the Vdbe.aMem array so that it is zero-based instead of one-based. FossilOrigin-Name: e07b0c47eb5a39623f5fe0e66b939bba0906691c	2016-03-19 23:32:58 +00:00
dan	b6eb666264	Fix exclusive.test so that it works with -DSQLITE_TEMP_STORE=3. FossilOrigin-Name: d7852c639683a1d305a1e731df3cccafa64b594b	2016-03-19 18:11:59 +00:00
drh	9bccde3d03	Updates to the Lemon documentation. FossilOrigin-Name: f095341471aa822e6d556cb65512ec081c3918da	2016-03-19 18:00:44 +00:00
dan	f27d737b88	Add the sqlite3rbu_bp_progress() API to the RBU extension. Used to obtain the percentage progress of an RBU update. FossilOrigin-Name: 209e31c729b9c8a09a80e43e3e4a2f3cd9384b3a	2016-03-19 17:48:12 +00:00
dan	fce07fb944	Fix a problem detecting invalid values in the rbu_control column of an rbu database table. FossilOrigin-Name: a1132dd9027d1c6dd845be307eeb38e535393f2c	2016-03-19 17:09:30 +00:00
dan	febfe02300	Update the sqldiff tool so that it generates an rbu_count table. FossilOrigin-Name: 1f7afb6e9be9f549a91bf2ab492df15698df89fd	2016-03-19 16:21:26 +00:00
dan	f11737fcc8	Merge latest trunk changes, including fixes to test script rbudiff.test, into this branch. FossilOrigin-Name: 734fc68fb12f06e97026d4637138b82b37809f5b	2016-03-19 15:34:42 +00:00
dan	49aed58b7f	Fix another problem in test script rbudiff.test. FossilOrigin-Name: 41c29c123ff347db720ed1a541c0b2ffc04670aa	2016-03-19 15:13:59 +00:00
dan	1e8dae0e43	Fix test scripts sqldiff.test and rbudiff.test so that they work with the --testdir option. FossilOrigin-Name: 1ffe3cde03f924bb8405a8729c8e1bc01f5b6d3b	2016-03-19 14:53:36 +00:00
dan	cf9bce34ab	Add test file fts5multiclient.test. FossilOrigin-Name: 7832466f9177cca59455ba272802145297fcd53d	2016-03-19 14:47:28 +00:00
drh	bb9b5f2608	Fix a register allocation bug in the VDBE code generator for PRAGMA integrity_check; FossilOrigin-Name: 88439a866b3b16ad7c308ebe59198662a05e7eeb	2016-03-19 00:35:02 +00:00
dan	eea627da9e	Add further tests for sqlite3rbu_bp_progress(). Fix a problem in handling WITHOUT ROWID tables in the same. FossilOrigin-Name: 65e02368e2b6cec349ea71af5a456d6783b0d15e	2016-03-18 20:12:28 +00:00
dan	789780d8f6	Change the name of the new API on this branch to "sqlite3_bp_progress". Add tests and documentation for the same. FossilOrigin-Name: 1a1b69e87eb7d18f76f5b733e44da75136a686b6	2016-03-18 18:56:45 +00:00
dan	fe485c0e56	Add tests for the changes on this branch. Fix a problem with calls to the new progress indicator API made after an rbu update has been resumed. FossilOrigin-Name: bf82321724d3b0feb51e26d9b76090e03cc3964a	2016-03-18 10:29:47 +00:00
drh	860443da10	Fix FTS5 so that it works with SQLITE_OMIT_AUTORESET. FossilOrigin-Name: b199637d81d7e2a767131ac03c7679b101fd459c	2016-03-18 00:39:40 +00:00
drh	b99185f278	Fix more test-case errors in the Apple configuration. FossilOrigin-Name: 6631e1e655604a7d1fb45b4d151938d4a13b47f3	2016-03-18 00:19:48 +00:00
drh	108e5a9a6c	Fix some test cases so that they work under the Apple configuration. FossilOrigin-Name: 399c60764d3fdec49dab1745c30930ca1dbafe70	2016-03-17 23:56:23 +00:00
drh	9493cafe6f	Fix harmless compiler warnings that arise with -DSQLITE_OMIT_LOAD_EXTENSION FossilOrigin-Name: 65ba2f0b465a1493de6e467f55e0300ac3fb08e1	2016-03-17 23:16:37 +00:00
drh	790b151e19	Changes to releasetest.tcl: Rename the "OS-X" configuration to "Apple". Bring the "Apple" configuration up to date. Allow the use of "if:os=..." arguments in a configuration. Run the Apple configuration on Linux in addition to on Macs, but without -DSQLITE_ENABLE_LOCKING_STYLE. FossilOrigin-Name: beb2a80b430e0ad561a58d8f00b80cbdc74b5e03	2016-03-17 23:00:42 +00:00
dan	e42195b044	Add an API to indicate the percentage progress of an rbu update. FossilOrigin-Name: ffc58d2c2576a5b6e1c2c7112612c5760e711afd	2016-03-17 21:06:42 +00:00
drh	762e32bcfa	Change the xGetLastError() method on the windows VFS so that it continues to format an error message (if requested) but returns the system error code, rather than an SQLite status code. FossilOrigin-Name: 1602f6b53698bd3a1a4be218c2e3145dd895f1f1	2016-03-17 19:28:19 +00:00
drh	de845c2f5a	A more compact implementation of the code generator for the IS and IS NOT operators. FossilOrigin-Name: 8607e3ac7a9d44372a4a66da21bbb3d28ae2528a	2016-03-17 19:07:52 +00:00
drh	0e80e50984	Simple test cases for sqlite3_system_errno() FossilOrigin-Name: 185bc8644c806597194dc532a4d547a03a2dedaa	2016-03-17 17:23:11 +00:00
drh	1b9f2141a7	Experimental implementation of the sqlite3_system_errno() interface. FossilOrigin-Name: 6782c87b3722fbd09684a5b1e5df05247956f1c6	2016-03-17 16:01:23 +00:00
dan	c9460ab989	Fix some OOM-handling issues in the fts5 changes on this branch. FossilOrigin-Name: 020a0bda59fe93b9361ceeed0d5a8ba4177380c8	2016-03-17 12:39:56 +00:00
dan	db8d320d63	Merge the changes on the reusable-pragma branch into this one. FossilOrigin-Name: 6c4a17b963916d19c9433d7a251338042f32e9fa	2016-03-16 20:53:33 +00:00
dan	f55fb6615b	Have fts5 cache the decoded structure of fts5 indexes in memory. Use "PRAGMA data_version" to detect stale caches. FossilOrigin-Name: 33ef2210ef19e55c8d460bfe9d3dc146034c8acc	2016-03-16 19:48:10 +00:00