Commit Graph

230 Commits

Author SHA1 Message Date
dan 240e36c0e3 Avoid coding unindexed WHERE constraints if an equivalent transitive term has already been coded by an outer loop.
FossilOrigin-Name: 644bb77f8bb2c3499db7c280440c92fd2649a98ce714ae0d5d690eabfdc04326
2021-04-05 16:20:59 +00:00
drh 2f2091b10e Use the canonical name, not the AS name, when showing the MATERIALIZE or
CO-ROUTINE lines for a CTE.

FossilOrigin-Name: 1b83e232c490fe6e9f999d30399faebfac6094bf940dc21dc459ed2c97657cb2
2021-03-20 15:46:01 +00:00
drh a979993b25 Improved labeling of EXPLAIN QUERY PLAN output. Many test failures due to
the different output format.

FossilOrigin-Name: 6f8faec0222a7ca07cc1a5ed16cc08d92c6e20bbcb34851b4eff9e624de53601
2021-03-19 13:00:28 +00:00
drh 7601294ad3 Rename the "struct SrcList_item" object to the more succinct "SrcItem".
This is a symbolic change only.  The logic is unmodified.

FossilOrigin-Name: bfd5bf2c73110fcb36db9ba2a949ff516131fbd3e89325f88fe9f5c2b4ed87b2
2021-02-21 21:04:54 +00:00
dan df1b52e727 Ensure a cursor used by the SeekScan operator does not point to a valid row on the first iteration of the loop. Possible fix for [2d6e8400].
FossilOrigin-Name: 390cf60a286b13f454429f4652a133f95a7891a75c1ec6d16cd39990590fd3fb
2021-01-27 17:15:06 +00:00
drh 7d14ffe4e1 Additional corner-case fixes for the in-scan-vs-index or OP_SeekScan
optimization.

FossilOrigin-Name: ad5ab24ebd557e7af1d92ab3fbcb3747c04da5ad4ed779fb6391dc94042687dd
2020-10-02 13:48:57 +00:00
drh 04e70ce07d For the OP_SeekScan optimization, the OP_IdxGT does not necessarily come
right after the OP_SeekGE.   So use the P2 operand of OP_SeekScan to point
to the first instruction after OP_IdxGT.  Problem found by dbsqlfuzz.

FossilOrigin-Name: 091d71939411bf2079098f09ddc0525f11f380119fd27114e2b3eec9a8ac0db4
2020-10-02 11:55:07 +00:00
drh 4ad1a1c74c Disable the OP_SeekScan opcode of the in-scan-vs-index optimization when
in PRAGMA reverse_unordered_selects mode, as the OP_SeekScan only works
with forwards scans.  Thanks to OSSFuzz for pointing out the problem to
us.

FossilOrigin-Name: c75c3a3b756635bfdab44e4b56a337e4a88af3a8803cd3e9a67abf3d0d3450dc
2020-10-02 02:02:18 +00:00
drh 644f43c0f7 Remove a debugging printf() accidentally left in the previous check-in.
FossilOrigin-Name: e9d1efa597fb073dc4bc4286e03059a350ed05d3e6476cd3247e6cde626f458b
2020-10-01 14:38:39 +00:00
drh 67306cb3b4 The IN-early-out optimization does not work for virtual tables. Do
not try to use it there.  Fix for ticket [2d5a3163563d559f].  Test
cases in TH3.

FossilOrigin-Name: dbec6910d9648f4907373c818a5cb347a2e487784208751ccecfacca752ff99d
2020-10-01 14:36:15 +00:00
drh fad4dd0fd1 Improved query optimization for multi-column indexes where the second or
later columns are constrained by an IN operator and the earlier index columns
limit the search to a small number of rows.  Use the new OP_SeekScan opcode
which does scanning of the relevant range of the index but gives up and
falls back to doing a seek if the number of rows scanned grows to large,
in order to guard against pathological cases where the estimated number
of rows to be scanned is far too small.

FossilOrigin-Name: 4a43430fd23f88352c33b29c4c105b72f6dc821f94bf362040c41a1648c402e5
2020-09-30 18:06:51 +00:00
drh 4f65b3bbfb For the OP_SeekScan opcode, adjust the number of steps run before giving
up based on the estimated number of comparisons needed to perform a seek.

FossilOrigin-Name: dc4172e6b8e1f62dc7329a3adb2223f290bc4c8055c265e88182ef432f4bcf10
2020-09-30 18:03:22 +00:00
drh 395a60dad4 Back out the incorrect optimization of check-in [e51ecadcbdef5ce6] as
ticket [5c4e7aa793943803] reports a case where the optimization does not
work.

FossilOrigin-Name: 7395e96b8cc370c8ac2657fb805915b0992a15d80f8bf256d277b423fec64675
2020-09-30 17:32:22 +00:00
drh 0b2949c36e Coverage testing of the OP_SeekScan opcode. Fix a problem that comes up when
OP_SeekScan reaches the end of the table.

FossilOrigin-Name: 9e57e758a6a33f54d28a546b4eebfb5cfacef30dc4e0207e43bb9d2c06fc3439
2020-09-29 20:22:19 +00:00
drh 46f0f4e56d Typo fix to repair the early-out optimization. Also avoid unnecessary
OP_SeekHit opcodes.

FossilOrigin-Name: 8fd7d8dfcd515aa6b65d6eb27b033d3b3a31db467b9100cc13c62bc60113019e
2020-09-29 15:32:54 +00:00
drh f761d937c2 The OP_SeekScan opcode works, but using it requires disabling the
IN-earlyout optimization because the OP_IfNoHope opcode might move the
cursor.

FossilOrigin-Name: f3c36b840c9a29c0add28039db216f4207a308e5057fc76e3f0004024a8267ac
2020-09-29 01:48:46 +00:00
drh 68cf0ace3d Revisiting the IN-scan optimization to try to fix it for the corner case
where the statistics deceive the query planner into using a scan when
an indexed lookup would be better.  This check-in changes the code
generation to do the IN-scan using a new OP_SeekScan opcode.  That new
opcode is designed to abandon the scan and fall back to a seek if
it doesn't find a match quickly enough.  For this work-in-progress check-in, 
OP_SeekScan is still a no-op and OP_SeekGE still ends up doing all the work.

FossilOrigin-Name: d720b6981eeb0ffdb14494ca63eca298ee724ae4ad4863c7c7cbfdad7fa52519
2020-09-28 19:51:54 +00:00
drh 75fa266341 Small performance improvement and size reduction by reducing the size of
the WhereTerm object.

FossilOrigin-Name: 43f7ddad800acf40917c5cc3d926640dbec17c34d5f1cbbb74bd80f44eeed0a5
2020-09-28 15:49:43 +00:00
drh bc2e95140b Prevent use-after-free of the u.vtab.idxStr string following an OOM
while generating the OP_VFilter opcode.

FossilOrigin-Name: 751fe4edb2d4602e652523c2759de3f4fffd29d5c66cae68caf45b30fd8b750a
2020-09-17 11:32:14 +00:00
drh fa17e134b2 Improvements to the IN-early-out optimization so that it works more
efficiently when there are two or more indexed IN clauses on a single table.

FossilOrigin-Name: 35505c68c1945c35babd2496e02bc4907a15c8e7b8d77f05f230bd0e9d4891d7
2020-09-01 01:52:03 +00:00
drh 398221e2ad Omit the unnecessary not-NULL check on the upper bound of a forward index
range scan.  The subsequent OP_IdxGT will always catch the NULL.  Similarly,
omit the not-NULL check on the lower bound of a reverse index range scan,
as the SeekLE opcode will disallow the NULL.

FossilOrigin-Name: e51ecadcbdef5ce6e41d68412aee70aa4a1416c850358a6fa7e9a1ab443e719d
2020-08-15 19:58:59 +00:00
drh 68c0c71065 For UPDATE and DELETE, use OP_DeferredSeek always. If the seek must later
be resolved, add the OP_FinishSeek opcode after all WHERE clause terms have
been processed.  This obviates the need for the WHERE_SEEK_TABLE and
WHERE_SEEK_UNIQ_TABLE flags to sqlite3WhereBegin() and the ensuing
complication, and it allows the covering index optimization to be used
further into WHERE clause processing.

FossilOrigin-Name: a495f60d315e34b1a1bc5fb1336e05047add52c8fb2710b577c97b10a5e734f6
2020-08-14 20:04:26 +00:00
drh abc3815860 Continuing work toward supporting unsigned 32-bit page numbers.
FossilOrigin-Name: 9ce1710aad43cebe5ad50859c7685fb83e40cdd4a60913bd2b7e659bc59942fd
2020-07-22 13:38:04 +00:00
drh 7f05d52c4c Ensure that the NULL-scan pass counter is initialized when a ORDER BY NULLS LAST
is used on the right table of a LEFT JOIN.  Ticket [e12a0ae526bb51c7].

FossilOrigin-Name: 704bb9a39acbee420c1d6ac9eb1466a02dd77d3334b938bfddf235973129b5fe
2020-03-02 01:16:33 +00:00
dan d03f77ae46 Fix a problem with the processing of IN(...) constraints handled by virtual table implementations that do not set the "omit" flag when the virtual table column contains at least one NULL value.
FossilOrigin-Name: dcb4838757ca49cf149a6e883b3eb0ac8a075147387a078280dfabe39b1a3e8d
2020-01-29 15:03:01 +00:00
drh 3b8eb08b17 Better than removing the incorrect assert() is to change it into a
testcase() together with a comment indicating where an appropriate test case
can be found, and the ticket that provoked the change.
Ticket [614b25314c766238]

FossilOrigin-Name: 3e3c60d8124672ff72a57dfa2e51276316bcac8c8ae8a5739da358f26924b0fb
2020-01-12 22:38:17 +00:00
drh ca7d3e1d42 Remove an incorrect assert(). Fix for ticket [614b25314c766238]
FossilOrigin-Name: 6a999c5aa0d1d5569a5bda50d27e3a15c13e93e69d00db97c5294901522064a3
2020-01-12 22:20:07 +00:00
dan 74ebaadcdd Fix a problem where the loop for the RHS of a LEFT JOIN uses values from an IN() clause as the second or subsequent field of an index.
FossilOrigin-Name: 95ef68966c50f311830cba8c9257a4085c93011d205e0e31867c2917fa62a48e
2020-01-04 16:55:57 +00:00
drh 36e678bc39 The sqlite3WhereEnd() call now unwinds all Expr modifications made by the
sqlite3WhereBegin().

FossilOrigin-Name: 7bfd42f1dc0c94f9bb74516e62fec8e39c20f4749f4e0972f66c2886d3c64f73
2020-01-02 00:45:38 +00:00
drh be3da24134 Add the OP_FinishSeek opcode which completes an OP_DeferredSeek if the seek
has not already completed.  Also add the sqlite3WhereUsesDeferredSeek()
interface to the query planner.  The UPDATE implementation adds an
OP_FinishSeek before running the final OP_Insert if one is needed.
Ticket [ec8abb025e78f40c] and also an assertion fault reported by Yongheng.

FossilOrigin-Name: 21ef6e99331210b80fa7c71b4f02e8f768a748d01aef884368af2f6b51a067e0
2019-12-29 00:52:41 +00:00
dan 4adb1d0000 Change an assert() in where.c to a testcase() macro, since the condition may be false. This was a problem with the assert() only, there is no bug in release builds that omit assert().
FossilOrigin-Name: 82be135dee7ccfde5f8a67f3621b7ced449dce89bae9cadf025154a4de848c11
2019-12-28 18:08:39 +00:00
drh a4b2df5ce2 Disable early coding of transitive constraints at the end of each loop in
the WHERE clause processing if the loop being coded is for a LEFT JOIN,
even if the loop is part of an OR-clause optimization for virtual tables.
Test cases in TH3.

FossilOrigin-Name: 9421b442cad9858ec21050c106aa935a1c0723ab08ef9a867a79638b040b6e68
2019-12-28 16:20:23 +00:00
drh f1bb31e219 Further improvements to .wheretrace during loop code generation.
FossilOrigin-Name: c4d5b75c9381255ec8d9a284eedb6b27be46ca868cae5985cf8a5769b15290c3
2019-12-28 14:33:26 +00:00
drh 118efd1626 New enhancements to .wheretrace. The 0x20000 flag shows the WHERE clause
before and after coding each loop.  The 0x800 flag shows status at the start
and at the end of each loop.  An extra "C" tag is shown on coded terms.

FossilOrigin-Name: 59cc46e5a6d8dbb030f27716ad5446ecccf81cf0cfff95338b9133777f2059e7
2019-12-28 14:07:22 +00:00
drh e86f3402ac Disables the optimization that tries to pull the value of an expression
from an index on that expression if the expression is a constant.

FossilOrigin-Name: e5fd8b50500f9225e435ef5afee98e3c2cccd22785f99c718d7e6c9b5b653c56
2019-12-26 00:20:56 +00:00
dan 2811ea6be7 For expressions like (x, y) IN (SELECT ...) where the SELECT uses window-functions, require that all columns on the LHS be indexed before an index can be used. Fix for [d9ed4ebe].
FossilOrigin-Name: 0b1dbd60f5db3abe2097dbc0b6de9671685ca5eaf7d3fc8e3f87ff5065a9d114
2019-12-23 14:20:46 +00:00
drh 6c1c85ca48 Fix to the optimization of check-in [a47efb7c8520a011] that reads the values
of expressions used in an index-on-expression directly from the index rather
than recomputing the value.  If the expression has a top-level COLLATE or
unlikely() operator, be sure to clear the corresponding flags prior to
converting it into a TK_COLUMN expression.  Failure to do this is most likely
harmless in production, but might cause an assertion fault in debugging builds.
Ticket [b0cb8aff561a6dcd].  Test cases in TH3.

FossilOrigin-Name: 56539e1c132632c075efc217ad5951a35e4459605ac128914aec3a7be1e25718
2019-12-22 18:55:04 +00:00
dan 69843342de Ensure sqlite3WindowRewrite() is called on a SELECT statement before any terms aremoved from it as part of IN() clause processing. Fix for [f00d096ca].
FossilOrigin-Name: 8c856404b4e98d295449a4e89a41495dc007319a8e9c35c1a763718d7c5f67e8
2019-12-22 17:32:25 +00:00
drh ed0c34857a Do not try to access a generated column through an index if the collating
sequence for the generated column is non-standard.
Part 2 of ticket [e0a8120553f4b082]

FossilOrigin-Name: 056bb8dcbdc45989c5c6e86d2966200062e3c01c382ec52aae37c828104b4496
2019-12-20 22:46:41 +00:00
drh 7a9698456d Fix a gramfuzz find. If a partial index that does not reference any column
of its table is used by an UPDATE statement in one-pass mode, then avoid the
use of OP_DeferredSeek since the seek might not be resolved prior to the
OP_Delete and OP_Insert that implement the UPDATE.

FossilOrigin-Name: e3398c5ffb060b2b26334b8598e2c63953741e2d6f5124dbd6bdfc8e94742539
2019-12-09 14:34:37 +00:00
drh 57f7ece784 Fix a problem that comes up when using generated columns that evaluate to a
constant in an index and then making use of that index in a join.

FossilOrigin-Name: 8b12e95fec7ce6e0de82a04ca3dfcf1a8e62e233b7382aa28a8a9be6e862b1af
2019-11-21 18:28:44 +00:00
mistachkin bde3a4f680 Minor simplification to the changes from check-in [36c11ad51f].
FossilOrigin-Name: 7bc8205dd9c1657c736a9c6a1a90dd9dad442accfbb77d296eaae2c09ab46bd1
2019-11-06 19:25:45 +00:00
drh 4fb24c82cd Fix the OP_DeferredSeek index-to-table column map in P4 so that it works
with generated columns.  Ticket [ce22a07731530118]

FossilOrigin-Name: 36c11ad51fe9ab1bde0b98d0ea9b8588e07d168cd8027486749372894941ad93
2019-11-06 17:31:18 +00:00
drh 06fc24555f Changes an unreachable testcase() into an assert().
FossilOrigin-Name: 5710845b6314f924b490434b4446169f4bb50576aaa940472af68150db16e127
2019-11-04 12:49:15 +00:00
drh db535390db The optimization of check-in [9b2879629c34fc0a] is incorrectly reasoned.
The WHERE clause of the partial index might not be true if the table of
the partial index is the right table of a left join.  So disable the
optimization in that case.  Ticket [623eff57e76d45f6]

FossilOrigin-Name: 3be19e1151af1850b65991edb82420f9412a7798dd756c86eaa9ffdde573263a
2019-11-03 00:07:41 +00:00
drh 4485ac1a25 The previous fix was incomplete. It is also necessary to disable the
Expr.y.pTab field when making the translation.

FossilOrigin-Name: b99d5701312f7472e6b606bd824f9273617e2655920485bf50aa96d408064721
2019-10-24 21:02:06 +00:00
drh c7476735c9 Fix handling of covering indexes that use virtual columns.
FossilOrigin-Name: e0f7e321eca91c49102649f70728de69347cbd7b16599b854203c24cfdc348e6
2019-10-24 20:29:25 +00:00
drh b9bcf7ca60 Refactor names of column index transformation functions, for clarity.
Get generated columns working with ALTER TABLE RENAME COLUMN.

FossilOrigin-Name: 27ab41c9102e7801ff829488fc123a8040da008bef373d6704efbe2f93e1da90
2019-10-19 13:29:10 +00:00
drh 6df9c4b990 Claw back some performance from the sqlite3ExprGetColumnOfTable() routine.
FossilOrigin-Name: e8426acb94179ff49549aced6ea3c26c49ba4761c2f414fa1772d6a031edc79d
2019-10-18 12:52:08 +00:00
drh 81f7b37270 Initial experimental code for generated column support. Non-functional.
FossilOrigin-Name: 11d472c1df707b8d03ec57d8fc582a34f5eb89a9d02a154a9871650c65065b45
2019-10-16 12:18:59 +00:00