Commit Graph

1341 Commits

Author SHA1 Message Date
drh 473dc7347e New testcase() macros on virtual table xBestIndex logic.
FossilOrigin-Name: 3bbd9c5f2f1990f7d187951474a86996650dcede12c2608484d43250ae0b30d6
2018-09-10 14:43:15 +00:00
dan eb3fe0b9d4 Fix a problem with processing a "vtab.col IS NULL" expression within
the WHERE clause of a query when "vtab" is a virtual table on the rhs of a
LEFT JOIN.

FossilOrigin-Name: 83da4d4104ee1870a2a95bb5fa15ee6584c655d8b314b6b8ab97592dad4ee811
2018-09-10 12:17:16 +00:00
drh f7ded14792 Fix an unreachable branch in the new sqlite3WhereOrderByLimitOptLabel()
function of the query planner.

FossilOrigin-Name: 5a954533edbde58aa7158572ece7ceeb1c6e610b71f3ae45d0b8371d74f9fea5
2018-09-08 20:29:04 +00:00
drh 6ee5a7b481 Fix multiple issues with the ORDER BY LIMIT optimization. This is the
proposed resolution to ticket [9936b2fa443fec03ff25].

FossilOrigin-Name: 206720129ed2fa8875a286266d05b99fb2caf8671e4b74b26a6286a2073fcd8b
2018-09-08 20:09:46 +00:00
drh 02ceed058a Completely remove the column cache logic, which has been a persistent source
of bugs for many years.  We recent enhancements to the performance of the
OP_Column opcode, the removing the column cache actually makes speed-check.sh
run faster.  It also saves about 1,800 bytes of code space.

FossilOrigin-Name: 3f5f60cd7529330209027fdae8129cca420cec1050eae50a7750d0b715b56972
2018-08-03 23:04:16 +00:00
drh f559ed3400 Add a small cost penalty to sorting to bias the query planner in favor of
plans that do not require a final sorting pass.

FossilOrigin-Name: 85b9beb4605eb0cfe2ed063c2a1925186c9e37031f78c875e60a347cce891638
2018-07-28 21:01:55 +00:00
drh 7810ab6464 Only run the constant propagation optimization on joins since that is the
only scenario where it is useful.  This saves prepare time for the common
case of a simple query.

FossilOrigin-Name: 598d608359005b13fa1151c6c69f2fd12ef3ab7f563a035fc4a6f3c569dc6c67
2018-07-27 17:51:20 +00:00
drh 72d03a646c Change the SQLITE_Stat34 bit of the optimization test-control so that it
prevents STAT4 data from being used but allows it to be loaded into the
Index objects.  This permits STAT4 to be turned on and off on a per-statement
basis.

FossilOrigin-Name: 489f3caf5dcc8cace20a0da58ceb32fdea5770ba570fa3c9f712cd35e726846a
2018-07-20 19:24:02 +00:00
dan e882551935 Add a test-control to disable the skip-scan optimization.
FossilOrigin-Name: 650a3fe03d61068e06f3097878a777ca8ed713a769444cdb3f8be7d1e19a0b83
2018-07-12 19:14:39 +00:00
drh fef37760da Identify Select objects within a single statement using small sequential
integers rather than "0x%p".  This is more readable and yields the same result
on successive runs.

FossilOrigin-Name: a7cdc5bc85e0edfcc38f920c2ce91599bcbfdb49522d88b08c64596546d13881
2018-07-10 19:48:35 +00:00
drh 299bf7c2f0 Add the OP_SetTabCol and OP_VerifyTabCol opcodes, only when compiling with
SQLITE_DEBUG, to do run-time verification of the column cache.

FossilOrigin-Name: b37614a3670770919a7c7f8157c5fd6578d736447998640b52e5cef8174cadc2
2018-06-11 17:35:02 +00:00
drh a46ba6cca5 Improved comments an presentation for the recent IN operator decision
improvement.

FossilOrigin-Name: 31e480f68dfd887cfd9114f9f9fec53d751d3ecc27e36f55c0166b51b2fbb08c
2018-06-09 14:13:46 +00:00
drh 1fe3ac7347 Compute the bitmask of indexed columns for each index once when the Index
objecct is constructed, instead of recomputing it every time it is needed.

FossilOrigin-Name: d735872ec383bbd220b08c61d25db9ff3675d2542b9e7867e7d6323a12e0cc23
2018-06-09 01:12:08 +00:00
drh da230bd484 Avoid invoking the whereLoopAddOr() routine in the query planner if there
are no OR operators in the WHERE clause, thus speeding up query planning
slightly.

FossilOrigin-Name: 292724ffc4bfca435fff309383d488ffdbe1e314e5eb26da21cf2f621b64bce5
2018-06-09 00:09:58 +00:00
drh 6d6decb8d9 Only choose to scan an IN operator rather than use an index if we have
real STAT1 data to suggest it is advantageous.

FossilOrigin-Name: 30e874661dcc1a2ecb40df2ef74582151d85bb36c754a38548829a3b6285f18d
2018-06-08 21:21:01 +00:00
drh da4c409aea Consider doing a partial table scan to fulfill an IN operator rather
than using an index.  Try to pick the plan with the lowest cost.

FossilOrigin-Name: 1fa40a78fef4516c39b217bff67efe7e7d2077cca00aae0ef5c2c9cff94f008b
2018-06-08 18:22:10 +00:00
drh f1949b6634 Remove the NextIfOpen and PrevIfOpen opcodes which are no longer needed
when the IN-early-out optimization is working.

FossilOrigin-Name: 439c8162272795b422a0e01b01b832fbc12b39914c9632a674162af8bdecff98
2018-06-07 17:32:59 +00:00
drh f7b0a5f3c3 Add the WHERE_IN_EARLYOUT flag and use it to clarify the logic of this
optimization.

FossilOrigin-Name: 522f1eacc20f11002cad58232a7c2610f369568653510e54f46088f579f778dc
2018-06-07 14:59:22 +00:00
drh 8c2b6d784b Add the OP_IfNoHope and OP_SeekHit opcodes used to reduce the number of
unnecessary sqlite3BtreeMovetoUnpacked() calls when checking for an early
exit on IN-operator loops.  Futher optimizations are likely possible here.

FossilOrigin-Name: 6bf251af4347165a470d39457d61ab6d2a06c206db8f30bd8be5dbb388ae8a5b
2018-06-05 20:45:20 +00:00
drh 86d0ea7558 Use an OP_NotFound opcode to cancel futile IN operators early. The current
implementation is suboptimal because it always runs teh OP_NotFound.  This
still needs to be enhanced to only do the OP_NotFound if no results have been
seen on the current loop.

FossilOrigin-Name: 87a9fc504f9a78caf7a7949cc7ada0a19d61bfab51bb49a00a1607194c116212
2018-06-05 15:16:25 +00:00
drh a0368d9396 Demonstration code on a possible technique for optimizing the use of IN
operator on columns to the right of multicolumn indexes.  If the OP_Noop
generated where were really a new opcode that checked to see if there existed
any entries in the index with a matching prefix, it might prevent unnecessary
iterations of the IN operator.

FossilOrigin-Name: 92f0fe155d5546fc7f4a443d0630613dabe149061966308e5420fad652278f16
2018-05-30 00:54:23 +00:00
drh 492ad131bd Make more aggressive use of automatic indexes when processing materalized
views and subqueries.

FossilOrigin-Name: 172f5bd27e47cbdaaab54fe4383a5ee505d285257af6153ed626d9493a4adab3
2018-05-14 22:46:11 +00:00
drh e2188f0b0f Fix harmless compiler warnings associated with the new EXPLAIN QUERY PLAN logic.
FossilOrigin-Name: 374d8e264487b0437a8d995ced1bc026a92d495a2d0568f65f033e9ebe11d0e2
2018-05-07 11:37:34 +00:00
drh fa16f5d9d6 Enhance EXPLAIN QUERY PLAN to report the generation of constant rows using
VALUES or just a SELECT without FROM.

FossilOrigin-Name: c75eee69fa8a9b56ee58a4cc539e80cc982f43390dc3a357344d58479dd89a41
2018-05-03 01:37:13 +00:00
drh b3f0276b9e Fix test cases so that they work with the new EXPLAIN QUERY PLAN output
format.  Only some of the cases have been fixed.  This is an incremental
check-in.

FossilOrigin-Name: 5f0e803e33aa557865d5fc830d9202d628de9a94c9757058ca48f1a560702cd3
2018-05-02 18:00:17 +00:00
drh 84a01debf9 Improved EQP output for recursive CTEs and multi-value VALUES clauses.
FossilOrigin-Name: f2f525548c65f89f55cbe91da8a21512dedc6f7b68b58b7906d653e800a2963a
2018-05-02 16:13:48 +00:00
drh c447595df5 Do not attempt to use terms from the WHERE clause to drive indexes on the
right table of a LEFT JOIN.  Fix for ticket [4ba5abf65c5b0f9a96a7a40cd18b]

FossilOrigin-Name: aeb694e3f787f1f8b55650c17f90c197eee3f7f9b890a88f458c33e43009a082
2018-04-24 17:34:03 +00:00
drh 337679be4e Enhance the query planner so that it detects when the xBestIndex method
of a virtual table gives out-of-sequence argvIndex values and reports an
error.  Secondary fix for ticket [2b8aed9f7c9e6].

FossilOrigin-Name: 9506ec14fb9e58986c1b79a3ca78430ad94b10966944c864e0429a7688dd1454
2018-04-09 15:57:54 +00:00
drh 0f1631db06 Improvements to ".wheretrace" output.
FossilOrigin-Name: ea2e5b34bf35ade602dfcef359b64f810674f304fda14326b7f2dedefcc5f340
2018-04-09 13:58:20 +00:00
dan 4ea48144f9 Fix a failing assert() in the new code on this branch.
FossilOrigin-Name: 74d857d178dfadea7e07ba47439fe11aa9d282b54caf78cd6961e593b37406d0
2018-01-31 14:07:01 +00:00
dan 8433e7166b Update the omit-left-join optimization so that it works in some cases when the
RHS is subject to a UNIQUE but not NOT NULL constraint.

FossilOrigin-Name: 88411a402b29f19d69e0d06d9099eabb98c238ddb7c07ce110236316f008753e
2018-01-29 17:08:52 +00:00
drh 33f10207aa Proposed fix for the query planner problem of ticket
[ec32177c99ccac2b1].

FossilOrigin-Name: eef8cbef3c49e6331301a8f3f8a5fd12090b1e99e68487c2d405e53ef771f688
2018-01-27 05:40:10 +00:00
dan 0c2ba13e02 Fix a problem causing an infinite loop or other malfunction in some UPDATE
statements with an OR term in the WHERE clause.

FossilOrigin-Name: feb2c2b6f66b0f45490beb1642d99cdb89fa220e299a8c118929df557c814189
2018-01-16 13:37:43 +00:00
drh 092457b18c Change the function name to sqlite_unsupported_offset(X). Only enable the
function if compiled with -DSQLITE_ENABLE_OFFSET_SQL_FUNC.  The makefiles add
that definition to shell builds.

FossilOrigin-Name: 7a7f826e324b1a2c332e2f1d0740fd0babffcaca6275a798572f02ad367b99ab
2017-12-29 15:04:49 +00:00
drh fe6d20e9f4 Enhance location(X) so that it works with indexes and WITHOUT ROWID tables.
The function might return an offset to the main table or to an index,
depending on whether the column X would be loaded from the main table or
from the index.

FossilOrigin-Name: dd94d6a880dfec4bddd247239b815b84964f804d24841e25f33f1d46a4b5274d
2017-12-29 14:33:54 +00:00
drh efc88d020c Modify the new sqlite3_vtab_collation() interface so that it takes a
pointer to the sqlite3_index_info object passed into xBestIndex rather than
an sqlite3 connection pointer, which the xBestIndex method might not have
access to.

FossilOrigin-Name: 5c1fe6666019147a26480b5db1bf2f474a5d072c234c736f16ed5d2a9a040b3f
2017-12-22 00:52:50 +00:00
dan e42e1bc594 Fix crash in sqlite3_vtab_collation() when called for an IS NOT NULL
constraint.

FossilOrigin-Name: ad38d2c4f073705c02c7b38675e8ae86fe4a794d54eb796e7ed51a905824d5f5
2017-12-19 18:56:28 +00:00
dan 11546779b7 Merge latest trunk changes into this branch.
FossilOrigin-Name: d5b597b52a1213cdf382d96f4df3535727be0852b25bafd12bbef54da946c5f2
2017-12-16 19:16:24 +00:00
drh 7e8515d8be The query planner tries to avoids using indexes that use unknown collating
functions.

FossilOrigin-Name: 02013fc120bf71a8be3550c696a588af8c92f2209f8e5db530624878ddc8aa7e
2017-12-08 19:37:04 +00:00
drh b7571b4ecb When doing a table scan using an index, do not error out if collating
functions used by that index are unavailable, since they will not be used.

FossilOrigin-Name: bbd69fa6fa7f3c86fb5cd1b69e1abbe1bbad61aa281e6c073a402d1b202e42ec
2017-12-07 22:10:16 +00:00
dan 3d9c7c3a4f Merge latest trunk changes into this branch.
FossilOrigin-Name: 373fa21bffd37053bb1a741d3ace2f003a5639bf373c20ea7bb24a31080888b3
2017-11-29 16:16:29 +00:00
drh 53bf7175af Fix a problem in the omit-table-from-left-join optimization
from check-in [0cd82ee9a8413cf] that was discovered by OSSFuzz.

FossilOrigin-Name: b016c28fa5617a20ad34c005372e738d28f7fc4388d19ee0cb7add4ed19d74aa
2017-11-23 04:45:35 +00:00
drh fa337cc115 Completely disable the skip-ahead-distinct optimization for all but the
inner-most loop of a DISTINCT join.  Fix for ticket [ef9318757b152e3a26e9592]

FossilOrigin-Name: b7595cf2cadcba486e60b3c230ccc412a7cf449c1d56cbe65869e6d5d9f1374d
2017-11-23 00:45:21 +00:00
drh 065b34f6e5 Fix the skip-ahead-distinct optimization on joins for cases there the table
in the inner loop of the join does not contribute any columns to the
result set.  Proposed fix for ticket [ef9318757b152e3a2]

FossilOrigin-Name: 2dcef5a9ae7f347da65207bf6bf612fb12e18e1a6704799322f0cf2a86154cfd
2017-11-21 23:38:48 +00:00
dan 41203c6cf1 Update the omit-table-from-left-join optimization so that it can omit tables
from the middle of the join as well as the end.

FossilOrigin-Name: 618ca9fe53d8d2d7b4f368e6ee404d5fceeecac0d689f32ab62af8a6cbb37401
2017-11-21 19:22:45 +00:00
dan 75dbf68b5b Fix a problem preventing the planner from identifying scans that visit at most
one row in cases where that property is guaranteed by a unique, not-null,
non-IPK column that is the leftmost in its table.

FossilOrigin-Name: 299d7ca52fec32f04ffd2b8561dd4b839e891792162f8b00259368683436b02d
2017-11-20 14:40:03 +00:00
dan 85e1f46eb9 Fix a problem causing an INDEXED BY specifying an unusable partial index to be
mishandled.

FossilOrigin-Name: 292a04086a902634fc514b379a2b245eb2681c1b84d9bb950b6ecb9aab28b468
2017-11-07 18:20:15 +00:00
drh 9d9c41e25e Remove unnecessary NEVER() and ALWAYS() conditionals.
FossilOrigin-Name: 1c80c75d4be2f3d44fb18bb4c07eccac2aba79b688215a741317dfc47dc7c2ce
2017-10-31 03:40:15 +00:00
drh 989d7278b0 Improved comments, including a typo fix, on the whereLoopCheaperProperSubset()
function.

FossilOrigin-Name: 700a3c694438ca6cca185d0097f24799e82717ef38cb47bd83666c80f0e3cb2f
2017-10-16 11:50:12 +00:00
drh 47b1d68f25 In the query planner, do not consider index X to be a proper subset of
index Y if X is a covering index but Y is not.

FossilOrigin-Name: ee31c04353cd75ea4bbadee2994c30d3808b696a4f680187502d104902988a5d
2017-10-15 22:16:25 +00:00