Commit Graph

334 Commits

Author SHA1 Message Date
dan 4cd3a59a79 Improve support for sqlite3_stmt_scanstatus_v2() profiling of sub-queries implemented as co-routines.
FossilOrigin-Name: 4e8718dc35dbbaf75f17265a88d14acd9750dc75efbadf41377f9c97e732009c
2023-06-30 18:23:53 +00:00
larrybr bc91738e66 Add a C-source spell-checking facility. make misspell (on Nix)
FossilOrigin-Name: 26c1bb4bd9e9f56613c3aa87407a7f562fd4ebde5bfd6dece02078001d9a45f8
2023-06-07 08:40:31 +00:00
drh 653882ed8c Simplification, performance improvement, and size reduction in the
codeAllEqualityTerms() function of the code generator.

FossilOrigin-Name: 4fbe02651e877e442d62308d7f607e503695104cd71c5565132fcd1398c120df
2023-05-17 16:13:48 +00:00
drh c1e40a3a02 Fix a bug in cursor hints that can cause references to tables that have not
been opened.  Cursor hints are intended for use by COMDB2 only and should not
appear in production builds, so this should not be a factor for the vast
majority of users.

FossilOrigin-Name: d3370d59cffb7ab982d6c620c93d22aa6a9dc786e1c4af95ca8d45ff0b9b7d6f
2023-05-04 11:29:15 +00:00
drh e4d8e7e5df Better handling of OOM errors in the cursor-hint logic.
dbsqlfuzz 60cd5fff91974af91c2c3692beb4a2d7fdafef46

FossilOrigin-Name: 68fcfb58df837b40e401dfa4e505f4f0b87554b9cfeb2c527f3558348b73c1be
2023-04-11 02:10:34 +00:00
drh ed36917835 New assert() statements to validate the parameters to
sqlite3BtreeCursorHint().  Fix a problem with the construction of those
parameters discovered by
[forum:/forumpost/0b53708c95|forum post 0b53708c95].

FossilOrigin-Name: 4c5a3c5fb351cc1c2ce16c33314ce19c53531f09263f87456283d9d756002f9d
2023-04-10 18:44:00 +00:00
drh 09db37c022 Omit a branch that is no longer needed following [c9c4f287652933eb].
FossilOrigin-Name: 960a488a2db92f5437bc8171cdbed44618d9381c57fd41b2f0e357b6012c80b5
2023-03-30 16:08:54 +00:00
dan e1afa2bd24 Avoid having OP_SeekScan jump over an OP_IdxGT or OP_IdxGE that follows the OP_SeekGE opcode. Fix for [b50528af4468237c].
FossilOrigin-Name: c9c4f287652933eb6262a3419efe7e7288f55d3db7e2ac032eeae025f665d306
2023-03-29 21:58:06 +00:00
dan 45163fc45e Add an sqlite3_db_config() option - SQLITE_DBCONFIG_STMT_SCANSTATS - for enabling and disabling the collection of sqlite3_stmt_scanstats() statistics in SQLITE_ENABLE_STMT_SCANSTATUS builds. Collection of statistics is disabled by default.
FossilOrigin-Name: 0f5579bef27b84ee855065cfe87703c51e1f9773906a9e0d4e4dafc90bd0e553
2023-02-28 19:39:59 +00:00
dan 0c69938511 Allow vector-IN expressions like "(a, b) IN ( (?,?), (?,?) )" to use an index.
FossilOrigin-Name: 1815b15ddb8785a25b7617aab19e13c2410b9377389c16a85176025b3d9400e8
2023-02-13 16:10:31 +00:00
dan 2adb309ead Have sqlite3_stmt_scanstatus_v2() return an NCYCLE value for all loops, not just virtual tables ones. The value returned is the sum of the NCYCLE counts for the various opcodes that move or read data from the table or index cursor associated with the loop.
FossilOrigin-Name: 9499b2f51e8174c6b8a67840c92ba23b7dd1dc8dc2b91fca0c5dc07b71662149
2022-12-06 18:48:06 +00:00
dan ad23a47acd Enhance SQLITE_SCANSTAT_NCYCLE so that it reports on virtual tables.
FossilOrigin-Name: 622d8eb3724bee617b55d6fb71f1a2d683db6858065adced6bf3ce9525bcd6b5
2022-12-03 21:24:26 +00:00
dan 231ff4b027 Enhance the sqlite3_stmt_scanstatus() API and add sqlite3_stmt_scanstatus_v2(). For creation of easier to read query performance reports.
FossilOrigin-Name: 55800833645739efeddcacef464c623931cb6aeb43f4219b4e4faf473c25c8bb
2022-12-02 20:32:22 +00:00
drh 2a757658a0 Attempt to rationalize the bits associated with ".wheretrace". Provide
a decoder key in sqliteInt.h for what each bit is intended to do.

FossilOrigin-Name: 8ec361695a107a94f2cf6a7fe509656a99d85d49bd7c74133c69903f059a2675
2022-11-30 19:11:31 +00:00
drh f55a7dad19 This branch attempts to improve the detection of covering indexes. This
first check-in merely improves a parameter name to sqlite3WhereBegin() to
be more descriptive of what it contains, and ensures that a subroutine is
not inlines so that sqlite3WhereBegin() runs slightly faster.

FossilOrigin-Name: cadf5f6bb1ce0492ef858ada476288e8057afd3609caa18b09c818d3845d7244
2022-10-22 14:16:02 +00:00
drh 63b3a64ca6 The generalized indexed expression optimization of [2435112867fbd7b6] makes
the prior [a47efb7c8520a011] enhancement from 2017 obsolete.  This check-in
removes the older optimization.

FossilOrigin-Name: 56442c9bdd7844454c79acf83323380ea3509cb2e0b9a3d538a6b51726ae6d07
2022-10-20 13:36:32 +00:00
drh 135c9ff6f7 Remove a use of alloca() that does more harm than good.
FossilOrigin-Name: 2ab3cf4d71b23a4c0b69f977828d82df7bbc080c29ae189e0d78d38efee50c96
2022-10-17 09:56:51 +00:00
dan 73c586bcbf Fix a problem causing the seek-scan optimization to skip over valid rows that could occur when it is used with expressions of the form (a IN (?,?..) AND b >= ?). dbsqlfuzz ab1db6dc0efb04cba1cd3431ee6da4894fdc4520.
FossilOrigin-Name: 63d9efe277759d4daa29794846b60c6f55491496618f423f61468df72d0a4633
2022-10-07 18:57:15 +00:00
dan 73ac958c74 Fix a problem that occurred when using a bloom filter to optimize an (ipk = ?) lookup in the case where the RHS of the expression is a TEXT value. First reported by [forum:/forumpost/f61a8b7053|forum post f61a8b7053].
FossilOrigin-Name: 8e14c351b29bb434ac4e2510681e95716424a73e340d39feff4919f0431c2e00
2022-10-06 14:10:11 +00:00
drh 41ce47c4f4 Add the new internal interface sqlite3DbNNFreeNN(db,ptr) where both the
db and ptr parameters are guaranteed to be non-NULL.  Use this where
appropriate to save more than 2 million CPU cycles on the standard
performance test.

FossilOrigin-Name: e5eaa80e81fdf86f2875a912b880272b8d099b82b08e945a7988c5dd0fe9d6b5
2022-08-22 02:00:26 +00:00
drh 3a17e2e5cf Improvement on the previous check-in.
FossilOrigin-Name: c1ab0ea2959259cbd29f624bfd5e7366035f2dbcaab72c0bf55723c0919b7a79
2022-08-03 17:26:32 +00:00
drh b5a6f33e50 For an IN operator used with a RIGHT JOIN, use the number of terms in the
vector, not the number of equality terms, to size the column map.
dbsqlfuzz 14cfdad6ca45e607163f54049ddf5065183dc657.

FossilOrigin-Name: b184c8d9222da6b420b5d7c439bbe4b737d92ffa27f5e933f4e1a9c8117b0ee5
2022-08-03 15:58:44 +00:00
drh 058e99502a Small performance increase and size reduction by splitting out the
sqlite3VdbeGetLastOp() from sqlite3VdbeGetOp().

FossilOrigin-Name: 92ac01d41d46ab73e189b1e5596ea63e5edb5b15639c5d7bdb981b95366c069b
2022-07-25 19:05:24 +00:00
drh 0f4b534b60 Candidate fix for the problem with row-value WHERE clause constraints on
a RIGHT JOIN.

FossilOrigin-Name: fdd782a7bb217e2aceda97a588c0d7874ee19885abfcf0658472d6c3387da6ee
2022-06-01 13:32:47 +00:00
drh c9dfe2b808 Add back in a test case for ON-clauses that was removed yesterday by
[3869fd9a2b9483cbf] but which turned out to be necessary after all.

FossilOrigin-Name: aba52a90eef2a5d06e7c499fe84b2efcde7f01e67129058f8223a6a0272aacb9
2022-05-31 11:13:55 +00:00
drh 404bf6bac6 For an outer join, then ON-clause constraints need to be evaluated at just
the right moment - not too early and not too late.  Fix for the problem
reported by [forum:/forumpost/3902c7b833|forum post 3902c7b833].

FossilOrigin-Name: 3869fd9a2b9483cbbf83d8f369c1744abc42f12e63abba402be35dd7e136161c
2022-05-30 17:33:22 +00:00
drh c93bf1d462 Improvements to the decision of when to check ON constraints for an inner
join that is an operand to a RIGHT JOIN.  Fix for issue identify by
[forum:/forumpost/c06b10ad7e|forum post c06b10ad7e].

FossilOrigin-Name: 9d17233c7d98bf25c1a518d067e778708b3db6d6302edd8d7e376ba0ba4f1c30
2022-05-14 15:59:42 +00:00
drh 556527a154 Transitive equality constriants do not work on a RIGHT JOIN, since the
right-hand side might be a non-matched row.

FossilOrigin-Name: 0f96810b840dd6f209562635b21f55a7ed6210c01336fcfeb3b79e08a615a28d
2022-05-13 20:11:32 +00:00
drh f69dad8c53 Walk back the optimization from check-in [cc458317bd77046c] that tries to
reuse the same ephemeral cursor of a list subquery when that subquery is
reused, as it does not work in cases where the list subquery is used both
for lookups and for scans.

FossilOrigin-Name: 12ee29d632ae4b585ef6bc07d3289d00c121268945dffd5673b251d95874e3f8
2022-05-13 19:50:29 +00:00
drh 767bc8de8e Defer generating WHERE clause constraints for a RIGHT JOIN until after the
ON-clause processing for the RIGHT JOIN has done its own row elimination.
This fixes and incorrect output from some RIGHT JOINs that was identified
by [forum:/forumpost/41cc3851d864c5e6|forum post 41cc3851d864c5e6].

FossilOrigin-Name: 238d9c247cf69cc77fdb1af9d42ebe258610a533ac4204e2ddf8af17f24d18c4
2022-05-13 17:45:52 +00:00
drh 67a99dbee8 Improved names for flags on the Expr object: EP_FromJoin becames
EP_OuterON and EP_InnerJoin becomes EP_InnerON.

FossilOrigin-Name: 1ffea07ff98b894729c698b681cc7433df3bbfccd8a0529a706908602a636937
2022-05-13 14:52:04 +00:00
drh d631c6af80 Ensure that ON clauses are applied to the correct outer join.
FossilOrigin-Name: c7e3a13a3288c577209be99c630fbe924e19880e8af1aa8a83b517acaa8b43d7
2022-05-12 11:56:44 +00:00
drh 95cb5ccf92 Code EP_InnerJoin constraints at the same time that EP_FromJoin constraints
are coded.

FossilOrigin-Name: b57033d2af2196e2f7e5305f3504fc20da5cadeb66026d38b999fe909cab8ac5
2022-05-04 17:30:51 +00:00
drh 8aa7f4d813 Fix the Bloom filter pull-down optimization so that it jumps to the correct
place if it encounters a NULL key.  Fix for the bug described by
[forum:/forumpost/2482b32700384a0f|forum thread 2482b32700384a0f].

FossilOrigin-Name: 6eda9b1a7784cf6d58c8876551f67ab98e78a08e726a0579d4def5ba881985bb
2022-05-03 14:01:48 +00:00
drh 56f38db700 Remove a testcase() macro taht was made obsolete by [a8da85c57e07721d].
FossilOrigin-Name: 053cf45ecaa7eac8cc64c7b2672807f736a45575a84b426263b2e1feb060b401
2022-05-02 15:47:57 +00:00
drh 3a45d30ea5 Improvement on check-in [a193749730d6cfba] so that the subroutine call to
the IN operator right-hand side generator from the RIGHT JOIN no-match logic
does not generate unreachable byte code.

FossilOrigin-Name: cc458317bd77046c4328715ae9e3409f3f4cd422a01162cb33405ef3a142b0a3
2022-05-02 14:32:56 +00:00
drh 7497364749 Simplified fix to the problem with subroutine reuse in the RIGHT JOIN no-match
logic for a subquery on the right-hand side of an IN operator.  The code still
needs simplification.

FossilOrigin-Name: a193749730d6cfba617f2a64e7254c78f92bbf29b109c19055fc33f6cc643d4c
2022-04-30 18:40:22 +00:00
drh 2aa1086cab Preserve the COLLATE operator on an index on an expression when resolving
the use of that expression into a reference to the index.  See
[forum:/info/7efabf4b03328e57|forum thread 7efabf4b03328e57] for details.

FossilOrigin-Name: a8da85c57e07721dc1c918d67433d6c99ce48421e369123cc3194d855e55f7e8
2022-04-30 12:35:51 +00:00
drh a25bbaf74a Further improvements to codeEqualityTerm() for cases when an IN operator with
a right-hand side subquery is used as a constraint that needs to be processed
by the RIGHT JOIN non-matched logic.

FossilOrigin-Name: bb2798be3fb5737827eec505db2d4c8cb510d227d98fc26ce748bf2da7e8b382
2022-04-30 00:06:52 +00:00
drh 3d5665366f Enhance the codeEqualityTerm() routine inside the code generator so that it is
able to reuse an IN operator that has an invariant subquery on its right-hand
side while coding the non-matched loop of a RIGHT JOIN.
dbsqlfuzz 19f1102a70cf966ab249de56d944fc20dbebcfcf

FossilOrigin-Name: 56dd3065469e49320b13af039471b3f0b85e6f7368cfd97b1cdc0cf8fa8e1956
2022-04-29 19:00:11 +00:00
drh b77c312956 For debug builds, if the RIGHT JOIN body subroutine contains a jump that
escapes the subroutine, then abort the prepared statement with a descriptive
error and SQLITE_INTERNAL.  This extra sanity check causes many tests to
fail.

FossilOrigin-Name: 2c5bb2bff26cc70d8cac78ddd12d5ac37ab1472f5f88afbd975950a18ac2804d
2022-04-23 18:04:31 +00:00
drh 503ad9c72d Make sure the code generator knows to invoke row-value subroutines which
running the right-join post-processing loop.

FossilOrigin-Name: fd328e52aee1dace12b1c2f44b6f7e9d15d8f77c8e9d9e3d85840a129a4b1808
2022-04-21 14:48:40 +00:00
drh 7348ca4e0b Ensure that left-hand side WITHOUT ROWID tables are set to OP_NullRow prior
to running the right-join loop.  This fixes a problem introduced by the
previous check-in.

FossilOrigin-Name: aab7665ce6c004df701a72aef1a5135f2c78f16c6ff728a00076afb66ba9d3a0
2022-04-18 16:20:59 +00:00
drh 2e1bcc9dd0 Fix problems with WHERE clauses that follow multiple FULL JOINs. This
introduces some new faults in the tests, probably due to something
unrelated.

FossilOrigin-Name: 95b242d4c2fed7c713299565ac1010f8a7534a5695589e87d5a0204c7bf5f3dc
2022-04-18 16:10:07 +00:00
drh d875c7eefe Do not allow an empty table bypass to jump outside of a right-join body
subroutine.

FossilOrigin-Name: 1549dcd2353903b70abadd428eeef971ab940df04fb05a6b83b04ee30932db6d
2022-04-18 10:26:50 +00:00
drh 2c31c00bfa Rerun the subroutines that compute row-values if necessary from within
the RIGHT JOIN body subroutine.

FossilOrigin-Name: 9b9038bcd0ab5c4f01661456635526cef764f854ff24018a5e6e43825d07eb59
2022-04-14 16:34:07 +00:00
drh f7ecd956ce Move the explain comment for the RIGHT-JOIN post-processing loop to the
verify beginning of the loop - to make the start of the loop clearer to
human bytecode readers.

FossilOrigin-Name: 7ed2a271e6fcbb5e69a7f3a88d3f45fe6318819c0cc6a0dcc06c3dae5aa1503f
2022-04-13 16:46:35 +00:00
drh 146e64d2e4 Add missing VdbeCoverage() macros on new branch byte-code opcodes.
FossilOrigin-Name: 218c7167e562f5c327124f02a92de85079315320a221fb0508310d927596b14c
2022-04-13 01:52:32 +00:00
drh 949e2ab49a Factor out the RIGHT JOIN non-matched row loop from sqlite3WhereEnd(). This
reduces the register pressure on that routine and helps it to run faster
in the common case where there is no RIGHT JOIN.

FossilOrigin-Name: beeecf1604d4fb11e45058f48cb2289c6542e0bc218d63a245198113d8d5476b
2022-04-12 18:04:29 +00:00
drh c583719b65 Show LEFT and RIGHT JOIN processing in the EXPLAIN QUERY PLAN output.
FossilOrigin-Name: d91faeffea5cf0585fb71e5311fdcc6b8be85c7e9c732050b4448e617c970101
2022-04-11 14:26:37 +00:00