Commit Graph

24578 Commits

Author SHA1 Message Date
dan aee4049128 Update things to use sqlite3_recover_step() internally.
FossilOrigin-Name: f4b15aad3005237b7ac507eed2b9b07e0f5c9407ab28f2656a21c9845f13d35f
2022-09-24 18:05:52 +00:00
dan 9bbd91b39d Internal changes to the recover extension towards a step() style interface.
FossilOrigin-Name: 73926d5c8cd1ecece134b5a73b44ee1dfca74dc200606e3f009b06cdecf8cee9
2022-09-23 21:01:10 +00:00
dan 80b1f6fcde Fix various compiler warnings in new code on this branch.
FossilOrigin-Name: ae49e9efde3012158061def6e0a8d993abbc5933514a21f84bc10f700f61b5e2
2022-09-23 11:40:43 +00:00
dan bdbd097280 Merge latest trunk changes into this branch.
FossilOrigin-Name: e87fa70ab0f9b95bbcde18567f47906a222a3fd02b4f3c2903d2bb087d361b7c
2022-09-23 11:30:24 +00:00
drh f333370e1f Reduced and improved comments on the constraintCompatibleWithOuterJoin()
function.

FossilOrigin-Name: 38aaf26e082bd95df6b64df43e1772fe6e20c4eb71307dcd97559cac7f11f8f1
2022-09-21 18:21:31 +00:00
drh 2897b88f9e The RIGHT JOIN logic added to version 3.39.0 added a new restriction on the
query flattener which resulted in a performance regression for some queries,
as documented by [forum:/forumpost/96b9e5709cf47cda|forum post 96b9e5709cf47cda].
This change removes that restriction (thus restoring performance) and fixes the
problem that RIGHT JOIN was having in a different way.

FossilOrigin-Name: 501609eddf2a46d51e8cd56477a22d4df142a166e78fe4322c0db11d1eed6687
2022-09-21 09:54:41 +00:00
drh faaacd3fbb Break out the decision of whether or not a constraint term is usable by
an outer join into a subroutine: constraintCompatibleWithOuterJoin().

FossilOrigin-Name: c7fbc9b0453050e2746af27e3a11e0c3701bef8c56da8e19173242c6ea3aff8b
2022-09-21 00:16:59 +00:00
drh 0f51aa3972 Do not allow EP_InnerON terms to be used on a outer join.
FossilOrigin-Name: f47aa745690c018800243bf76930b2499ff4537411c4e27b4b16ba3854cc2bf8
2022-09-20 22:19:13 +00:00
drh 0cc6f78bcf Add new test cases that (current) get an incorrect answer.
FossilOrigin-Name: fe5c37736444e397ace387b761ec2491229785d7a437f3f60232aedf9d1f00ec
2022-09-20 21:07:52 +00:00
drh 4c5c941b6e New test cases for query flattening when there are ON clauses and outer joins.
FossilOrigin-Name: db55387b1972bc7bc6855bd3497af744a111e7d28d09893cd985fbb01be41bcc
2022-09-20 19:45:04 +00:00
drh 8b9a3d1fc8 Modify the OP_IfNotOpen opcode so that the jump is taken if the cursor is open
but was previously set to a NULL row using OP_NullRow.

FossilOrigin-Name: 1292d68caa7086610ddda343f3852e63de8da1eb66536ee4716b6529f5a31bc6
2022-09-20 19:22:17 +00:00
drh 5a0771a1f4 Remove code that has been previously commented out.
FossilOrigin-Name: 852b385a5de622aa32026824210d4bd23db52a4a8a697b83d22d6000edeba487
2022-09-20 18:18:00 +00:00
drh a5f8c42c98 Merge the fuzzcheck fix from trunk into the flattener improvement branch.
FossilOrigin-Name: 930a8c1561d0d5dc70dc5378edac77f79ac59ab6569b561de3dfb02f160c38c5
2022-09-20 17:43:40 +00:00
drh 703c208d30 Update the fuzz invariant checker to conform to the latest changes in
dbsqlfuzz.

FossilOrigin-Name: 8d8c124349d2a25200ace9a60d76751594b722468ad0c39d927a5d2c6e2464e0
2022-09-20 17:21:54 +00:00
drh 40c9becc81 Is the query flattener restriction 29 (do not allow both EP_InnerON and
EP_OuterON constraints on the same FROM clause term) really necessary?   This
branch explores what happens if we omit that restriction.  This is an effort
to address the performance regression reported by
[forum:/forumpost/96b9e5709cf47cda|forum post 96b9e5709cf47cda] that is caused
by flattener restriction 29.

FossilOrigin-Name: 19270577ad1d08c9e6e5f0600396b8a1ccda9df1e67a0df845aee6b60437f379
2022-09-20 16:57:49 +00:00
drh b22b493338 Clarify comments that describe query flattener restrictions associated with
outer joins.  No code changes.

FossilOrigin-Name: 2647d73bb1876d01851f52cb3fd6af08a962ab336b6d51c160d27b1baf94f75f
2022-09-19 19:25:15 +00:00
drh 316cd40122 Fix a minor comment typo. No changes to code or documentation.
FossilOrigin-Name: 8032db4ade66f70a102abb9d3a49e20620e204c45ad7a954a49d20403985bf07
2022-09-19 17:41:59 +00:00
drh bcdb4cc71b Remove a NEVER macro in defragmentPage() that dbsqlfuzz discovered can be
true.  crash-32d9312f145cdce41613573f6431d9a3e439e3d7

FossilOrigin-Name: 90403759e1cf30f7c11fa8170d6df59a0dd138e1de59d2b60bec7ba15d74f67c
2022-09-18 17:59:28 +00:00
dan 73390f09c2 Fix windows builds of recover extension.
FossilOrigin-Name: abcbb6abfe08fc590123f0aa1bd72cfdecc75908f078f1348dc0e957de98bf52
2022-09-15 03:54:52 +00:00
dan 6fa0a11c7b Include recovery extension in configure/make builds of testfixture.
FossilOrigin-Name: d2f4652144c4a54012a2176c178c5ade3463bd017515f472cfa8dafdde192df3
2022-09-14 20:45:46 +00:00
dan 3c24342841 Extra tests and fixes.
FossilOrigin-Name: deed5336931b23bb507c064d08e9899b33f04e4f7eee03beb8d7147eb4caa030
2022-09-14 18:57:46 +00:00
dan b8eaf9a10d Fix recovery of utf-16 databases.
FossilOrigin-Name: 5b05be0861f35804270fbd184ad4b89c23e98cc2fbd56b9e4fe6197daef5fe49
2022-09-14 16:37:59 +00:00
dan 322967df59 Add OOM tests for the recovery extension.
FossilOrigin-Name: 9b6b4c7162439a889144edb561356afc66436db921a867c20871f0c556716502
2022-09-13 20:40:57 +00:00
dan a3a6d63b40 Fix a broken error handling case in the recovery extension.
FossilOrigin-Name: 60089547e1fc77ecc02f207ebf75ee3160e5ff25f41d12e02e170fd7fde66602
2022-09-13 19:47:01 +00:00
dan 7985e05a97 Add tests. Deal issues surrounding aborting recovery from within the SQL callback, and avoiding the pending-byte page.
FossilOrigin-Name: 4e97dd31f5240d9231167ae172a5116426c42177a1ed3c5422b9d51b762d5a87
2022-09-13 18:08:24 +00:00
dan 5ca0b38605 Fix some problems with handling corrupt database in the recovery extension.
FossilOrigin-Name: ed318be8241981ef96334ba13d3201a717cc812a59aed64e3dc67f7e7e71854b
2022-09-12 20:02:33 +00:00
dan aede9b5bd3 More tests for the recover module.
FossilOrigin-Name: 37fb093b95c6b7d7ad07a275697df73b69f9fb5c5549aea8544b26e38f24833f
2022-09-12 19:23:50 +00:00
drh 3e4e697d1c New run-time options to the speed-check.sh script.
FossilOrigin-Name: ff46bc9d2a844dde9dafa157114f4d9ee157205ca5c08ed9ee32f018de310e62
2022-09-12 16:07:24 +00:00
dan 65660916dc Rework recover extension code for readability.
FossilOrigin-Name: 1a2540960e40e3c8c622448fd3862e249bd463c29ae4ce5e39942e942533f60a
2022-09-10 20:01:49 +00:00
dan bc2e7fc228 Further fixes to comments in sqlite3recover.h. Also rework some data structures in sqlite3recover.c.
FossilOrigin-Name: 599d1f8ec2f9e24924a6f9e66c85664360c7b95531b07a4efe1dd8c096b3fc99
2022-09-09 20:44:56 +00:00
mistachkin bdb6368d72 Fix harmless compiler warning seen with MSVC.
FossilOrigin-Name: 73c4c68d3b4c16caf8281dabddd7365d24691e5ee2da89b1783eca66bfbd9280
2022-09-09 17:50:29 +00:00
dan 7920162093 Handle recovery of virtual tables by recovering each shadow table individually, then writing the CREATE VIRTUAL TABLE statement directly into the sqlite_schema table.
FossilOrigin-Name: 5f2d5ccd56c06c3468377126acfd4be39b79b05bb6fb09b674b2e185df143aa3
2022-09-09 16:25:19 +00:00
dan 46d4398440 Update comments in sqlite3recover.h.
FossilOrigin-Name: 14164047c43e3ca43aa010c77ea00dfa85400e15645ee0f5b90a677898b6a836
2022-09-08 21:43:18 +00:00
dan 9a27d65044 Update the shell to use the recover extension for the .recover command.
FossilOrigin-Name: ae832e77084eddd696c80cb926d070a5db9d45dce34156a02522b3140e8f5e8b
2022-09-08 19:22:29 +00:00
dan 3887ffe82a Fix problems with recovering the sqlite_sequence table.
FossilOrigin-Name: 356d2209ea5f6b69ce15b62027c63419c2d039e52f01c74a3810a6317abf4fb0
2022-09-08 17:42:33 +00:00
dan 4a08e793c6 Add new test file recoverclobber.test.
FossilOrigin-Name: cb4e950c472bd24a79a8505a7f8e4c3a0f7821648297d05cc760738b777d5149
2022-09-08 11:04:23 +00:00
drh b53e0992b2 Enhance an assert() to impose for tighter constraints on the operation of pcache.
FossilOrigin-Name: 42105eb43a916a7cd839fa6c582eaffaef17f50b23c3e05142400c379feb73c7
2022-09-07 20:11:22 +00:00
drh 8c983ddb67 An improved technique for ensuring that the Rekey() operation does not
overwrite an existing page number.  This approach does not change the
semantics of the underlying pluggable pcache and it is easier to prove
correct.  This replaces the changes at
[aadd38f99a3e5abc] and [81cff9f1955874aa].

FossilOrigin-Name: 7dbddde79e3ca1b81e00288616fc27434698ce6dcb47714728fce8602ae91f45
2022-09-07 19:28:18 +00:00
dan abb28667f1 Ensure that the recover extension properly escapes CR and NL characters in text mode. Also that it holds transactions open on both input and output databases for the duration of a recovery operation.
FossilOrigin-Name: 6cca8913e703635ad89415a60fc84000ac188d9df43f45594b8ad87facb91d54
2022-09-07 16:41:33 +00:00
dan 497b3e6a11 Tests and a fix for the SQLITE_RECOVER_ROWIDS option.
FossilOrigin-Name: 1d5000f5718004110776ff58284d824854a93fe1cd1f6d8a4e8b8b0c2b2c3076
2022-09-06 19:38:06 +00:00
drh 2e440a0801 Add the tool/omittest-msvc.tcl script that will try to build using MSVC
using various compile-time options, to ensure that the compile-time options
all build without errors.

FossilOrigin-Name: 6b00ecb59fd303f7985902c35a46db9e729201d4beaedea46596b728d9e4b1c8
2022-09-05 22:54:36 +00:00
drh 0c4517a61a Get the tool/omittest-msvc.tcl script working.
FossilOrigin-Name: 82b89f8a074858a81d841dfc26436e8d39ce4907b8f989eba4d848db16758303
2022-09-05 22:53:24 +00:00
dan 108c51b6ae Fix a problem with script mode and lost-and-found tables.
FossilOrigin-Name: 09ec588d2fe24dd321e88318fe90a9ae912cbc73c8a2d59a10c821625dd12d9d
2022-09-05 21:22:35 +00:00
dan d5e3fe50f8 Add a mode to output SQL statements instead of populating a database to the recover extension.
FossilOrigin-Name: 73058416e7da6581000898b7988a7010e2ce6632246f4c12b4398700c7744b83
2022-09-05 21:00:22 +00:00
drh 4c8404e555 Add a prototype for a script, similar to tool/omittest.tcl, that is designed
to test builds using MSVC to ensure they work with various OMIT options.

FossilOrigin-Name: b1e37a0bd31fa9f8e58c2859a80c23a682b4702eefab2c55b243f1e21d475d6c
2022-09-05 20:42:01 +00:00
drh 6e440ab3de Fix the windows build so that it works with -DSQLITE_OMIT_AUTOINIT.
FossilOrigin-Name: f74a5ea8c986dc33d3afcda169c38abbe55728c56716cf9991a5e2ef7fc4917a
2022-09-05 18:17:03 +00:00
dan a768b67dcc Add the SQLITE_RECOVER_ROWIDS option. To specify that rowid values that are not also explicit INTEGER PRIMARY KEY values should be preserved.
FossilOrigin-Name: 69cc9aba56a196bbd159bd24868aa5ccc60bed0dc612d09ed8a3ae898f156809
2022-09-05 15:56:09 +00:00
dan 7302079dbe Add an option to assume the freelist is corrupt when recovering data.
FossilOrigin-Name: 253e498f5200b8b3e2bc309587af108dd1cec8a884b3d2a49d5406525c9a4b4c
2022-09-03 21:08:38 +00:00
dan be2e212cf9 Take the freelist into account when recovering data that is not linked in to any tree associated with a schema entry.
FossilOrigin-Name: dbd1f1efb349a9c8886e42b3f07d3f4c576924136f111558c7294d0a272e415a
2022-09-03 20:31:36 +00:00
dan f2f8a3a348 Further work on making the recover extension compatible with the .recover command.
FossilOrigin-Name: f2ac315844d8db1bd1c6950a4fef7c459ddd37cc21a8f3daafa5639fad8118e2
2022-09-03 20:07:39 +00:00