IBD Files Are Incomplete and Pages Are Scattered on Disk?
Recover Scattered InnoDB Pages
Scenario Description
Recovered IBD files are incomplete, or only scattered InnoDB pages remain in a disk image. Normal file chains cannot read the table, but clustered pages, SDI or dictionary evidence may survive. Preserve candidates first, then bind source, generation and definitions rather than joining pages by tablespace ID alone.
Possible Causes
- Files were truncated, directories lost or media partially damaged.
- The image contains pages from different object generations.
- External values, Undo or Redo sources are incomplete.
Challenges
Conventional recovery often requires complete files, a running instance or a usable backup. In these situations, choose a method based on the evidence that survives.
Common Errors and Symptoms
- - Whole-file parsing fails, but scanning finds clustered-page candidates.
- - Page chains are broken, with attribution conflicts or sparse dependencies.
- - Pages contain delete marks, bad checksums or incomplete LOB values.
Why Traditional Methods Fail
- - Matching space_id/index_id values do not establish one object.
- - Secondary-index and auxiliary pages are not directly business rows.
- - Damaged content and confirmed records need separate delivery.
PDU Solution
DropScan scans the confirmed file manifest and saves pages, metadata and conflict records. Once the engineer confirms the binding, PDU extracts recoverable rows using the fixed physical layout. Supplemental files are handled in separate tasks. Unconfirmed content stays separate from normal export results.
Recovery Principle
- Scan Declared Sources
- Inspect Candidates and Conflicts
- Freeze Definition Binding
- Export Recoverable Rows
PDU Core Capabilities
- Retain source ranges, domains and object generations.
- Inspect pages, metadata and conflicts separately.
- Bind clustered-index identity and physical layout explicitly.
- Retain LOB, compression and key dependencies.
- Preserve parent-task relationships for supplemental sources.
- Support bounded scanning and native checkpoint resume.
Recovery Steps
- 1
Initialize the Scan Task
Launch PDU in the result directory. The first scan reports DS_TASK beneath .pdu-work. Replace <DS_TASK> with the absolute native task path from this run; parent uses the corresponding parent recovery-task path.
chunks limits persistent scan windows for a run. PHASE=PAUSED CHECKPOINT_SAVED=1 is a safe pause, not completion of the full scan.
- 2
Obtain Schema and Source Evidence
Version 8.0 uses explicitly selected SDI candidates. Version 5.7 uses SYS_TABLES/SYS_INDEXES and matching FRM. Retain raw and record offsets, mappings, tablespaces, clustered indexes and metadata-write versions separately. Metadata versions do not directly identify the source server version.
- 3
Bind Table Definitions
DDL, SDI, SYS/FRM, compressed and encrypted sources use their corresponding binding formats. Explicitly select matching sources and reasons for external LOB, keys and metadata replacement pages. Retain AMBIGUOUS when multiple definitions decode successfully.
Important: Schema Compatibility and Business Attribution
Successful trial decoding shows that a page can be parsed with the selected definition; it does not identify the business table. ATTRIBUTION_STATE may remain UNKNOWN, and trial batches cannot be unloaded directly. Select the source generation, clustered-index identity and table definition before exporting.
- 4
Parse Sparse Pages
Parse records from clustered business pages. Secondary indexes, SDI and FULLTEXT auxiliary pages are not complete business rows. Retain separate reasons for missing pages, damaged pages, opaque ciphertext and I/O errors without zero filling.
- 5
Supplement LOB and Metadata Pages
Adding fragments or changing dependencies or bindings creates a new saved recovery view and internal task. SQL is exported directly to the launch directory. Specify parent to reuse completed units from the parent task. PDU's dependency rules determine what can be reused or must be recalculated; parent output is not modified.
- 6
Continue the Recovery Task
Keep original sources, bindings, build and policy unchanged. When sources or identity change, retain old evidence and use a new task. Do not resolve conflicts by appearance order or maximum LSN.
- 7
Handle Damaged and Delete-Marked Pages
Use PDU_DROPSCAN_SALVAGE_BINDING 1 to declare an explicit policy and reason for retaining bad-checksum pages, deleted/free/orphan records or proven retired leaves. Disabled by default, flags are 1, 2 and 4, accepting combinations 1–7. Confirm recovery scope from surviving sources.
Interpretable content with insufficient integrity is written to separate quarantine results rather than normal data SQL; use filenames reported by PDU. Untrusted boundaries, missing pages and incomplete LOB values retain isolation reasons without being treated as confirmed business records.
Quick Command Examples
Preserve frozen sources and launch PDU outside them. Write configuration snippets to pdu.ini and execute PDU commands after startup. Replace example paths, versions, times and task identities with actual evidence.
# Initialize the Scan Task
ds idx /recovery-input/sources.manifest chunks 4;
ds show <DS_TASK> sources;
ds show <DS_TASK> candidates;# Obtain Schema and Source Evidence
ds show <DS_TASK> metadata;
ds show <DS_TASK> objects;
ds show <DS_TASK> conflicts;# Bind Table Definitions
ds bind <DS_TASK> /recovery-input/binding.manifest orders-A;# Parse Sparse Pages
ds unload <DS_TASK> orders-A units 4;# Supplement LOB and Metadata Pages
ds unload <DS_TASK> orders-B parent <PARENT_RECOVERY_TASK>;# Continue the Recovery Task
ds resume idx <DS_TASK> chunks 4;
ds resume unload <DS_TASK> orders-A units 4;Export Results
Pages and Bindings
Internal tasks record file sources, candidate pages, fixed bindings and object generations.
Recovered Data
Exported SQL is saved in the launch directory. Only records meeting the recovery rules are included in the normal output.
Quarantine and Checkpoints
Record damage, missing pages, ambiguity and historical dependencies separately, with native pause state retained.
After export, record PDU’s native success and failure counts and original errors, retaining checkpoints. Resume uses native records and file metadata without rereading exported data or automatic trial import.
Prerequisites
- Preserve stable read-only sources, explicit scan ranges and source domains/generations.
- Clustered identity, definitions and physical layouts require independent evidence rather than guesses from filenames or space_id.
- Prepare a matching program, license and sufficient space. Keep original evidence read-only and place the PDU launch directory outside the sources.
Important Notes
- Binding identities and physical parameters must come from actual evidence rather than copied example numbers.
- Policies for bad checksums and deleted/free/orphan records must be declared explicitly; they do not enter normal data by default.
- Overwritten bytes, missing LOB values and untrusted boundaries cannot be recreated.