Data Files Survive, but Their Tables Are Unknown?
Identify Unknown Data Files
Scenario Description
A set of PostgreSQL data files has been recovered, but the original directory or system catalogs are missing. Numeric filenames do not identify tables, and pg_dump cannot export them directly. Identify candidate table definitions first, then recover data with confirmed mappings.
Possible Causes
- Directory relationships and file-to-table mappings are incomplete.
- Only relation-file copies remain, with unreadable catalogs.
- Several tables share field types and require additional attribution evidence.
Challenges
Conventional recovery often requires complete files, a running instance or a usable backup. In these situations, choose a method based on the evidence that survives.
Common Errors and Symptoms
- - Files are readable, but reliable table names or filenode mappings are missing.
- - A file matches multiple candidate definitions.
- - The base file survives, but segments or TOAST files may be missing.
Why Traditional Methods Fail
- - Creating an empty database does not restore original file ownership.
- - A filename or one successful sample does not identify the business table.
- - Recovering only the main table may omit values stored in TOAST.
PDU Solution
Probe samples relation files directly and lists possible table matches using saved definitions. ProbeBatch identifies multiple files. After confirming which file belongs to which table, use the export commands to extract data without starting the damaged instance.
Recovery Principle
- Preserve Files and Definitions
- Identify Schema Candidates
- Confirm File Mappings
- Export Readable Data
PDU Core Capabilities
- Single-file sampling and batch candidate analysis.
- Preserve multiple candidates and unmatched reasons.
- Generate file whitelists and export mappings.
- Parse segments with matching versions and logical page sizes.
- Retain TOAST dependencies and missing-fragment reasons.
- Resume from native completion records without duplicate export.
Recovery Steps
- 1
Initialization
Keep data files, schema evidence and working directories separate. With an existing restore/tab.config, specify a file directly. To generate definitions from a readable database directory, use scan manual;. Disk recovery and scan manual do not require b; first.
Important: Program and Source Must Match
Logical page size is selected by the program build. Use the matching program for 8KB or 32KB sources. The ds idx candidate grid is not the logical page size of a relation file.
- 2
Use the Default Definition File
PDU samples the file using restore/tab.config. Retain all decodable candidates and reasons for unmatched input. Successful sampling is not a complete file export.
- 3
Use an Explicit Definition File
Use the corresponding command example below, replacing paths, objects and identity parameters with evidence from this run.
- 4
Analyze Data Files in a Batch
Batch identification retains candidate inventories. valid generates the restore/dropscan/datafile.txt file whitelist. Retain tab.config, tab.fullmeta and native candidate files when complete schema candidates are needed.
- 5
Confirm Schema Candidates
Several tables with identical field types may match the same file. Confirm the table using source information, field meanings, catalog relationships and business records. A definition that can decode the file is not enough on its own.
Important: Preserve Ambiguity
ds probe distinct <id>; narrows duplicate candidates using the current displayed ID and updates mappings. Use it only after explicitly confirming a selection. Take IDs from current show output rather than reusing example IDs in other tasks.
- 6
Export Data
Export uses selections in probe_candidates.json and probe_map.txt. Files with native completion records are skipped accordingly. If sources, definitions or page size change, retain the old task and create a new working directory.
- 7
Handling TOAST Files
Preserve the main table, TOAST files and segments together. Probe-generated TOAST indexes and page-size records must match the export program. Retain original reasons for missing fragments and decoding failures. A main-table sample match does not prove complete external values.
Quick Command Examples
Preserve frozen sources and launch PDU outside them. Write configuration snippets to pdu.ini and execute PDU commands after startup. Replace example paths, versions, times and task identities with actual evidence.
# Initialization
scan manual;# Use the Default Definition File
ds probe /recovery-input/datafiles/12345;# Use an Explicit Definition File
ds probe /recovery-input/tab.config /recovery-input/datafiles/12345;# Analyze Data Files in a Batch
ds probebatch /recovery-input/tab.config /recovery-input/datafiles;
ds probebatch valid /recovery-input/datafiles;# Confirm Schema Candidates
ds probe show;# Export Data
ds unload tab 12345;
# Or export all selected files:
ds unload all;
ds log;Export Results
Schema Candidates
probe_candidates.json retains file candidates and selections.
Export Mappings
probe_map.txt retains confirmed mappings and page sizes.
Data and Scripts
CSV and optionally generated COPY scripts, at paths reported by PDU.
After export, record PDU’s native success and failure counts and original errors, retaining checkpoints. Resume uses native records and file metadata without rereading exported data or automatic trial import.
Prerequisites
- Preserve stable data-file copies and all segments, including the base file and numeric .1, .2 segments.
- Prepare table definitions from matching sources and use the correct major-version and logical-page-size program.
- Probe lists possible schema matches. Confirm which table the file belongs to using catalogs, source information and other available evidence.
- Prepare a matching program, license and sufficient space. Keep original evidence read-only and place the PDU launch directory outside the sources.
Important Notes
- A sample match is neither a complete export nor confirmed business attribution.
- Candidates with identical layouts require source, field-semantic and catalog evidence.
- Missing segments or TOAST remain unresolved rather than being presented as complete records.