AI Search Expanded File Support Phase 4: PPTX Extraction Prompt

Status: Ready for implementation
Priority: High
Dependencies: Expanded file support Phase 1 PDF, Phase 2 DOCX, Phase 3 XLSX

Prompt

You are working in C:\code\stratofusion on Windows 11 PowerShell. Do not use WSL or bare bash. Use pnpm.

First:
- Run `git status --short` and inspect relevant `git diff`.
- If there are unrelated uncommitted changes, summarize them and ask before committing, overwriting, reverting, or cleaning them up.
- Do not commit user changes unless explicitly approved.
- Run `pnpm env:guard`.
- Read README.md and the relevant canonical docs under public/docs before coding.
- For this non-trivial feature, read docs/ai/CODEX_FEATURE_TEMPLATE.md and docs/ai/CODEX_PATTERNS.md before editing.
- Review Phase 1 PDF extraction in src/lib/search/ai/extraction/pdf-text.ts.
- Review Phase 2 DOCX extraction in src/lib/search/ai/extraction/docx-text.ts.
- Review Phase 3 XLSX extraction in src/lib/search/ai/extraction/xlsx-text.ts.
- Follow the same extraction architecture: dedicated parser module, typed parser error, narrow policy routing, existing normalization, and byte-limit enforcement.

Goal: implement Expanded File Support Phase 4 for AI Search: deterministic PPTX text extraction for AI indexing only.

Important architecture notes:
- AI indexing does NOT use src/lib/content-extraction.ts.
- The live AI indexing pipeline uses src/lib/search/ai/extraction/extract-document.ts via BackendIndexingRunner.
- Implement PPTX support only in src/lib/search/ai/extraction.
- Preserve existing text-like, PDF, DOCX, and XLSX extraction behavior.
- Do not implement legacy PPT, Google Slides export, ODP, Keynote, embedded media extraction, OCR, animations, transitions, or speaker timings in this phase.
- Do not mark PPT supported. Mark PPTX supported only after the PPTX extractor branch is implemented and tested.
- Legacy binary .ppt files must remain unsupported unless explicitly approved.

Scope:
1. Select and verify a PPTX extraction approach before adding dependencies.
   - Prefer a deterministic server-side JS implementation with no external binaries.
   - Current recommendation: spike a minimal OpenXML parser using direct dependencies on `jszip` and `fast-xml-parser`, unless a maintained PPTX parser proves clearly better.
   - Add direct dependencies only when the implementation imports them directly; do not rely on transitive dependencies from ExcelJS or other packages.
   - Verify license compatibility, TypeScript types, ESM/CJS import shape, Vitest behavior, Next build behavior, and production bundle boundaries.
   - Run `pnpm audit --prod` after adding any runtime dependency.
2. Extend src/lib/search/ai/extraction/extract-document.ts so PPTX inputs are parsed as PPTX, not decoded as UTF text or routed through other extractors.
3. Update src/lib/search/ai/extraction/supported-file-policies.ts so `.pptx` and the OpenXML presentation MIME are supported, while legacy `.ppt`, ODP, Keynote, and Google-native formats remain unsupported.
   - Important: the current priority unsupported presentation policy catches `.pptx` before supported policies. Remove only `.pptx` and its OpenXML MIME from that unsupported policy when adding the supported PPTX policy.
4. Add focused tests under src/lib/search/ai/extraction.
5. Add BackendIndexingRunner-level coverage in a separate PPTX runner test file.
6. Update README.md, relevant public/docs files, and 9.AI_Search_Plan_Consolidated_Implementation_Spec.md.
7. Confirm this prompt is linked from public/docs/prompts/INDEX.md; do not churn the catalog if it is already linked.
8. Run focused verification and commit the phase with a clear message such as `feat: add pptx ai indexing extraction`.

Supported PPTX inputs:
- Extension: .pptx
- MIME: application/vnd.openxmlformats-officedocument.presentationml.presentation
- Parameterized MIME must resolve correctly, for example:
  application/vnd.openxmlformats-officedocument.presentationml.presentation; charset=utf-8

Unsupported formats that must remain unsupported:
- .ppt and application/vnd.ms-powerpoint
- .odp and application/vnd.oasis.opendocument.presentation
- .key
- Google Slides native MIME types and exports
- Embedded media extraction for images, video, and audio
- Animations, transitions, and timings
- OCR for image-only slides
- Legacy binary Office formats and non-PPTX presentation packages

Library and parser guidance:
- Avoid broad multi-format extractors unless they are clearly safer than a narrow PPTX parser. They can pull in unrelated PDF/XLS/OCR behavior and make sanitized failure semantics harder.
- Treat `pptx-parser`, `officegen`, and `node-pptx` as unlikely primary choices unless a spike proves otherwise. `pptx-parser` is stale and render-oriented; `officegen`, `node-pptx`, `nodejs-pptx`, and `pptxgenjs` are primarily generators.
- `node-pptx-parser`, `pptx-content-extractor`, and similar newer packages may be spiked, but require extra scrutiny for maintenance, API stability, notes support, hidden slide support, and sanitized errors.
- A custom OpenXML parser is acceptable and probably the safest path if it stays narrow, typed, well-tested, and under the repo's 500 LOC file rule.
- If using `jszip`, the package is dual licensed `(MIT OR GPL-3.0-or-later)`; document that the MIT option is the accepted license path.

OpenXML extraction strategy for a custom parser:
- Validate PPTX as a ZIP package before parsing. Detect OLE Compound File magic bytes (`D0 CF 11 E0 A1 B1 1A E1`) as likely encrypted/password-protected Office content.
- Read slide order from `ppt/presentation.xml` and `ppt/_rels/presentation.xml.rels`; do not assume lexical ZIP entry order is presentation order.
- Resolve each slide target from presentation relationships, normalizing relative paths safely inside the package.
- For each slide, read `ppt/slides/slideN.xml`.
- Skip hidden slides only when visibility is reliably available. Common PowerPoint output stores hidden state as `show="0"` or `show="false"` on the slide root. If the chosen parser cannot detect this reliably, extract all slides and document that limitation in code/tests/docs.
- Extract plain text from slide text bodies in deterministic XML document order. Preserve paragraph line breaks and join adjacent text runs within a paragraph without formatting.
- Extract speaker notes by following the slide relationship in `ppt/slides/_rels/slideN.xml.rels` to a notes slide target such as `../notesSlides/notesSlideN.xml`; do not assume notes slide numbering matches slide numbering.
- For notes, prefer only the speaker-notes body placeholder when it can be identified. Avoid duplicating slide text, date/footer/header placeholders, or "Click to add notes" placeholder text.
- Include image/shape alt text from non-visual properties such as `descr` only if it can be extracted deterministically and tested. Otherwise document alt text as unsupported for this phase.
- Skip slide masters and layouts in this phase to avoid indexing repeated template text unless the selected parser exposes already-de-duplicated slide content.

Output format:
- Extract all non-hidden, non-empty slides in presentation order.
- Use 1-indexed slide numbers based on presentation order after reading `presentation.xml`.
- Skip entirely empty slides that have no title, body text, notes, or included alt text.
- If every slide is empty, return a non-indexable unsupported result with:
  PPTX did not contain extractable text for AI indexing
- Concatenate slide content with blank lines between slides:

  Slide 1: Title if present
  Body text
  Notes: Speaker notes if present

  Slide 2: Title if present
  Body text
  Notes: Speaker notes if present

- Do not emit a `Notes:` line when a slide has no speaker notes.
- Preserve metadata on successful extraction.

Text extraction rules:
- Extract plain text only; do not preserve bold, italic, font, color, size, animations, transitions, or timing metadata.
- Preserve line breaks between paragraphs and within text boxes where the XML structure exposes them.
- Join multiple text boxes/shapes in deterministic XML document order. Do not claim visual top-to-bottom or left-to-right ordering unless the parser explicitly implements and tests coordinate sorting.
- Skip charts, SmartArt, diagrams, and embedded objects unless their text is represented as normal extractable PPTX text or deterministic alt text.
- Do not include file paths, raw XML, raw parser errors, or slide text in failure messages.

Error and result semantics:
- Current extractDocument result types are only `success` and `unsupported`.
- Known non-indexable PPTX cases should return `unsupported` when cleanly identifiable:
  - empty/no extractable text
  - password-protected/encrypted files
- Parser failures should throw a sanitized `PptxExtractionError`; BackendIndexingRunner will mark the item failed with that message.
- Corrupt PPTX parser failure message:
  PPTX text extraction failed: file could not be parsed for AI indexing
- Password/encrypted unsupported message:
  Password-protected PPTX files cannot be indexed without a password
- Parser timeout failure message:
  PPTX text extraction timed out after {timeoutMs} ms
- Text-source PPTX input should fail with:
  PPTX extraction requires binary PPTX bytes for AI indexing

Limits and byte handling:
- Keep DEFAULT_MAX_EXTRACTED_BYTES behavior unless there is a strong reason to change it.
- Enforce source byte limit before parsing using the existing extractDocument path.
- Normalize extracted text through normalizeExtractedText().
- Enforce extracted UTF-8 text byte limit after parsing using the existing extractDocument behavior.
- Do not truncate and succeed in this phase. Oversized extracted text should fail clearly, consistent with PDF/DOCX/XLSX/text behavior.

Timeout requirement:
- Add DEFAULT_PPTX_EXTRACTION_TIMEOUT_MS = 15_000.
- Make timeout internally configurable/testable.
- If the selected parser is synchronous, a normal Promise.race timeout does not interrupt CPU-bound parsing once it starts.
- If using synchronous ZIP/XML parsing, either implement a real bounded parser path such as worker-thread parsing, or document the timeout as best-effort and do not claim hard cancellation.
- If using async ZIP/XML APIs, verify the timeout test proves the intended behavior without faking hard cancellation.

Suggested design:
- Add src/lib/search/ai/extraction/pptx-text.ts:
  - DEFAULT_PPTX_EXTRACTION_TIMEOUT_MS
  - PptxExtractionError
  - PptxExtractionFailureCode = "encrypted" | "parse_failed" | "timeout"
  - ExtractPptxTextOptions with timeoutMs and any test-only parser injection needed
  - isPptxPolicy(policy)
  - extractPptxText(bytes, options)
- Add src/lib/search/ai/extraction/pptx-test-support.ts for generated minimal fixtures.
- Add src/lib/search/ai/extraction/extract-document-pptx.test.ts rather than growing extract-document.test.ts too far.
- Keep parser-specific complexity inside pptx-text.ts. If pptx-text.ts approaches 500 LOC, split internal XML/package helpers into narrowly named files under src/lib/search/ai/extraction.
- Do not route text-like, PDF, DOCX, or XLSX formats through PPTX logic.
- Add the PPTX branch in extract-document.ts after XLSX and before text fallback, including the text-source guard.
- Catch PptxExtractionError with code "encrypted" in extractDocument and return unsupported.
- Let parse_failed and timeout errors throw so BackendIndexingRunner marks the item failed.
- Add `// Reason:` comments only for non-obvious tradeoffs, especially parser timeout limitations, hidden slide handling, notes placeholder filtering, or alt text extraction.

Tests to add/update:

1. supported-file-policies.test.ts
- PPTX supported by .pptx extension.
- PPTX supported by OpenXML presentation MIME.
- PPTX supported by parameterized OpenXML presentation MIME.
- Legacy .ppt remains unsupported.
- ODP, Keynote, and Google Slides native formats remain unsupported.
- Existing supported formats (PDF, DOCX, XLSX, text-like) still pass.
- getV1SupportedFilePolicies includes pptx only after implementation works.

2. extract-document-pptx.test.ts
- Extracts text from a simple single-slide PPTX with title and body.
- Extracts text from multi-slide PPTX and includes slide number delimiters.
- Extracts speaker notes by following slide relationships.
- Skips slides with no extractable text.
- Skips hidden slides if reliable hidden-slide metadata is implemented; otherwise documents and tests the extract-all limitation.
- Does not decode raw PPTX ZIP bytes as UTF text.
- Enforces source maxBytes before parsing.
- Enforces extracted text byte limit after parsing by rejecting with the existing extracted-size error.
- Handles corrupt PPTX by rejecting with sanitized PptxExtractionError text.
- Handles empty PPTX as unsupported with the no-extractable-text reason.
- Handles password-protected/encrypted PPTX as unsupported. If generating a real encrypted fixture is impractical, add a testability hook or OLE magic-byte fixture.
- Preserves metadata on successful PPTX extraction.
- Handles slides with only title, only body, only notes, and mixed combinations.
- Handles slides with multiple text boxes/shapes.
- Handles text-source PPTX inputs as parser failures, not text fallback.
- Respects the configured timeout according to the chosen parser design. Do not fake a hard timeout if the parser cannot be interrupted.
- Existing UTF/text, PDF, DOCX, and XLSX extraction tests still pass.

3. pptx-test-support.ts
- Generate small PPTX fixtures without large binary files.
- Suggested helpers:
  - createMinimalPptxBytes(slides: { title?: string; body?: string; notes?: string; hidden?: boolean; altText?: string }[]): Buffer
  - createEmptyPptxBytes(): Buffer
  - createCorruptPptxBytes(): Buffer
  - createEncryptedPptxBytes(): Buffer using OLE magic bytes, or a testability hook if a real encrypted package is needed
- Prefer hand-crafted minimal OpenXML ZIP fixtures when using a custom parser. Fixture generation may share low-level XML-writing helpers, but extraction tests must exercise real extraction behavior.

4. backend-indexing-runner-pptx.test.ts
- PPTX item with extractable text is chunked, embedded, and upserted with correct metadata.
- Empty/non-indexable PPTX is skipped clearly without vector upsert.
- Corrupt PPTX is failed clearly without vector upsert.
- Multi-slide PPTX text reaches the chunker; do not require exact chunk count unless the test controls chunking deterministically.
- PPTX with speaker notes includes notes in extracted text.

Docs to update:
- README.md:
  - AI indexing supports deterministic text-like formats, PDFs with embedded text, DOCX with extractable text, XLSX spreadsheets with extractable visible cell values, and PPTX presentations with extractable slide text and speaker notes.
- public/docs/SEARCH_FEATURES.md:
  - Add PPTX support.
  - Document slide-by-slide extraction, slide number delimiters, speaker notes inclusion, and no legacy .ppt.
  - Document no embedded media extraction, animations, transitions, timings, or OCR.
  - Document hidden slide behavior exactly as implemented.
- public/docs/PRIVACY_ARCHITECTURE.md:
  - Mention PPTX slide text and speaker notes may be extracted into text chunks and embedded when AI indexing is enabled.
- public/docs/API_REFERENCE.md:
  - Update AI indexing supported file type language to include PPTX.
- 9.AI_Search_Plan_Consolidated_Implementation_Spec.md:
  - Update expanded file support Phase 4 status and existing support language.
  - Avoid confusing this expanded-file-support phase with the separate Ask AI rollout phase; clarify the distinction if needed.

Verification:
Run at minimum:

pnpm test -- src/lib/search/ai/extraction
pnpm test -- src/lib/search/ai/indexing/backend-indexing-runner.test.ts
pnpm test -- src/lib/search/ai/indexing/backend-indexing-runner-pdf.test.ts
pnpm test -- src/lib/search/ai/indexing/backend-indexing-runner-docx.test.ts
pnpm test -- src/lib/search/ai/indexing/backend-indexing-runner-xlsx.test.ts
pnpm test -- src/lib/search/ai/indexing/backend-indexing-runner-pptx.test.ts
pnpm typecheck
pnpm exec eslint --config .eslintrc.json src/lib/search/ai/extraction
pnpm exec eslint --config .eslintrc.json src/lib/search/ai/indexing/backend-indexing-runner-pptx.test.ts

If package or bundling changes are nontrivial, also run:

pnpm build

Deliverable:
- Commit the changes with a clear message, for example:
  feat: add pptx ai indexing extraction
- Summarize changed files, including new files and dependency changes.
- Summarize exactly which PPTX cases are supported:
  - .pptx extension
  - OpenXML presentation MIME
  - slide text (titles, body, text boxes)
  - speaker notes
  - slide order preservation
  - multi-slide presentations
  - hidden slide behavior exactly as implemented
- Summarize skipped/unsupported cases:
  - empty PPTX
  - password-protected/encrypted PPTX
  - legacy .ppt
  - embedded media extraction
  - animations, transitions, timings
  - charts/SmartArt without extractable text
  - master slides/layout templates
  - ODP, Keynote, Google Slides native formats
- Summarize failed cases:
  - corrupt PPTX
  - parser timeout
  - text-source PPTX input
  - source bytes over maxBytes
  - extracted text over maxBytes
- Include verification command results.
- Call out remaining risks:
  - parser fidelity for complex presentations
  - hidden slide metadata reliability
  - slide text order is deterministic XML/package order, not necessarily visual reading order
  - speaker notes placeholder filtering may vary by producer
  - alt text extraction availability
  - password-protected detection relying on package/parser signals
  - synchronous parser timeout limitations if applicable
  - large presentations increasing chunks, embedding cost, and indexing latency
  - no OCR, embedded media extraction, animations, or legacy PPT support
  - no master slide/layout template extraction