---
title: "Folders"
description: "Control search scope, file types and ignore rules."
---

## Add and index

Use the sidebar to add a directory in folder management. SAG Wiki indexes the selected file types while applying ignore rules. Later changes trigger incremental indexing. Indexing runs in the background: adding a large folder returns immediately, progress appears at the bottom of the sidebar, and search and Q&A stay available during indexing. During file-monitor setup, a preparation status is shown; large folders may take a few minutes before file scanning starts. If file monitoring fails, scanning continues and the completed index retains a warning that automatic updates are limited. Rebuild the index in indexing settings to retry. Explicitly adding a folder again resets its previous monitor before registering a new one. After the directory is restored, new file changes continue to update the index automatically.

Markdown, plain text, RTF, Word, PowerPoint, Excel, OpenDocument, CSV, EPUB and text-based PDFs are enabled by default. Built-in anydoc parses Office, RTF, tables, ebooks and PDFs locally without downloading components. Code and configuration files follow their type settings. Install Docling when needed for scanned PDFs and other formats.

![The example folder after indexing](/images/product-folders-en.png)

## Progressive indexing and search availability

PDFs first attempt to extract their basic text layer. Mixed text/scanned PDFs and image-only scans continue with background OCR and table structure processing. PDFs with usable text on every page retain built-in anydoc completion without an additional Docling enhancement pass. Once basic text is indexed, it can be searched while enhancement continues. Search coverage increases as background processing finishes. Basic indexing and enhancement use separate background workers. New or changed files in the same folder can continue extracting and indexing basic text while an older PDF undergoes OCR or table enhancement. Index details list basic indexing and background enhancement progress separately. Actual concurrency remains subject to memory budgets, parsers and source-change validation. The file list shows:

| Status | Search availability |
| --- | --- |
| Basic text searchable · Enhancement pending | Extracted text layer; OCR and table enhancement have not finished |
| Basic text searchable · Enhancing | Basic text is searchable while scanned content and tables are processed |
| Basic text searchable · Enhancement failed | Basic text remains searchable; check indexing details and retry |
| Awaiting parsing | No searchable text is available; background processing will try applicable parsers |
| Parsing content · Not searchable yet | Document processing has not produced searchable text yet |
| Awaiting OCR | Scanned content confirmed to lack a usable text layer must wait for OCR |
| Recognition failed · Not searchable | No completed basic text is available; check indexing details and retry |

With local built-in parsing, supported non-PDF documents such as Word, RTF and spreadsheets also extract text on the basic-indexing worker, without waiting for a large PDF to finish enhancement. Files that fail built-in parsing or need additional components enter the background parsing queue. Formats configured for cloud parsing retain their existing parser order.

The basic stage may omit scanned images, complex tables or final source positioning. Wait for enhancement and check the original before relying on complete coverage. Image-only scans are not marked searchable simply because they are queued. Pausing background processing preserves access to completed basic indexes.

![Progressive indexing with basic-text and background enhancement statuses](/images/product-progressive-index-en.png)

## Office text only

Enable **Office text only** in **Settings → File settings** to index DOCX/PPTX text while skipping images and embedded objects. It is off by default and does not cover legacy DOC/PPT, DOCM/PPTM or other formats. Body text, tables, text boxes and supported notes retain built-in parsing. Known Word text parts, such as footnotes and endnotes, are retained by their declared content type, including custom names or paths. Text inside images and embedded attachments is not recognized.

Filtering occurs before ZIP members are opened, read or decompressed, retaining only necessary Office XML by path and content type. Container directories, relationships and text XML still need reading; source fingerprinting still reads the original file in chunks, so this does not promise zero disk I/O over compressed image bytes. Input, XML decompression, entry count, nesting, output and timeout limits remain enforced.

This mode takes precedence over ordinary cloud parsing for these formats. Fallbacks must provide the same resource-skipping guarantee. Currently only the restricted-package built-in path qualifies, so a failure never automatically invokes ordinary Docling or cloud parsing. Unsupported structures, such as nonstandard main-part locations or inline binary XML, report an error rather than silently claiming complete coverage. Toggling the setting rechecks affected documents; content and failure caches distinguish the two modes. Original files are unchanged.

![Office text-only setting](/images/product-office-text-en.png)

## Limit your search scope

Add only the directories you need. Query scope is independent from the folder you are currently browsing. Browsing a folder does not change the search or Q&A scope automatically.

## Ignore files

Both `.gitignore` and `.sagignore` apply in project roots and subfolders. `.sagignore` has higher priority.

```text
node_modules/
dist/
.env
**/*.generated.md
```

Exclude sensitive files that you do not want retrieved or sent to model services. After changing rules, check index status to confirm coverage.

## Remove and clear

Removing a directory from the knowledge base removes its registration and index while preserving original files. Removing a subfolder records an exclusion; adding the same subfolder restores it.

Clear files in settings clears the knowledge base index. Conversation records and file indexes are managed separately; read the action description before proceeding.

## Check from the CLI

```sh
sag list
sag check
sag add --types md,txt,html,py,js,ts,tsx,json,yaml
```

`--types` selects text or built-in document formats for that directory, such as `md,docx,pdf,xlsx`. Basic indexed text is searchable before background enhancement finishes.

## File settings in detail

Location: **Settings → File settings**. Global settings apply to folders without their own overrides. Save file-type, exclusion or chunking changes first, then check that indexing completes.

### File types

| Option | Contents | Default |
| --- | --- | --- |
| Markdown | `.md`, `.markdown` | On |
| Text and documents | `.txt`, `.rtf`, `.docx`, `.pptx`, `.xlsx`, `.odt`, `.epub`, `.pdf` | On |
| Code | `.py`, `.js`, `.ts`, `.rs`, `.html`, `.css` | Off |
| Configuration | `.json`, `.yaml`, `.toml`, `.xml`, `.ini` | Off |

Disabling a type removes it from indexing and retrieval while preserving original files and browsing. For code repositories, enable code and configuration or select extensions using CLI `--types`.

![Global file types and indexing settings](/images/product-file-settings-en.png)

### Other formats and conversion

Text and documents is one setting, enabled by default. TXT and RST are read directly; the following documents use built-in anydoc first when MinerU is disabled: Word (`.doc`, `.docx`, `.docm`), PowerPoint (`.ppt`, `.pps`, `.pot`, `.pptx`, `.pptm`, `.ppsx`, `.ppsm`), Excel (`.xls`, `.xlsx`, `.xlsm`, `.xlsb`), OpenDocument (`.odt`, `.ods`, `.odp`), RTF, CSV, EPUB and text-based PDFs.

Other formats lists scanned PDFs, images, audio, video, email, Apple iWork and some document templates. With MinerU disabled, text-based PDFs use the built-in parser. Choose Select and pick the formats you need. With MinerU enabled, its supported image formats can be applied without installing Docling. If other local support is missing, choose Install and apply, confirm installation, then apply the selection. Document and media components are installed separately according to the selected formats. Scanning files never downloads components automatically.

With MinerU disabled, an installed Docling handles PDFs requiring OCR or content unsupported by the built-in parser. With MinerU enabled, use the [MinerU-first flow](/en/docs/models#mineru-document-parsing). If Docling is absent, indexing details explain the required action; those files are not parsed again until the component is installed or the file changes, so install it and reindex. Damaged and encrypted documents report parsing errors. A failed conversion records its reason and is not parsed again on every scan until the file, the parsing engine or component, or the conversion budgets change; changing a conversion budget in file settings, or choosing Reindex, retries it.

Long PDFs are converted sequentially in parts of at most 50 pages. Each part independently tries the applicable parsers, and successful parts are cached for retries. Scanned PDFs that can be identified as having no text layer are handed off immediately to an installed OCR component, avoiding unnecessary text-conversion waits. The complete enhanced document is merged only after every enhancement part succeeds. Failures never publish incomplete enhanced text; completed basic indexes remain available. See [Document parsing](/en/docs/models#mineru-document-parsing).

Windows recognition workers report memory-limit failures even when the process exits during startup. Failed results do not replace a completed index.

Standalone images use a lightweight OCR path and do not load the layout and table models intended for scanned PDFs. Scanned PDFs and files requiring full layout structure still use the Docling document pipeline. Both paths run entirely offline. When several standalone images are processed together, the backend reuses OCR workers so models are not loaded for every image; workers exit after about 120 idle seconds and release their resources. Image recognition also follows **Settings → File settings → Docling conversion timeout**; waiting for a worker, memory admission and model loading all count toward this timeout. Increasing conversion workers processes more images at once, but excessive concurrency can cause CPU contention and heat, so throughput may stop improving.

- **Select all / Clear selection** selects or clears all conversion formats. An empty selection disables additional formats without affecting enabled built-in documents.
- **Apply** applies the format selection; global settings still need to be saved.
- **Conversion workers** defaults to 1, allows 1–16, is shared across folders and takes effect on the next indexing task. Actual concurrency is also limited by the shared resource budget. Previously saved values are preserved.
- **Conversion memory** defaults to automatic physical-memory allocation;[Runtime settings](/en/docs/preferences#runtime-settings) can adjust its percentage and per-job budget. By default, Docling tasks share half of physical memory; a regular document task may use one quarter. On smaller machines, the task allowance retains 4 GiB for OCR where possible, without exceeding the total budget. For example, a 64 GiB computer allows 16 GiB per task and 32 GiB in total. In automatic mode (per-task budget set to 0), each standalone-image OCR worker reserves up to 2 GiB, capped by the total budget. A nonzero per-task memory budget applies to both images and documents, subject to the total budget. Saved settings apply to new tasks; image tasks recreate idle workers that use old settings. Document accounting includes child processes, while lightweight image workers monitor their own process. Exceeding a limit stops processing without replacing an existing conversion result. Additional tasks queue when the budget is reserved; failure to read physical memory is reported as an error.
- **Uninstall related components** removes SAG-managed document and media components, runtimes, recognition models, FFmpeg and download caches. Original files, converted files and existing indexes remain; built-in document parsing stays available.

Native text is indexed first, built-in documents and enabled additional formats are parsed in parallel, and finished results are then indexed. Already indexed content can be searched before every conversion finishes.

Converted files go under `.sag-converted/` in the added root, preserving subfolders:

```text
material/report.pdf
→ .sag-converted/material/report.pdf.md
```

macOS hides dot-prefixed directories by default. Built-in parsing does not upload files. Docling may require additional recognition models for scans, audio and video; first use includes model preparation. After a parser upgrade, reindexing updates built-in conversion results that have not been edited manually.

### Import an offline document component after a download failure

In the file parsing support installer, choose **Import offline component**, then select the extracted complete document component folder containing `manifest.json`. SAG checks its platform, component version and file integrity, then runs a sample conversion. Once ready, choose **Apply selected formats**. Reindex the folder if previous files failed to convert.

The component must include Python, recognition dependencies and models. A standalone Python installer or model download is not a complete component. This entry supports document components only; audio and video still use online installation. When document and media formats are selected together, the entry is still shown; after the import finishes, choose **Retry installation** to install the media component online. A failed import does not fall back to online downloads. Select another folder to retry offline, or choose Retry installation to install online.

#### When only the uv download fails

If the error says the runtime manager could not be downloaded (for example, `github.com/astral-sh/uv` timed out), open the address from the error in a browser to download the uv archive (`uv-x86_64-pc-windows-msvc.zip` on Windows x64). Extract it, choose **Import offline component** and select the extracted folder. Once SAG confirms the uv version matches the application, it skips that download and continues downloading Python, dependencies and models online.

The remaining steps still need network access. If those addresses are also unreachable, set the following environment variables, run `sag service restart` in a terminal opened after setting them, then retry:

| Download | Environment variable | Notes |
| --- | --- | --- |
| Python runtime | `UV_PYTHON_INSTALL_MIRROR` | A python-build-standalone mirror, for example `https://registry.npmmirror.com/-/binary/python-build-standalone` |
| Recognition dependencies | `UV_DEFAULT_INDEX` | A PyPI mirror, for example `https://pypi.tuna.tsinghua.edu.cn/simple` |
| Recognition models | `HF_ENDPOINT` | A Hugging Face mirror, for example `https://hf-mirror.com` |

Dependencies are checked against pinned hashes, so a mirror serving different files fails the installation instead of installing altered files. If the computer is fully offline, use a complete offline component instead.

Environment variables only affect newly started processes. The installation runs in the background service, and the app reuses a service that is already running, so quitting and reopening the app alone does not apply them; after setting the variables run `sag service restart` from a new terminal, or confirm the restart with `sag service status`.

While the recognition models download and the runtime is unpacked, the installer commits about 2.5 GiB of memory; when memory is tight the installation stops and the error shows a message such as `memory allocation ... failed`. Close memory-hungry programs and retry; if it keeps failing, open **Open install log** for the last lines, or switch to a complete offline component.

Obtain a complete folder matching your system, architecture and component version from your administrator or maintainer. Extract archives before selecting the folder. Current official Releases do not distribute a separate complete offline component; the application installer is not a component package. The import entry is available in application builds containing this feature, not the official 0.2.7 package.

![Offline document component import after a download failure](/images/product-offline-component-en.png)

Component providers can use the repository's `scripts/package-docling-component.py` to assemble a complete manifest-verified component from a relocatable Python runtime, dependency directory and model directory. SAG copies and checks it on the target computer before activation; no application installation folder or service port needs configuring.

### Watch file changes

Enabled by default. Additions, edits, deletions and renames trigger indexing updates under the configured rules. When disabled, manually reindex as needed rather than assuming the index is current. Enabling it again triggers a file check.

Watching runs in the shared background service. Closing the desktop window does not stop that service.

### Show hidden folders

Off by default. It affects only the right-side file list; the left sidebar still hides these folders. It does not control indexing, which follows file-type and exclusion rules.

### Index workers

Defaults to 8, allows 1–64 and applies to the next indexing task. Changing only worker count does not rebuild existing indexes. Lower it for large files if resource use is too high.

Text index preparation shares a 512 MiB budget, with at least 64 MiB reserved per file. This budget also limits concurrency; later files wait when it is fully reserved. Total application memory includes additional database and cache usage.

Each file has a chunk budget of at least 16 MiB. Heading costs are counted per final chunk, without counting a heading again for paragraphs merged into that chunk. Files exceeding resource limits are reported in index history, and their original contents remain unchanged.

### Excluded directory names

Enter one name per line. Defaults include `.git`, `node_modules`, `dist`, `target`, `.venv`, `venv` and `__pycache__`. Excluded directories remain browsable but are not indexed.

Use `.sagignore` for more precise path or file patterns, directory rules and `!` negation. These combine with global exclusions and type selections to determine coverage.

### Reindex and history

**Rebuild failed files only:** when there are file-level failures, the indexing status card in the lower-left corner shows the failed-file count and a **Rebuild index** button. Click it to immediately retry current failed files across all added folders, without opening a list or confirming. Successful indexes and converted files are kept; this action does not scan the entire folder. Click the status text above the button to inspect failure details.

Retrying a failed PDF also restores available basic text first, then continues enhancement in the independent background worker so new files can keep building their basic indexes.

The card shows progress while rebuilding and prevents duplicate submissions. Both native and converted files count toward the rebuild history. Resume paused indexing before starting. Recovered files are removed from the current failures; unresolved component, file or permission problems remain visible. Stale failure records are removed for deleted or excluded files and files replaced by same-named directories; existing indexes inside those directories are preserved. If only a folder-reading or watcher error remains and there are no retryable files, the button is hidden; inspect the details and use folder-level or global reindexing.

![Rebuild failed files from the lower-left status card](/images/product-index-retry-en.png)

Global **Reindex** checks all registered directories using saved types and exclusions, updating changed indexes. Save relevant changes before running it.

Open **Index history → View** to inspect progress, completion and failures. Check error paths and messages rather than assuming a total file count means everything succeeded.

Exceptionally long table rows are indexed in smaller passages without editing the original file or its line breaks. Results still point to the original locations. File size and memory budgets remain enforced; exceeding them reports an error instead of writing an incomplete index.

The bottom of the sidebar shows the current stage, progress and partial failures. During file conversion, a separate **Parser** line identifies the parser actually in use, such as MinerU, anydoc or Docling. When concurrent files use different parsers, their names appear together. The label updates when a failed attempt falls back to another parser. Before a parser starts, it shows “Preparing conversion…”; the line disappears outside conversion stages. Index details also identify the parser for each active file. After pausing, a file that is still finishing continues to show its parser until that work ends.

During long-PDF conversion, the sidebar also shows the current document’s part number, total parts, and original page range. Parts do not count as separate files; cached parts also advance the indicator. The status card stays visible between parts and during parser fallback, showing “Processing document parts…” while waiting for the next parser. Index details show part progress for each document.

![PDF part progress alongside file counts](/images/product-pdf-part-progress-en.png)

Longer text wraps while keeping the close button available. Click the status to inspect current tasks and failed files.

After you dismiss a failure notice, the app remembers the current failure details and keeps that notice hidden when you reopen the interface. New indexing runs still show progress, and changed failure details trigger a new notice. You can still inspect dismissed failures through the index status card on the Files page or **Index history → View**.

![Sidebar conversion progress and active parsers](/images/product-index-status-en.png)

![After dismissal and reopening, the sidebar notice stays hidden while index details remain available on the Files page](/images/product-index-status-dismissed-en.png)

**Removed files and folders → Restore** adds an excluded path back and starts indexing. Restore only content you want searchable again.

![Completed indexing history and file counts](/images/product-index-history-en.png)

### Recent indexing and failures

Index details shows the latest record for each folder. Expand older records to inspect earlier runs. A failed run includes a readable explanation and file details; use the file entry to locate the affected source. Clearing history removes records, not the existing index.

![Index failure details](/images/product-index-failure-en.png)

Indexing details show the **Text indexing → File conversion → Index converted files** stages, with progress for the current stage. Already indexed content remains searchable. Select **Pause indexing** to pause after in-progress files finish, or **Resume indexing** to continue.

## Per-folder index settings

![Index options for one folder](/images/product-folder-options-en.png)

Open **Index settings** from a folder menu to select native types and conversion formats for that folder. Settings apply to the directory and subfolders without their own configuration; child overrides take precedence.

Options are stored in the managed section of that folder's `.sagignore`. Existing handwritten rules are preserved.

- **Save** stores options and updates the index under the new rules.
- **Reindex** clears that folder's old index and rebuilds it. Other folders and original files are preserved.
- **Cancel** discards unsaved option changes.

Per-folder Reindex rebuilds that directory; global Reindex checks and updates changed content under saved rules. Their scope and behavior differ.

## Web clipping folder

[Web Clipper](/en/docs/web-clipper) saves independently. To index its output, choose Add clipping folder in Web Clipper settings and select the actual save folder. Existing watch and indexing rules then apply.

First-time initialization creates `Documents/sag-wiki/clippings` and adds `Documents/sag-wiki` as the default managed folder. Existing libraries and removed folders are not added again on restart.

## Follow import progress

Open an imported folder to see a persistent status above its file list: queued, scanning, converting documents or indexing converted results. It includes current filenames and counts for the current stage, rather than a misleading overall percentage. Unknown totals have no percentage. Use **View indexing details** to inspect active tasks and errors. If another folder is being processed, the waiting state is explicit; completed folders retain their indexed-file count.

![Conversion progress and current file in a folder](/images/product-folder-progress-en.png)

Adding a folder restores the sidebar status. The app also checks backend snapshots to recover missed progress events. Removing a top-level folder cancels its ongoing or paused conversions and remaining indexing so queued folders can proceed, without deleting original files or stopping other folders.

Selecting a sidebar file switches directly to the document once its content is ready, keeping the current content during loading without an intermediate folder list. Rapid selections honor the latest file; a failed read preserves the previous page and displays an error.

Previously failed files display “Retrying failed conversions” while being retried. First conversions retain “Converting files”. This status distinction does not change reuse of successful results.