← XeFM crftwr/xefm on GitHub · craftware

Directory Diff Viewer System

Overview

The Directory Diff Viewer is a sophisticated component that provides recursive directory comparison with a tree-structured display. It enables users to compare two directory trees side-by-side, identifying differences in files, directories, and their contents.

Architecture

flowchart TB
    View["DirectoryDiffView (Widget) — main thread<br/>dual-pane tree · navigation · events · render"]

    subgraph WORK["Background threads (daemon)"]
        direction LR
        Coord["scan coordinator"]
        ScanW["scanner worker"]
        CmpW["comparator worker"]
        Coord --> ScanW
        Coord --> CmpW
    end

    Scanner["DirectoryScanner<br/>lists dir → {relative_path: FileInfo}"]
    Diff["DiffEngine<br/>builds + classifies the unified tree"]

    subgraph DATA["Data model"]
        direction LR
        Tree["TreeNode<br/>children · left/right FileInfo · difference_type"]
        FInfo["FileInfo<br/>path · size · mtime · is_dir · accessible"]
        DType["DifferenceType (enum)<br/>IDENTICAL · ONLY_LEFT · ONLY_RIGHT<br/>CONTENT_DIFFERENT · CONTAINS_DIFFERENCE · PENDING"]
    end

    View -->|"enqueue visible/expanded first (priority queues)"| Coord
    ScanW --> Scanner
    CmpW --> Diff
    Scanner --> FInfo
    Diff --> Tree
    Tree --> DType
    Diff -->|"update tree under RLock, repaint"| View

    classDef main fill:#1a5490,stroke:#7fb3d5,color:#fff;
    classDef work fill:#8b2e24,stroke:#e0897f,color:#fff;
    classDef engine fill:#1e7e34,stroke:#7fd39b,color:#fff;
    classDef data fill:#9a6308,stroke:#e0b45f,color:#fff;
    class View main;
    class Coord,ScanW,CmpW work;
    class Scanner,Diff engine;
    class Tree,FInfo,DType data;

Core Components

DifferenceType Enum

Priority scheduling

FileInfo Class

Main Classes

DirectoryDiffView (Widget)

DirectoryScanner

DiffEngine

TreeNode

Key Features

Progressive Scanning

The viewer implements progressive scanning to provide immediate feedback:

  1. Initial Display: Shows directory structure immediately
  2. Priority Scanning: Scans visible items first
  3. Background Completion: Continues scanning in background
  4. On-Demand Expansion: Scans subdirectories when expanded

Threaded Comparison

File comparison runs in background threads:

Tree Navigation

Users can navigate the comparison tree:

Implementation Details

Scanning Algorithm

# Pseudo-code for progressive scanning
def scan_directory(path, priority):
    # Scan immediate children first
    entries = list_directory(path)
    
    # Yield results immediately for display
    for entry in entries:
        yield entry
        
    # Queue subdirectories for later scanning
    for subdir in subdirectories:
        queue_scan(subdir, lower_priority)

Comparison Strategy

The viewer uses different comparison strategies based on file type:

  1. Directories: Compare by existence and structure
  2. Small Files: Full byte-by-byte comparison
  3. Large Files: Size and timestamp heuristics first, then content
  4. Symlinks: Compare target paths

Performance Optimizations

Thread Safety and the Thread → UI Bridge

Scanning and comparison run off the main thread, but all rendering stays on the main thread — PuiKit has no cross-thread draw. Two mechanisms keep that safe.

One reentrant lock

A single threading.RLock (self._lock) guards every tree mutation and the reflow that rebuilds the flattened visible list. The PuiKit port deliberately collapsed the pre-port viewer’s multi-lock hierarchy (separate queue / data / tree locks, acquired in a fixed order to avoid deadlock) into this one reentrant lock: with a single lock there is no lock-ordering rule to get wrong. The *_locked methods (_reflow_locked, _insert_children_locked, _reclassify_ancestors_locked, …) are the ones that must run while holding it.

The discipline that does still matter is never hold the lock across I/O: a worker lists a directory level (DirectoryScanner.scan_level) or byte-compares a file (DiffEngine.compare_file_content) with no lock held, then takes _lock only to merge the result — a short critical section. visible is reassigned wholesale under the lock, so draw / _draw_rows read it lock-free (an atomic snapshot). The _dirty, _scanning, and _cancel booleans are plain attributes (assignment is atomic in CPython) and need no lock.

_dirty + animation ticks

Workers mutate the tree and set self._dirty = True; a per-frame animation tick drains it on the main thread. On push the viewer registers panel.request_animation_ticks(self._tick). _tick() runs each frame on the main thread:

This keeps every DrawContext / render call on the main thread while preserving the progressive-scan UX. It is the one genuinely new-in-port decision: the pre-port viewer drove redraws directly from the worker threads through the layer stack’s dirty loop, which does not exist under PuiKit.

Threads and queues

Both queues are queue.PriorityQueue holding (-priority, seq, node) — the seq (an itertools.count) keeps items unique so two TreeNodes are never compared, and negating the priority turns the min-heap into highest-first. Priorities: _PRIO_IMMEDIATE (1000, user just expanded), _PRIO_VISIBLE (100, on-screen), _PRIO_NORMAL (10, discovered off-screen). Re-prioritisation runs on the main thread in _update_priorities (on scroll / expand / collapse) — there is no separate priority-handler thread. children_scanned guards a directory from being scanned twice; _hi_pri bounds a node to one viewport re-enqueue.

Cancellation: cancel() sets _cancel and cancels the live scanners; workers poll _scan_q.get(timeout=0.1) and check _cancel, so they wind down promptly. Tests construct with background=False, running a full recursive walk plus one-shot classification synchronously (_scan_sync) — no threads, deterministic.

Active Side and Cross-Side File Operations

The viewer tracks an active side (self.active, "left" or "right"), switched with Tab (the arrow keys expand/collapse the tree instead). The active side is shown by the accent-colored, bold pane header; _active_side_path(node) returns the focused node’s path on that side.

The file operations the pre-port design listed as “future” have shipped, driven through the same config KEY_BINDINGS and the shared FileOperationService the main file manager uses:

Each op completes via _on_op_complete, which rescans (_restart_scan) so verdicts re-evaluate (a merged file’s ! flips to = live) while _save_expansion / _restore_cursor preserve the expanded set and focus across the rebuild.

Integration Points

File Manager Integration

The Directory Diff Viewer integrates with the main file manager:

Progress System

Integrates with XeFM’s progress system:

Configuration

The viewer respects several configuration options:

Error Handling

The viewer handles various error conditions:

Testing Considerations

Key areas for testing:

Future Enhancements

Potential improvements: