The Directory Diff Viewer is a sophisticated component that provides recursive directory comparison with a tree-structured display. It enables users to compare two directory trees side-by-side, identifying differences in files, directories, and their contents.
flowchart TB
View["DirectoryDiffView (Widget) — main thread<br/>dual-pane tree · navigation · events · render"]
subgraph WORK["Background threads (daemon)"]
direction LR
Coord["scan coordinator"]
ScanW["scanner worker"]
CmpW["comparator worker"]
Coord --> ScanW
Coord --> CmpW
end
Scanner["DirectoryScanner<br/>lists dir → {relative_path: FileInfo}"]
Diff["DiffEngine<br/>builds + classifies the unified tree"]
subgraph DATA["Data model"]
direction LR
Tree["TreeNode<br/>children · left/right FileInfo · difference_type"]
FInfo["FileInfo<br/>path · size · mtime · is_dir · accessible"]
DType["DifferenceType (enum)<br/>IDENTICAL · ONLY_LEFT · ONLY_RIGHT<br/>CONTENT_DIFFERENT · CONTAINS_DIFFERENCE · PENDING"]
end
View -->|"enqueue visible/expanded first (priority queues)"| Coord
ScanW --> Scanner
CmpW --> Diff
Scanner --> FInfo
Diff --> Tree
Tree --> DType
Diff -->|"update tree under RLock, repaint"| View
classDef main fill:#1a5490,stroke:#7fb3d5,color:#fff;
classDef work fill:#8b2e24,stroke:#e0897f,color:#fff;
classDef engine fill:#1e7e34,stroke:#7fd39b,color:#fff;
classDef data fill:#9a6308,stroke:#e0b45f,color:#fff;
class View main;
class Coord,ScanW,CmpW work;
class Scanner,Diff engine;
class Tree,FInfo,DType data;
DifferenceType Enum
IDENTICAL: Files/directories are the sameONLY_LEFT: Item exists only in the left directoryONLY_RIGHT: Item exists only in the right directoryCONTENT_DIFFERENT: Two-sided files whose content differsCONTAINS_DIFFERENCE: A directory that contains differences below itPENDING: Not yet scanned / compared (progressive)Priority scheduling
(-priority, seq, node), so visible and expanded directories are processed before deep background items. Priority is an integer, not an enum.FileInfo Class
DirectoryDiffView (Widget)
Widget) for the dual-pane comparison treeDirectoryScanner
{relative_path: FileInfo}cancel()DiffEngine
compare_content=False, two-sided files stay PENDING so the tree structure appears before any file is readTreeNode
FileInfo, and difference_typeThe viewer implements progressive scanning to provide immediate feedback:
File comparison runs in background threads:
Users can navigate the comparison tree:
# Pseudo-code for progressive scanning
def scan_directory(path, priority):
# Scan immediate children first
entries = list_directory(path)
# Yield results immediately for display
for entry in entries:
yield entry
# Queue subdirectories for later scanning
for subdir in subdirectories:
queue_scan(subdir, lower_priority)
The viewer uses different comparison strategies based on file type:
Scanning and comparison run off the main thread, but all rendering stays on the main thread — PuiKit has no cross-thread draw. Two mechanisms keep that safe.
A single threading.RLock (self._lock) guards every tree mutation and the
reflow that rebuilds the flattened visible list. The PuiKit port deliberately
collapsed the pre-port viewer’s multi-lock hierarchy (separate queue / data /
tree locks, acquired in a fixed order to avoid deadlock) into this one reentrant
lock: with a single lock there is no lock-ordering rule to get wrong. The
*_locked methods (_reflow_locked, _insert_children_locked,
_reclassify_ancestors_locked, …) are the ones that must run while holding it.
The discipline that does still matter is never hold the lock across I/O: a
worker lists a directory level (DirectoryScanner.scan_level) or byte-compares a
file (DiffEngine.compare_file_content) with no lock held, then takes _lock
only to merge the result — a short critical section. visible is reassigned
wholesale under the lock, so draw / _draw_rows read it lock-free (an atomic
snapshot). The _dirty, _scanning, and _cancel booleans are plain attributes
(assignment is atomic in CPython) and need no lock.
_dirty + animation ticksWorkers mutate the tree and set self._dirty = True; a per-frame animation tick
drains it on the main thread. On push the viewer registers
panel.request_animation_ticks(self._tick). _tick() runs each frame on the
main thread:
self._cancel, return False (unregisters the tick — no busy spin after
close);self._dirty, clear it and call self._panel.render();self._scanning or self._dirty — keep ticking while a scan is live or a
repaint is pending, then stop once idle.This keeps every DrawContext / render call on the main thread while preserving
the progressive-scan UX. It is the one genuinely new-in-port decision: the
pre-port viewer drove redraws directly from the worker threads through the layer
stack’s dirty loop, which does not exist under PuiKit.
_scan_coordinator, the one joinable thread): scans both
roots’ top level, starts the two workers, waits for both queues to drain, then
finalises._scanner_worker): pulls directories off _scan_q, lists
one level each (_scan_node), enqueues child directories (breadth-first) and
two-sided files for comparison._comparator_worker): resolves two-sided files’ content
verdicts off _cmp_q (_compare_node), decoupled so neither queue blocks the
other.Both queues are queue.PriorityQueue holding (-priority, seq, node) — the
seq (an itertools.count) keeps items unique so two TreeNodes are never
compared, and negating the priority turns the min-heap into highest-first.
Priorities: _PRIO_IMMEDIATE (1000, user just expanded), _PRIO_VISIBLE (100,
on-screen), _PRIO_NORMAL (10, discovered off-screen). Re-prioritisation runs on
the main thread in _update_priorities (on scroll / expand / collapse) —
there is no separate priority-handler thread. children_scanned guards a
directory from being scanned twice; _hi_pri bounds a node to one viewport
re-enqueue.
Cancellation: cancel() sets _cancel and cancels the live scanners; workers
poll _scan_q.get(timeout=0.1) and check _cancel, so they wind down promptly.
Tests construct with background=False, running a full recursive walk plus
one-shot classification synchronously (_scan_sync) — no threads, deterministic.
The flag alone is only read between levels, and one level is an iterdir()
plus a stat() per entry — the slow part on a remote pane or a woken disk. So
scan_level() polls _cancel between entries too, and each run gets a single
DirectoryScanner (its lister), registered in _scanners by
_start_coordinator and passed down to _seed_root / _scan_node the same way
the queues are. Before #438 neither half existed: the background path built a
throwaway scanner per call, so cancel()’s loop had nothing to cancel, and
scan_level() had nothing to check — a cancelled viewer kept listing to the end
(400 entries and 4.4s past the cancel, in a 10ms-per-entry repro; 41 and 0.00s
after). iterdir() is a single call and stays uninterruptible.
A cancelled scan_level() returns a partial level. Nothing may insert that:
_scan_node drops it on the _stale(run) check that follows, which is the only
reason returning early is safe. _lazy_scan still builds its own scanner — it
runs on the UI thread after the background pass is done, so there is no one to
cancel it.
Scan runs. _cancel alone cannot say whether a thread still owns the view,
because a rescan (_restart_scan) clears it again for its own run while the
previous run’s threads are still finishing — and a byte-compare part-way through
a large file cannot be interrupted at all. So each run carries a number, _run,
bumped by _restart_scan; the coordinator and both workers take it plus their
own queue objects as arguments, and ask _stale(run) (cancelled or
superseded) before touching the tree, the counters or the scan flags. Two
consequences worth keeping:
get and its task_done always name the same queue. Reading
self._cmp_q for both let a rescan land in between, so task_done decremented
the queue the rescan had just installed — ValueError: task_done() called too
many times on the worker thread, and a join that returned before the new run
had actually finished (issue #429)._restart_scan retires the old run instead of joining it. It runs on the UI
thread, and the joins it used to do (1s each for the coordinator and both
workers) froze the view for as long as the in-flight compare took.A retired worker drains its own queue without doing the work, so its
coordinator’s join returns and that thread exits too; nothing leaks.
The viewer tracks an active side (self.active, "left" or "right"),
switched with Tab (the arrow keys expand/collapse the tree instead). The
active side is shown by the accent-colored, bold pane header;
_active_side_path(node) returns the focused node’s path on that side.
The file operations the pre-port design listed as “future” have shipped, driven
through the same config KEY_BINDINGS and the shared FileOperationService the
main file manager uses:
copy_files, default C) / Move (move_files, default M) —
_copy_focused / _move_focused transfer the focused node from the active
side to its mirrored location on the opposite side. _mirror_dest_dir maps
sub/a.txt on the active side to sub/ under the opposite root, keeping the
two trees aligned.delete_files, default K / Del) — _delete_focused removes
the focused node from the active side.edit_file, default E) — _edit_merge launches the configured
TEXT_DIFF tool (e.g. vimdiff, code --diff) on a two-sided local file via
the backend’s suspend/resume.view_file, Enter) — _open_file_diff opens the per-file diff for
a two-sided differing file (reusing xefm.diff_viewer.show_diff_viewer).Each op completes via _on_op_complete, which rescans (_restart_scan) so
verdicts re-evaluate (a merged file’s ! flips to = live) while
_save_expansion / _restore_cursor preserve the expanded set and focus across
the rebuild.
The Directory Diff Viewer integrates with the main file manager:
Integrates with XeFM’s progress system:
The viewer respects several configuration options:
The viewer handles various error conditions:
Key areas for testing:
ProgressiveSearchDialog)Potential improvements: