The UwView series — search with UwView, compare with UwDiff
Compare two files or two folders with UwDiff; open and search huge files with UwView. Both run on Windows, macOS and Linux. → UwView overview
UwDiff — compare two files, or two folders
Logs, CSV, Excel or PDF. Shift-JIS or UTF-16. One 10 GB file or a folder of 10,000 files.
UwDiff compares two files (or two folders) and shows what changed. It ships with a GUI (UwDiff) and a CLI (uvd) that use the same comparison engine, and it runs on Windows, macOS and Linux.
Search with UwView, compare with UwDiff. Opening and searching a huge file is UwView‘s job; comparing two of them is UwDiff’s. UwDiff is a separate product, not a part of UwView.
UwDiff on GitHub (source, documentation, questions)
[Download] [User manual] [Benchmarks]
Six axes
| Axis | What it does |
|---|---|
| Formats | text (logs, CSV, TSV, source), the content of 7 compression formats (gz, bz2, xz, lzma, zst, lz4, br), the text of Word (.docx), Excel (.xlsx) and PDF |
| Encodings | UTF-8, Shift-JIS (CP932), EUC-JP and UTF-16 (with or without BOM), detected automatically; old and new may differ |
| CLI and GUI | compare with the uvd CLI; add -open and the result appears in the GUI (the GUI does not compare again) |
| Three OSes | macOS (Apple Silicon, Intel), Linux (x86_64, aarch64), Windows (x64, ARM64). No separate .NET installation is needed |
| Many and huge | files of 10 GB and more, folders with thousands to tens of thousands of files. Spills to temporary files instead of running out of memory |
| Speed | key matching of two 10 GB CSV files in about 21 to 23 s, two 10 GB text files in about 21 s (conditions under “Speed” below) |
Four ways to compare
- Text: order-preserving comparison, written in the same format as
diff -u(unified diff). Applying the diff to the old file withpatchgives the new file. - Tables (key matching): CSV, TSV and Excel matched by key columns such as an ID — row order may differ. One output row per changed cell.
- Order-independent lines: ignores line order and counts which lines were added or removed.
- Folder vs folder: lists only-in-old, only-in-new, different and same, down through subfolders. Different files can be compared pair by pair.
You can also pick the changes you want from a table diff and apply only those. UwDiff writes a new file that keeps the old file’s encoding, delimiter, line endings, quoting and row order (the original files are not modified).
Speed (measured, with conditions)
Losses and limits first.
- Text where almost every line differs — the worst case (two 10 GB files, the new one shuffled every 4,096 lines) takes 110 s (the same files with lines in order take 20.7 s).
- Huge tables in shuffled row order keep rows waiting for their counterpart in memory (11.1 to 11.5 GiB for two 10 GB files). With the memory limit set to 476 MB the rows spill to temporary files and the run still completes, but takes 58 to 60 s.
- Where millions of lines are rearranged in one place, that range may be reported as a replacement (a correct diff, but not necessarily the minimal one).
- Word, Excel and PDF are compared by their extracted text (formatting, images and formulas are not compared). Legacy .doc and .xls, password-protected files and PDFs without text cannot be read.
| Comparison (both files the same size) | Time (s) | Memory |
|---|---|---|
| CSV 10 GB × 2, key matching, same row order (UTF-8 / CP932) | 21.8 / 20.5 | 4.0 to 4.3 GiB |
| CSV 10 GB × 2, key matching, shuffled rows (UTF-8 / CP932) | 23.3 / 22.3 | 11.1 to 11.5 GiB |
| Text 10 GB × 2, order-preserving comparison | 20.7 | 0.4 GiB |
| GUI: text 10 GB × 2, from Compare until the result is on screen | 20.5 | — |
| Folders of 10,000 files (2.2 GB per side, about 1% different) | 3.1 (data in the page cache) | — |
Conditions: Mac mini (Apple M4, 10 cores, 32 GB memory), macOS, data on an external SSD / uvd 1.0.0 / the same test data as tools/run_tests.sh (the 10 GB CSV has 105.34 million rows). Seconds are /usr/bin/time real. The cache was not cleared (runs directly after generating the data; two 10 GB files do not fit in memory, so this is close to cold). Every time was taken after confirming the result matches the expected answer exactly. Source: UwDiff benchmarks (docs/benchmarks.md).
CLI examples
uvd old.log new.log -o changes.diff # text (same format as diff -u)
uvd old.log.gz new.log.gz -open # compare compressed files, open the result in the GUI
uvd old.csv new.csv -key ID -o diff.csv # tables by key
uvd ledger_09.xlsx ledger_10.xlsx -key ID # Excel, sheet by sheet
uvd release_prev/ release_now/ -o list.csv # folder vs folder
uvd apply old.csv new.csv picked.csv -o out.csv # apply only the picked changes
Exit codes follow diff (0 = same, 1 = different, 2 = error).
Getting it
The current version is v1.0.0 “First Light”. Packages (.dmg for macOS, .tar.gz for Linux, .zip for Windows) are on GitHub Releases and on this site’s download page (with SHA256 values). Building from source with the .NET 10 SDK is described in the UwDiff repository on GitHub.
License
UwDiff is distributed under the UwDiff License (PolyForm Internal Use 1.0.0 plus a permission to redistribute the official binaries as packages).
- Free for internal use by individuals and companies (internal business operations).
- Redistribution, bundling into a product or service, resale and offering to third parties are not permitted; they require a separate commercial (redistribution) license.
- However, the official, unmodified binary releases may be redistributed free of charge as packages (Flathub, Snap, AUR, Linux distribution repositories, Homebrew, Scoop, winget and so on). The conditions are in the “Additional Permission” section of the LICENSE.
- UwDiff is not open-source software.
- Questions: GitHub Issues.
Sister tool
- UwView (open and search; CLI
uvf): reads and searches huge text files the moment they open → What is UwView - UwDiff (compare; CLI
uvd): this page
From the developer: a list of my apps, Kindle books and public projects is on GitHub: amru195704.