I’m a huge fan of Scooter Software’s magnificent Beyond Compare tool, having used it for years in many different scenarios. It does a brilliant job of lining up the similarities and differences in multiple file types – especially text based file formats.
A Kubernetes container orchestration project likely involves defining multiple YAML manifests for creating different objects in the API so I often spend many happy hours comparing differences between folders full of files in different environments.
Unfortunately within Beyond Compare there isn’t any inbuilt YAML file support so it sometimes makes mistakes when aligning functionally equivalent but structurally different files.
Here’s an example:
By default red text shows differences, blue text means minor differences and hatched areas show where there is a lack of alignment possible. I have defined certain non-important grammar elements to ignore, e.g. secretName:
so these are shown in blue, but why is it not possible to line up the other elements?
In discussion with Scooter Software they explained that they would need a YAML parsing algorithm to be developed in order to pre-sort such lists into the correct layout, and this isn’t possible right now.
Alternatively they suggested creating a custom File Type which uses an external program to do the sorting work before it attempts to line up and search for differences.
With that in mind I came across the yq
project in Github by Mike Farah which provides amongst other things a sorting tool along the lines of jq
(for JSON) for YAML documents instead. The following brief instructions show how to create a custom file type in Beyond Compare to have yq
sort the keys and values in a temporary file before showing the comparison view. You can then choose to integrate and save changes on either side.
Follow preparation steps
- Download
yq_windows_amd64.exe
from the release page and rename the file tojq.exe
for ease of use - Create a folder called
YAML
in%USERPROFILE%\AppData\Roaming\Scooter Software\Beyond Compare 4\Helpers
- Copy the
jq.exe
file into the above folder - Create a new file within the same folder
yaml_sorted.bat
containing:@echo off
Helpers\YAML\yq.exe e -M "sort_keys(…)" %1 >>%2 - Open Beyond Compare and configure as follows
Configure Beyond Compare
Choose Tools, File Formats
Choose the [+] button and select Text Format
On the General tab enter a file mask to use when applying the File Format correctly, e.g. *.yaml,*.yml
Click on the Conversion tab and select External program (Unicode filenames) then enter the following path and parameters
Helpers\YAML\yaml_sorted.bat %s %t
Optionally, choose whether to Disable editing (when displaying the resulting comparison) and whether to ‘Trim trailing whitespace’ when saving – I find this helpful sometimes.
Click on the Misc tab and choose ‘Insert spaces instead of tabs’ and tab stop 8.
Click Save and Close.
Now find two YAML files to compare and double click in Beyond Compare. It will use the file type mask to apply the custom file format you just created.
In the background it will run the yq
executable twice (via the batch file), once for each input file (left and right comparison). The executable takes the input file (%1
) then sorts the keys and values alphabetically in sequence first before outputting the content to a new temporary file (%2
).
Now open a YAML file comparison and you’ll find a much improved alignment based upon first the sorting of individual keys, then subsequent scalar values within each.
e.g. yq e -M -P "keys" C:\Users\stwa\desktop\example-ingress1.yaml
Here is the resulting comparison after processing with yq
. I think you’ll agree that it’s much easier to spot the differences and also allow standardisation of your files going forward.
There’s much more that you can do with both yq and Beyond Compare to further tune the formatting order, but hopefully this gives you some sort of light at the end of the tunnel when attempting comparisons of YAML documents.
I would like to investigate whether it’s possible to define a template format in yq so that it uses a custom key alignment order so that it matches the specification used by the Kubernetes API. In addition Beyond Compare allows line weighting within the algorithm so that it can assign preferential weights to certain grammar elements, although this has not been particularly fruitful when used according to the example below.
Final thoughts
Custom file types don’t just affect comparisons – If you right click on a file and choose Open With, Text Edit you will also see the sorted version of the file – not the original.
NB – If you save the modified content it will be saved using the new sorted format.
Documentation for yq is available here: https://mikefarah.gitbook.io/yq