Sort and de-dupe without merging together?

[DDNK]

Is there anyway to "sort and de-dupe" the entirety of my Wordlist folder without merging them together?

It's hard to explain what I exactly mean, hopefully someone understands, lol.

[rsberzerker]

Hashcat utilities rli and rli2 might do what you want, although it will take several passes if you have multiple dictionary files.

[atom]

Yes, rli2 is the way to go.

Make sure to

LC_ALL=C sort -u

before