Look at Didier Stevens' count.py described in the video at
https://videos.didierstevens.com/count-py/ and downloaded from
https://blog.didierstevens.com/my-software/#count
It was written for password lists. This is its displayed help:
Usage: count.py [options] [files ...]
count unique items
files:
wildcards are supported
@file: run command on each file listed in the text file specified
Source code put in the public domain by Didier Stevens, no Copyright
Use at your own risk
https://DidierStevens.com
Options:
--version show program's version number and exit
-h, --help show this help message and exit
-d, --descending sort descending
-k, --keys sort on keys in stead of counts
-t, --totals print totals
-n, --nocounts don't display the counters
-s, --split split lines
-l, --lowercase convert all input to lowercase before counting
-I INPUTSEPARATOR, --inputseparator=INPUTSEPARATOR
input separator (default ,)
-O OUTPUTSEPARATOR, --outputseparator=OUTPUTSEPARATOR
output separator (default ,)
-H, --header start with a header
-p, --percentage include percentage
-R, --rank include rank
-i IGNORE, --ignore=IGNORE
element to ignore
-o OUTPUT, --output=OUTPUT
output file
-b, --bothoutputs if used together with option o, output is also
displayed
-r RESUME, --resume=RESUME
resume from saved data
-e EXPORT, --export=EXPORT
export: save data
