How to Merge 2 Big Wordlists in Windows

[marcusx]

My intention is to combine 2 large wordlists, remove all duplicate entries, remove everything under 7 characters and over 40 characters and sort the list randomly, at best remove all entries that contain only letters and are not combined.

I work with Windows and have tried PowerShell and The-Distribution-Which-Does-Not-Handle-OpenCL-Well (Kali) Linux.

Via Powershell I have managed the following so far (but works only with small txt files)

Code:

Select-String -Path "D:\test\*.txt" -Pattern "^.{7,40}$" | Select -Unique -Expand Line | Sort-Object -Unique | Set-Content "D:\output.txt"

I have also tried with hashcat-utils via The-Distribution-Which-Does-Not-Handle-OpenCL-Well (Kali) linux via the command:

Code:

./mli2.bin 1.txt 2.txt > out.txt

or via powershell with:

Code:

./mli2.exe 1.txt 2.txt > out.txt

it also creates an out.txt but after 3 hours nothing happens, PowerShell aborts at some point and says that the file is already accessed.

can you help me?

[aikiuslik]

Use command line (cmd). "type wordlist1.txt wordlist2.txt | sort /u /o combinedlist.txt"

Sort command converts from utf8 to ansi tho.

You can also do "type wordlist1.txt wordlist2.txt >> wordlist3.txt" and later sort any program you want. I use notepad++ usually.

https://learn.microsoft.com/en-us/window...mands/type
https://learn.microsoft.com/en-us/window...mands/sort