![]() |
best duplicate removal tool for wordlists - Printable Version +- hashcat Forum (https://hashcat.net/forum) +-- Forum: Deprecated; Previous versions (https://hashcat.net/forum/forum-29.html) +--- Forum: General Help (https://hashcat.net/forum/forum-8.html) +--- Thread: best duplicate removal tool for wordlists (/thread-4833.html) |
best duplicate removal tool for wordlists - turbogiant76 - 11-20-2015 Can anyone recommend one for a windows user? ULM chokes on large lists. I've also been using the tool found here, but it seems to get stuck in the same place on lists, I'm not sure if it is certain characters or what is causing it. Thank you. RE: best duplicate removal tool for wordlists - Mem5 - 11-20-2015 http://gnuwin32.sourceforge.net/packages/coreutils.htm in particular 'sort' program RE: best duplicate removal tool for wordlists - Kgx Pnqvhm - 11-21-2015 Have you tried the command line ULM tools? RE: best duplicate removal tool for wordlists - turbogiant76 - 11-21-2015 I have not. I assumed it used the same engine as the GUI. Can it handle large files? RE: best duplicate removal tool for wordlists - Kgx Pnqvhm - 11-21-2015 The command line tools are 64-bit executables, and more powerful than the GUI ULM. RE: best duplicate removal tool for wordlists - Mem5 - 11-22-2015 I had troubles using the ULM command line tools. I am using sort -u on 20 gb files without issues. |