+- hashcat Forum (https://hashcat.net/forum)
+-- Forum: Support (https://hashcat.net/forum/forum-3.html)
+--- Forum: hashcat-utils, maskprocessor, statsprocessor, md5stress, wikistrip (https://hashcat.net/forum/forum-28.html)
+--- Thread: how do i remove all duplicate words from a txt file? (/thread-6143.html)
how do i remove all duplicate words from a txt file? - cityscab - 12-22-2016
I found a script to do this online in my browser but was wondering if hashcat had a build in function for it
http://www.tracemyip.org/tools/remove-duplicate-words-in-text/
RE: how do i remove all duplicate words from a txt file? - Mem5 - 12-22-2016
You just want to remove duplicates ?!
Code:
sort file.txt | uniqRE: how do i remove all duplicate words from a txt file? - Xanadrel - 12-22-2016
or just sort -u
RE: how do i remove all duplicate words from a txt file? - kiara - 12-23-2016
or
https://forum.hashkiller.co.uk/topic-view.aspx?t=5512&p=0
RE: how do i remove all duplicate words from a txt file? - epixoip - 12-24-2016
''LC_ALL=C sort -u''
You can also ''man sort'' and look at the --parallel and -S switches to improve performance.
RE: how do i remove all duplicate words from a txt file? - cityscab - 12-24-2016
ah, i see, its a unix command.
you had me thinking you were a wizard there for a second.
RE: how do i remove all duplicate words from a txt file? - cityscab - 12-24-2016
This was driving me nuts. I came back to this post later and installed cygwin and finally got it to work. Funny how sometimes you get a wild hair up your ass about something and can't let it go.
RE: how do i remove all duplicate words from a txt file? - Mem5 - 12-24-2016
You can have sort & uniq & other *nix tools under windows without the cygwin sh** : http://gnuwin32.sourceforge.net
RE: how do i remove all duplicate words from a txt file? - epixoip - 12-25-2016
You can also use WSL and run Ubuntu under Windows 10
RE: how do i remove all duplicate words from a txt file? - atom - 12-25-2016
I'd go Msys2