12-16-2021, 08:57 PM
Compress into archive with 7z/zip/rar. Then the volume will decrease. But it is not a usable file then.
Other option is to keep this file unchanged in gzip format, where you will have compressed file but usable through streams in Linux.
To remove duplicates you would need to sort the file and then uniq it. There are Linux commands "sort" and "uniq". Uniq works only if two corresponding lines are the same, that's why the file should be sorted.
Other option is to keep this file unchanged in gzip format, where you will have compressed file but usable through streams in Linux.
To remove duplicates you would need to sort the file and then uniq it. There are Linux commands "sort" and "uniq". Uniq works only if two corresponding lines are the same, that's why the file should be sorted.