12-04-2015, 06:02 PM
For a long time I've been thinking about taking an important step -- a very important step for this project, I think. What I am talking about is making Hashcat and oclHashcat open source.
There have been so many discussions in the past about why Hashcat isn't open source, and I bet the same people will now ask the opposite: "Why are you going open source now?"" I will explain below, but for now, just take a minute to simply be happy about the fact (at least I hope you are!)
So, Why did I decide to go open source with the Hashcat project?
Actually, I am a big fan of open source software, and I've always held the idea of eventually going open source at some point in the future. The difficult questions were when would we be ready to do so, and when would be the best time to do it.
There are of course several additional reasons as well:
- A huge amount of hashcat/oclHashcat users are penetration testers or forensic scientists. They often have the special need of implementing their own GPU kernels. Not surprisingly, they frequently can't leak/include details about the algorithm, example hashes, or other crucial details about what should be implemented into a kernel due to restrictions placed upon them by their contract/NDA. Creating just an open interface to allow the user to easily add/modify algorithms would not be a very clever solution in this particular case, because performance is of course the thing hashcat/oclHashcat is most known for. I've already implemented most all of the widely-used generic hashing (and even some encryption) algorithms with GPU acceleration. Now they only need to be combined with each other to implement a new algorithm specific to the scheme used. When we would use an interface instead, these generic algorithms had to be reimplemented for each new scheme.
- There is a very important consideration that arises when you want to go open source: the license. My decision is to use the MIT license. This particular license allows an easy integration or packaging for the most common Linux distributions, for instance Ubuntu, but I've also planned to generate packages for The-Distribution-Which-Does-Not-Handle-OpenCL-Well (Kali) Linux which is very popular around penetration testers. The end goal is to make the installation and distribution of the hashcat project as easy as possible, most importantly for oclHashcat.
- After the switch to open source it will be much easier to integrate external libraries. Indeed, it was barely possible before due to license problems. A few crypto libraries have very restrictive licences, and some of them don't allow the integration of their code within binary files or only with very special prerequisites. At this point, hashcat/oclHashcat do not need any external libraries, but sometimes even just the parsing of the hash itself is very complicated and often even more challenging than the GPU kernel itself. GPG is a good example of this, it probably could be added easily if hashcat/oclHashcat were open source.
- Currently there is no native support for OSX. The main reason for this is that Apple does not support "offline" compiling of the kernel code. Technically, the missing piece is what AMD allows through CL_CONTEXT_OFFLINE_DEVICES_AMD in its OpenCL runtime. This would allow the compilation of GPU kernels for devices which are not currently attached to the development system. With an open source project, you can easily compile the kernels using the Apple OpenCL Runtime "just in time", also known as JIT, and hence lift that restriction. This means that support for oclHashcat on OSX would be possible for the first time.
... and why now especially?
The ultimate reason to decide to go open source was the implementation of the bitsliced DES GPU kernels. To reach maximal efficiency and performance, the salt has to be embedded within the kernel at compile time. The salt itself however depends on the given hash input. This hash of course is only known at run time, but not at compile time. This implies that the kernel needs to be compiled at run time by the system of the user. This type of compilation, with the kernel adapting according to the salt/hash, is only possible if the source code is available. Bit slicing allows to reach a much higher cracking rate of DES-based algorithms (LM, Oracle, DEScrypt, RACF). DEScrypt, for instance, which is well known on Unix-like systems, can reach a performance gain of 300-400% with the bit slice technique. These huge optimizations will be shipped with the release of oclHashcat v2.00, which will be available right after the open source announcement.
... and for those who may think I'm going to leave the project:
No way I'd do that! I'll stay here, providing the same effort as before.
Enough of me now, let the sourcecodes talk: https://github.com/hashcat/
Or simply download the new hashcat v2.00 or oclHashcat v2.00 binaries as you know them from previous versions.
There have been so many discussions in the past about why Hashcat isn't open source, and I bet the same people will now ask the opposite: "Why are you going open source now?"" I will explain below, but for now, just take a minute to simply be happy about the fact (at least I hope you are!)
So, Why did I decide to go open source with the Hashcat project?
Actually, I am a big fan of open source software, and I've always held the idea of eventually going open source at some point in the future. The difficult questions were when would we be ready to do so, and when would be the best time to do it.
There are of course several additional reasons as well:
- A huge amount of hashcat/oclHashcat users are penetration testers or forensic scientists. They often have the special need of implementing their own GPU kernels. Not surprisingly, they frequently can't leak/include details about the algorithm, example hashes, or other crucial details about what should be implemented into a kernel due to restrictions placed upon them by their contract/NDA. Creating just an open interface to allow the user to easily add/modify algorithms would not be a very clever solution in this particular case, because performance is of course the thing hashcat/oclHashcat is most known for. I've already implemented most all of the widely-used generic hashing (and even some encryption) algorithms with GPU acceleration. Now they only need to be combined with each other to implement a new algorithm specific to the scheme used. When we would use an interface instead, these generic algorithms had to be reimplemented for each new scheme.
- There is a very important consideration that arises when you want to go open source: the license. My decision is to use the MIT license. This particular license allows an easy integration or packaging for the most common Linux distributions, for instance Ubuntu, but I've also planned to generate packages for The-Distribution-Which-Does-Not-Handle-OpenCL-Well (Kali) Linux which is very popular around penetration testers. The end goal is to make the installation and distribution of the hashcat project as easy as possible, most importantly for oclHashcat.
- After the switch to open source it will be much easier to integrate external libraries. Indeed, it was barely possible before due to license problems. A few crypto libraries have very restrictive licences, and some of them don't allow the integration of their code within binary files or only with very special prerequisites. At this point, hashcat/oclHashcat do not need any external libraries, but sometimes even just the parsing of the hash itself is very complicated and often even more challenging than the GPU kernel itself. GPG is a good example of this, it probably could be added easily if hashcat/oclHashcat were open source.
- Currently there is no native support for OSX. The main reason for this is that Apple does not support "offline" compiling of the kernel code. Technically, the missing piece is what AMD allows through CL_CONTEXT_OFFLINE_DEVICES_AMD in its OpenCL runtime. This would allow the compilation of GPU kernels for devices which are not currently attached to the development system. With an open source project, you can easily compile the kernels using the Apple OpenCL Runtime "just in time", also known as JIT, and hence lift that restriction. This means that support for oclHashcat on OSX would be possible for the first time.
... and why now especially?
The ultimate reason to decide to go open source was the implementation of the bitsliced DES GPU kernels. To reach maximal efficiency and performance, the salt has to be embedded within the kernel at compile time. The salt itself however depends on the given hash input. This hash of course is only known at run time, but not at compile time. This implies that the kernel needs to be compiled at run time by the system of the user. This type of compilation, with the kernel adapting according to the salt/hash, is only possible if the source code is available. Bit slicing allows to reach a much higher cracking rate of DES-based algorithms (LM, Oracle, DEScrypt, RACF). DEScrypt, for instance, which is well known on Unix-like systems, can reach a performance gain of 300-400% with the bit slice technique. These huge optimizations will be shipped with the release of oclHashcat v2.00, which will be available right after the open source announcement.
... and for those who may think I'm going to leave the project:
No way I'd do that! I'll stay here, providing the same effort as before.
Enough of me now, let the sourcecodes talk: https://github.com/hashcat/
Or simply download the new hashcat v2.00 or oclHashcat v2.00 binaries as you know them from previous versions.