Scrypt + Salsa20/8 128 128

[0delay]

Hi,

I am having trouble figuring out why Scrypt is easily being cracked using John, however hashcat is not able to with both CPU and GPU (I understand that using GPU may not really work when working with Scrypt but CPU is not helping either)

Here are two examples:
< here was a hash >
< here was a hash >

(Each salt and hash lengths are equal to 43)

John identifies the hashes as Scrypt [Salsa20/8 128 128 AVX] and cracks them fine.


I have tried using the following format for hashcat (notice "=" have been added for both Salt+Hash to match the hashcat line-length):

< here was a hash >

I have also tried varying cost values of N, r, p, but still no luck.

Is it a bug or am I missing something?

[philsmd]

1. you are not allowed to post hashes (see forum rules https://hashcat.net/forum/announcement-2.html)
2. your conversion between the 2 formats used by john vs hashcat is completely wrong. hint: jtr does not use the "standard" base64 table, instead it uses another well known-one. That also means you can't just copy everything and add an equal sign ("="). You must convert it correctly

[0delay]

1. you are not allowed to post hashes (see forum rules https://hashcat.net/forum/announcement-2.html)
2. your conversion between the 2 formats used by john vs hashcat is completely wrong. hint: jtr does not use the "standard" base64 table, instead it uses another well known-one. That also means you can't just copy everything and add an equal sign ("="). You must convert it correctly

Thanks for quick reply.
1. I posted hashes just to clarify the question, my apologies, they are now removed.
2. Thanks for the hint. I tried using base64conv that comes with john to convert the base64 format using 'crypt,cryptBS,mime' and all 6 combinations since none of them worked... Still no luck.

Will try to find JtR's table in sources, let's see if that gives any results.

[philsmd]

It should be as easy as running the following script:

Code:

perl scrypt_converter.pl hashes.txt

where the file scrypt_converter.pl contains this code:

Code:

#!/usr/bin/env perl

#
# Author: philsmd
# Date: August 2017
# License: public domain
#

use strict;
use warnings;

use MIME::Base64;

#
# Constants:
#

my $ITOA64 = "./0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz";

#
# Helper functions:
#

sub convert_line
{
  my $line = shift;
  my $format = shift;

  my $out = "";

  if ($format == 1) # convert from Hashcat to jtr
  {
    my (undef, $N, $r, $p, $salt_base64, $digest_base64) = split (":", $line);

    return "" if (! defined ($salt_base64));

    return "" if (length ($salt_base64) < 1);

    return "" if (! defined ($digest_base64));

    return "" if (length ($digest_base64) < 1);

    my $salt = decode_base64 ($salt_base64);

    my $hash = decode_base64 ($digest_base64);
    $hash = unpack ("H*", $hash);
    $hash = hex_to_jtr ($hash);
    $hash = substr ($hash, 0, 43);

    # N
    my $N_conv = hex_to_jtr (decimal_to_hex (log ($N) / log (2)));
    $N_conv = substr ($N_conv, 0, 1);

    # r
    my $r_hex  = decimal_to_hex ($r);
    my $r_conv = uint32_to_jtr_conv ($r_hex); # pad to at least "30 bit"
    $r_conv = hex_to_jtr ($r_conv);
    $r_conv = substr ($r_conv, 0, 5); # truncate to 30 bit

    # p
    my $p_hex  = decimal_to_hex ($p);
    my $p_conv = uint32_to_jtr_conv ($p_hex);
    $p_conv = hex_to_jtr ($p_conv);
    $p_conv = substr ($p_conv, 0, 5);

    $out = sprintf ("\$7\$%s%s%s%s\$%s", $N_conv, $r_conv, $p_conv, $salt, $hash);
  }
  elsif ($format == 2) # convert from Jtr to Hashcat
  {
    my $index_last = rindex ($line, "\$");
    my $hash = substr ($line, $index_last + 1);

    my $settings = substr ($line, 3, 11);

    my $index = index ($line, "\$", 3);
    my $salt = substr ($line, 14, $index_last - 14);

    my $N_conv = substr ($settings, 0, 1);
    my $r_conv = substr ($settings, 1, 5);
    my $p_conv = substr ($settings, 6, 5);

    # N
    my $N = jtr_to_hex ($N_conv);
    $N = 1 << uint32_from_jtr_conv ($N);

    # r
    my $r = jtr_to_hex ($r_conv);
    $r = uint32_from_jtr_conv ($r);

    # p
    my $p = jtr_to_hex ($p_conv);
    $p = uint32_from_jtr_conv ($p);

    # convert hash
    my $hash_conv = jtr_to_hex ($hash);
    $hash_conv = substr ($hash_conv, 0, 64);
    my $hash_base64 = encode_base64 (pack ("H*", $hash_conv));
    $hash_base64 =~ s/[\r\n]//g;

    #convert salt
    my $salt_base64 = encode_base64 ($salt);
    $salt_base64 =~ s/[\r\n]//g;

    $out = sprintf ("SCRYPT:%i:%i:%i:%s:%s", $N, $r, $p, $salt_base64, $hash_base64);
  }

  return $out . "\n";
}

# Return codes - Scrypt formats
# 1 = Hashcat - Example: SCRYPT:16384:8:1:U29kaXVtQ2hsb3JpZGU=:cCO9yzr9c0hGHAbNgf046/2o+7qQT44+qbVD9lRdofI=
# 2 = Jtr     - Example: $7$C6..../....SodiumChloride$kBGj9fHznVYFQMEn/qDCfrDevf9YDtcDdKvEqHJLV8D

sub detect_format
{
  my $line = shift ;

  my $ret = 0;

  if (substr ($line, 0, 3) eq "\$7\$")
  {
    $ret = 2;
  }
  elsif (substr ($line, 0, 7) eq "SCRYPT:")
  {
    $ret = 1
  }

  return $ret;
}

sub jtr_to_hex
{
  my $str = shift;

  my $out = "";

  my $len = length ($str);

  for (my $i = 0; $i < $len; $i += 4)
  {
    my @nums = ();

    for (my $j = $i, my $k = 0; $k < 4; $j++, $k++)
    {
      if ($j > $len) # exception
      {
        $nums[$k] = 0;
      }
      else
      {
        $nums[$k] = index ($ITOA64, substr ($str, $j, 1));
      }
    }

    my $y = $nums[0] | ($nums[1] << 6) | ($nums[2] << 12) | ($nums[3] << 18);
    my $y1 =  $y        & 0xff;
    my $y2 = ($y >>  8) & 0xff;
    my $y3 = ($y >> 16) & 0xff;

    $out .= sprintf ("%02x%02x%02x", $y1, $y2, $y3)
  }

  return $out;
}

sub hex_to_jtr
{
  my $hex = shift;

  my $out = "";

  my $len = length ($hex);

  for (my $i = 0; $i < $len; $i += 3 * 2)
  {
    my @nums = ();

    for (my $j = $i, my $k = 0; $k < 3; $j += 2, $k++)
    {
      if ($j > $len) # exception
      {
        $nums[$k] = 0;
      }
      else
      {
        $nums[$k] = hex (substr ($hex, $j, 2));
      }
    }

    my $y = $nums[0] | ($nums[1] << 8) | ($nums[2] << 16);
    my $x1 =  $y        & 0x3f;
    my $x2 = ($y >>  6) & 0x3f;
    my $x3 = ($y >> 12) & 0x3f;
    my $x4 = ($y >> 18) & 0x3f;

    $out .= substr ($ITOA64, $x1, 1);
    $out .= substr ($ITOA64, $x2, 1);
    $out .= substr ($ITOA64, $x3, 1);
    $out .= substr ($ITOA64, $x4, 1);
  }

  return $out;
}

sub decimal_to_hex
{
  my $num = shift;

  $num = sprintf ("%x", $num);

  $num = "0" . $num unless ((length ($num) % 2) == 0);

  return $num;
}

# 1. left-to-right to right-to-left
# 2. pad to at least 30 bit (~ 4 x 8 bit blocks)

sub uint32_to_jtr_conv
{
  my $hex = shift;

  my $out = "";

  my $len = length ($hex);

  for (my $i = 0; $i < $len; $i += 2)
  {
    $out .= substr ($hex, $len - $i - 2, 2);
  }

  for (my $i = $len / 2; $i < 4; $i++)
  {
    $out .= "00";
  }

  return $out;
}

# 1. revert: right-to-left -> left-to-right
# (while skipping 00 at beginning (== end))

sub uint32_from_jtr_conv
{
  my $hex = shift;

  my $out = "";

  my $len = length ($hex);

  my $is_beginning = 1;

  for (my $i = 0; $i < $len; $i += 2)
  {
    my $num = substr ($hex, $len - $i - 2, 2);

    next if ($is_beginning && ($num eq "00"));

    $is_beginning = 0;

    $out .= $num;
  }

  return hex ($out);
}

#
# Start
#

my $INPUT = "-";

if ($ARGV[0])
{
  $INPUT = $ARGV[0];
}

my $detected_format = 0;

open (INFILE, "<" . $INPUT) or die ("ERROR: could not open input file");

while (my $line = <INFILE>)
{
  chomp $line;

  if ($detected_format == 0)
  {
    $detected_format = detect_format ($line);

    if ($detected_format == 0)
    {
      print STDERR "ERROR: could not detect hash format. EXIT\n";
      last;
    }
  }

  print STDOUT convert_line ($line, $detected_format);
}

close (INFILE);

[atom]

Also make sure to use CPU on hashcat as well, not GPU. Scrypt is an anti-GPU KDF. You can do that by installing Intel OpenCL runtime and use the -D 1 option.

[0delay]

It should be as easy as running the following script:

Code:

perl scrypt_converter.pl hashes.txt

where the file scrypt_converter.pl contains this code:

Code:

#!/usr/bin/env perl

#
# Author: philsmd
# Date: August 2017
# License: public domain
#

use strict;
use warnings;

use MIME::Base64;

#
# Constants:
#

my $ITOA64 = "./0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz";

#
# Helper functions:
#

sub convert_line
{
 my $line = shift;
 my $format = shift;

 my $out = "";

 if ($format == 1) # convert from Hashcat to jtr
 {
   my (undef, $N, $r, $p, $salt_base64, $digest_base64) = split (":", $line);

   return "" if (! defined ($salt_base64));

   return "" if (length ($salt_base64) < 1);

   return "" if (! defined ($digest_base64));

   return "" if (length ($digest_base64) < 1);

   my $salt = decode_base64 ($salt_base64);

   my $hash = decode_base64 ($digest_base64);
   $hash = unpack ("H*", $hash);
   $hash = hex_to_jtr ($hash);
   $hash = substr ($hash, 0, 43);

   # N
   my $N_conv = hex_to_jtr (decimal_to_hex (log ($N) / log (2)));
   $N_conv = substr ($N_conv, 0, 1);

   # r
   my $r_hex  = decimal_to_hex ($r);
   my $r_conv = uint32_to_jtr_conv ($r_hex); # pad to at least "30 bit"
   $r_conv = hex_to_jtr ($r_conv);
   $r_conv = substr ($r_conv, 0, 5); # truncate to 30 bit

   # p
   my $p_hex  = decimal_to_hex ($p);
   my $p_conv = uint32_to_jtr_conv ($p_hex);
   $p_conv = hex_to_jtr ($p_conv);
   $p_conv = substr ($p_conv, 0, 5);

   $out = sprintf ("\$7\$%s%s%s%s\$%s", $N_conv, $r_conv, $p_conv, $salt, $hash);
 }
 elsif ($format == 2) # convert from Jtr to Hashcat
 {
   my $index_last = rindex ($line, "\$");
   my $hash = substr ($line, $index_last + 1);

   my $settings = substr ($line, 3, 11);

   my $index = index ($line, "\$", 3);
   my $salt = substr ($line, 14, $index_last - 14);

   my $N_conv = substr ($settings, 0, 1);
   my $r_conv = substr ($settings, 1, 5);
   my $p_conv = substr ($settings, 6, 5);

   # N
   my $N = jtr_to_hex ($N_conv);
   $N = 1 << uint32_from_jtr_conv ($N);

   # r
   my $r = jtr_to_hex ($r_conv);
   $r = uint32_from_jtr_conv ($r);

   # p
   my $p = jtr_to_hex ($p_conv);
   $p = uint32_from_jtr_conv ($p);

   # convert hash
   my $hash_conv = jtr_to_hex ($hash);
   $hash_conv = substr ($hash_conv, 0, 64);
   my $hash_base64 = encode_base64 (pack ("H*", $hash_conv));
   $hash_base64 =~ s/[\r\n]//g;

   #convert salt
   my $salt_base64 = encode_base64 ($salt);
   $salt_base64 =~ s/[\r\n]//g;

   $out = sprintf ("SCRYPT:%i:%i:%i:%s:%s", $N, $r, $p, $salt_base64, $hash_base64);
 }

 return $out . "\n";
}

# Return codes - Scrypt formats
# 1 = Hashcat - Example: SCRYPT:16384:8:1:U29kaXVtQ2hsb3JpZGU=:cCO9yzr9c0hGHAbNgf046/2o+7qQT44+qbVD9lRdofI=
# 2 = Jtr     - Example: $7$C6..../....SodiumChloride$kBGj9fHznVYFQMEn/qDCfrDevf9YDtcDdKvEqHJLV8D

sub detect_format
{
 my $line = shift ;

 my $ret = 0;

 if (substr ($line, 0, 3) eq "\$7\$")
 {
   $ret = 2;
 }
 elsif (substr ($line, 0, 7) eq "SCRYPT:")
 {
   $ret = 1
 }

 return $ret;
}

sub jtr_to_hex
{
 my $str = shift;

 my $out = "";

 my $len = length ($str);

 for (my $i = 0; $i < $len; $i += 4)
 {
   my @nums = ();

   for (my $j = $i, my $k = 0; $k < 4; $j++, $k++)
   {
     if ($j > $len) # exception
     {
       $nums[$k] = 0;
     }
     else
     {
       $nums[$k] = index ($ITOA64, substr ($str, $j, 1));
     }
   }

   my $y = $nums[0] | ($nums[1] << 6) | ($nums[2] << 12) | ($nums[3] << 18);
   my $y1 =  $y        & 0xff;
   my $y2 = ($y >>  8) & 0xff;
   my $y3 = ($y >> 16) & 0xff;

   $out .= sprintf ("%02x%02x%02x", $y1, $y2, $y3)
 }

 return $out;
}

sub hex_to_jtr
{
 my $hex = shift;

 my $out = "";

 my $len = length ($hex);

 for (my $i = 0; $i < $len; $i += 3 * 2)
 {
   my @nums = ();

   for (my $j = $i, my $k = 0; $k < 3; $j += 2, $k++)
   {
     if ($j > $len) # exception
     {
       $nums[$k] = 0;
     }
     else
     {
       $nums[$k] = hex (substr ($hex, $j, 2));
     }
   }

   my $y = $nums[0] | ($nums[1] << 8) | ($nums[2] << 16);
   my $x1 =  $y        & 0x3f;
   my $x2 = ($y >>  6) & 0x3f;
   my $x3 = ($y >> 12) & 0x3f;
   my $x4 = ($y >> 18) & 0x3f;

   $out .= substr ($ITOA64, $x1, 1);
   $out .= substr ($ITOA64, $x2, 1);
   $out .= substr ($ITOA64, $x3, 1);
   $out .= substr ($ITOA64, $x4, 1);
 }

 return $out;
}

sub decimal_to_hex
{
 my $num = shift;

 $num = sprintf ("%x", $num);

 $num = "0" . $num unless ((length ($num) % 2) == 0);

 return $num;
}

# 1. left-to-right to right-to-left
# 2. pad to at least 30 bit (~ 4 x 8 bit blocks)

sub uint32_to_jtr_conv
{
 my $hex = shift;

 my $out = "";

 my $len = length ($hex);

 for (my $i = 0; $i < $len; $i += 2)
 {
   $out .= substr ($hex, $len - $i - 2, 2);
 }

 for (my $i = $len / 2; $i < 4; $i++)
 {
   $out .= "00";
 }

 return $out;
}

# 1. revert: right-to-left -> left-to-right
# (while skipping 00 at beginning (== end))

sub uint32_from_jtr_conv
{
 my $hex = shift;

 my $out = "";

 my $len = length ($hex);

 my $is_beginning = 1;

 for (my $i = 0; $i < $len; $i += 2)
 {
   my $num = substr ($hex, $len - $i - 2, 2);

   next if ($is_beginning && ($num eq "00"));

   $is_beginning = 0;

   $out .= $num;
 }

 return hex ($out);
}

#
# Start
#

my $INPUT = "-";

if ($ARGV[0])
{
 $INPUT = $ARGV[0];
}

my $detected_format = 0;

open (INFILE, "<" . $INPUT) or die ("ERROR: could not open input file");

while (my $line = <INFILE>)
{
 chomp $line;

 if ($detected_format == 0)
 {
   $detected_format = detect_format ($line);

   if ($detected_format == 0)
   {
     print STDERR "ERROR: could not detect hash format. EXIT\n";
     last;
   }
 }

 print STDOUT convert_line ($line, $detected_format);
}

close (INFILE);

Hey, philsmd!
Thanks a lot, this really helped me to see that my code wasn't wrong =) (I was comparing my results with john's converter).
However, hashcat is still unable to crack any Scrypt. I am using CPU with -D 1 option enabled. It also wasn't able to crack the example hash in your code (pleaseletmein).

[philsmd]

I just did several test with scrypt (even the "pleaseletmein" test) and I found no problem cracking scrypt with hashcat.

Could you please be more specific. What do you mean by "still unable to crack any Scrypt"?
- does hashcat accept/load your scrypt hash correctly without a warning
- does it run without errors
- does it end with Status...........: Exhausted
etc

just saying that it is "unable to crack" doesn't really help to troubleshoot the problem. you must be much more specific.
As far as I understood the problem is/was that you used a different format ($7$...) to run the hashes, which hashcat does not accept. Maybe this was just a misunderstanding and you threw that format-conversion problem in front of the real problem you have, i.e. that scrypt cracking does not work at all for you. To avoid confusion in the future always try the example hashes (https://hashcat.net/wiki/example_hashes) first (and avoid mixing it up with other problems).

There could be many reasons why it doesn't crack for you.

You could for instance try to use a different version of hashcat (do you even use the latest release version from https://hashcat.net/hashcat/ ?). You could try the beta version from https://hashcat.net/beta/.
We also need to understand the real problem. Which you didn't explain at all, but my guess is that you have a problem with false negatives (status exhausted even if it should crack).
It could also have to do with your command line. Maybe you are using hashcat incorrectly, e.g. wrong command line arguments provided (but as far as I understood it works for you with other hash types).

[0delay]

I just did several test with scrypt (even the "pleaseletmein" test) and I found no problem cracking scrypt with hashcat.

Could you please be more specific. What do you mean by "still unable to crack any Scrypt"?
- does hashcat accept/load your scrypt hash correctly without a warning
- does it run without errors
- does it end with  Status...........: Exhausted
etc

just saying that it is "unable to crack" doesn't really help to troubleshoot the problem. you must be much more specific.
As far as I understood the problem is/was that you used a different format ($7$...) to run the hashes, which hashcat does not accept. Maybe this was just a misunderstanding and you threw that format-conversion problem in front of the real problem you have, i.e. that scrypt cracking does not work at all for you. To avoid confusion in the future always try the example hashes (https://hashcat.net/wiki/example_hashes) first (and avoid mixing it up with other problems).

There could be many reasons why it doesn't crack for you.

You could for instance try to use a different version of hashcat (do you even use the latest release version from https://hashcat.net/hashcat/ ?). You could try the beta version from https://hashcat.net/beta/.
We also need to understand the real problem. Which you didn't explain at all, but my guess is that you have a problem with false negatives (status exhausted even if it should crack).
It could also have to do with your command line. Maybe you are using hashcat incorrectly, e.g. wrong command line arguments provided (but as far as I understood it works for you with other hash types).

Right, hashcat works fine with any other hash. (I guess I didn't realize that my question was vague)

For Scrypt:
It loads hashes fine without any errors/warnings.
It ends with Exhausted recovering 0/N hashes.
I give it a file with hashes and a dictionary where the correct results are listed plus some other random words.

The problem: It doesn't crack even when it should crack (just like you said)

Using hashcat versions v3.5.0, v3.6.0, and v3.6.0-413-g0783289 (beta)

Command line: hashcat -m8900 -a0 -w3 -D1 test.scrypt dictionary

Results:

Code:

hashcat (v3.6.0) starting...

OpenCL Platform #1: Intel(R) Corporation
========================================
* Device #1: Intel(R) Core(TM) i7-6700HQ CPU @ 2.60GHz, 3976/15907 MB allocatable, 8MCU

OpenCL Platform #2: NVIDIA Corporation
======================================
* Device #2: GeForce GTX 960M, skipped.

Counting lines in ./test.scrypt...
Counted lines in ./test.scrypt...
Parsing Hashes: 1/6 (16.67%)...
Parsing Hashes: 4/6 (66.67%)...
Sorting hashes...
Sorted hashes...
Removing duplicate hashes...
Removed duplicate hashes...
Sorting salts...
Sorted salts...
Comparing hashes with potfile entries...
Compared hashes with potfile entries...
Generating bitmap tables...
Generated bitmap tables...
Hashes: 4 digests; 4 unique digests, 4 unique salts
Bitmaps: 16 bits, 65536 entries, 0x0000ffff mask, 262144 bytes, 5/13 rotates
Rules: 1

Applicable optimizers:
* Zero-Byte

Watchdog: Temperature abort trigger set to 90c
Watchdog: Temperature retain trigger disabled.

Initializing device kernels and memory...
Initialized device kernels and memory...
Checking for weak hashes...
Checked for weak hashes...
Dictionary cache hit:
* Filename..: ./dictionary
* Passwords.: 9
* Bytes.....: 88
* Keyspace..: 9

[s]tatus [p]ause [r]esume [b]ypass [c]heckpoint [q]uit =>
                                                          
Approaching final keyspace - workload adjusted.
[s]tatus [p]ause [r]esume [b]ypass [c]heckpoint [q]uit =>
                                                          
Session..........: hashcat
Status...........: Exhausted
Hash.Type........: scrypt
Hash.Target......: ./test.scrypt
Time.Started.....: Wed Aug 23 09:07:14 2017 (1 sec)
Time.Estimated...: Wed Aug 23 09:07:15 2017 (0 secs)
Guess.Base.......: File (./dictionary)
Guess.Queue......: 1/1 (100.00%)
Speed.Dev.#1.....:        0 H/s (32.15ms)
Recovered........: 0/4 (0.00%) Digests, 0/4 (0.00%) Salts
Progress.........: 36/36 (100.00%)
Rejected.........: 0/36 (0.00%)
Restore.Point....: 9/9 (100.00%)
Candidates.#1....: pleaseletmein -> pleaseletmein
HWMon.Dev.#1.....: N/A

Started: Wed Aug 23 09:07:13 2017
Stopped: Wed Aug 23 09:07:16 2017