SHA512_HMAC

[ruro]

Found the issue

seems like byte order is different for the output

my quick'n'dirty solution was to convert to uchar array after every iteration

Code:

for (int i=0; i<8; i++) {
    U[i*8]   = (ctx0.opad.h[i] & 0xFF00000000000000) >> 56;
    U[i*8+1] = (ctx0.opad.h[i] & 0x00FF000000000000) >> 48;
    U[i*8+2] = (ctx0.opad.h[i] & 0x0000FF0000000000) >> 40;
    U[i*8+3] = (ctx0.opad.h[i] & 0x000000FF00000000) >> 32;
    U[i*8+4] = (ctx0.opad.h[i] & 0x00000000FF000000) >> 24;
    U[i*8+5] = (ctx0.opad.h[i] & 0x0000000000FF0000) >> 16;
    U[i*8+6] = (ctx0.opad.h[i] & 0x000000000000FF00) >> 8;
    U[i*8+7] =  ctx0.opad.h[i] & 0x00000000000000FF;
  }

would still be interested to know if am mis-using the API, and if there are better fixes.