?b means that hashcat should try all possible values a byte can have, i.e.
from 0x00 to 0xff, this means that each byte has 256 possibilities
256^5 = 256*256*256*256*256 = 1099511627776 combinations
2^40 = 2*2*2*2*....*2 = 1099511627776 combinations
As you can see, a 40 bit key is the same as 5 combinations of 8 bits (1 byte) key
Therefore ?b?b?b?b?b exhaust the whole 40-bit keyspace (and 256^5 is the same as 2^40)
BTW: I would try to troubleshoot the problem by trying to crack the example hashes of -m 9700 from https://hashcat.net/wiki/example_hashes with -m 9710 / -m 9720 . In addition to that you should create a freshly new and similar document for which you know the password and try to extract the hash with office2john and try to crack it with -m 9710
from 0x00 to 0xff, this means that each byte has 256 possibilities
256^5 = 256*256*256*256*256 = 1099511627776 combinations
2^40 = 2*2*2*2*....*2 = 1099511627776 combinations
As you can see, a 40 bit key is the same as 5 combinations of 8 bits (1 byte) key
Therefore ?b?b?b?b?b exhaust the whole 40-bit keyspace (and 256^5 is the same as 2^40)
BTW: I would try to troubleshoot the problem by trying to crack the example hashes of -m 9700 from https://hashcat.net/wiki/example_hashes with -m 9710 / -m 9720 . In addition to that you should create a freshly new and similar document for which you know the password and try to extract the hash with office2john and try to crack it with -m 9710