Handshake timestamps do not match packets no.
I have following M1-4 EAPOL produced by aircrack, ordered by packet number according to wireshark (first column):

105686 22:43:29,145939 Key (Message 1 of 4) AP1->STA1
105692 22:43:29,145909 Key (Message 2 of 4) STA1->AP1
105694 22:43:29,145940 Key (Message 3 of 4) AP1->STA1
105696 22:43:29,145909 Key (Message 4 of 4) STA1->AP1

Timestamp is frame arrival value. Replay counter is 1/1/2/2, ANonces 1/3 are equal, no retransmission flags, no deauth, RX level is great. 

Handshake looks legit to me and yet I feel like timestamp value is more relayable than packet number so it kinda bothers me. What am I missing? Please, advice.
You're missing nothing. Aircrack is running into several issues:
and the complete handshake detection need refactoring:
Oh, I see. Thanks!

How does cap2hccapx pair those M1/2 messages with inverted timeline tho? key(STA+AP+RC)?
cap2hccapx, hcxpcaptool and hcxpcapng tool calculating EAPOL time values (count time between EAPOL messages), because you can't trust the RC in case of a packet loss (therefore we have nonce-error-corrections, too).
If this values are outside the defined EAPOL timer values, the hash will not be converted.
Additional hcxpcaptool has options to control this behavior:
--ignore-replaycount : allow not replaycount checked best handshakes
--time-error-corrections=<digit> : maximum time gap between EAPOL frames - EAPOL TIMEOUT (default: 600s)
--nonce-error-corrections=<digit> : maximum replycount/nonce gap to be converted (default: 8)
example: --nonce-error-corrections=60
convert handshakes up to a possible packetloss of 59 packets
hashcat nonce-error-corrections should be twice as much as hcxpcaptool value

hcxpcapngtool (provide new hashmode 22000) have them too:
--eapoltimeout=<digit> : set EAPOL TIMEOUT (milliseconds)
: default: 20 ms
--nonce-error-corrections=<digit> : set nonce error correction
warning: values > 0 can lead to uncrackable handshakes
: default: 0

Read more about this timers, used by APs, here: