[NETFILTER]: xt_hashlimit: fix mask calculation