Turns out that the Python mouse library is losing the fine-grained mouse position data, which we really need.