the object is like a mass-less plane in the liquid. We know from experience that in a still fluid no mass movement is possible. On the other hand, a brownian particle can move across the fluid in a short time.
There is a correlation time for the fluctuating force. This time becomes shorter when the absolute value of instantaneous net force becomes larger, and this causes a lot of cancellation in time.