I've pulled a lot of trailers and, knock on wood, have never had this happen thankfully. I always use a padlock to ensure that the mechanism doesn't come loose when hitting bumps etc. Always make sure to keep slightly more weight in front of the trailer axle to keep pressure down on the trailer tongue/hitch. It sounds to me like either the ball is too small, or the clamp inside the hitch is warn/broken/warped.