It's a drain hole and belongs there. Should the rotating seal leak just a tiny amount, maybe 2 drops/day, the coolant runs on out without getting into the bearing.
Yours has been leaking enough to be noticeable, even showing a rust trail. Probably will just get worse. It's time to replace the pump before water ruins the bearing.