In many cases, the load connected to a voltage regulator is not returned to ground. It goes to an even lower voltage or perhaps even the negative power supply voltage. (Here we make the assumption of using positive voltages, when using voltage regulators with negative output voltages the reverse is true.)
Op-amps and level-shifters come to mind. In such cases, a diode (1N4001 or equivalent) connected across the output of the regulator IC usually provides sufficient protection (see Figure 1).
Polarity inversions which could occur, for example, during power on or during a short circuit could prove fatal for the regulator IC, but such a diode prevents the output of the IC going lower than ground (well, minus 0.7 V, to be accurate).
A short-circuit proof voltage regulator (such as the 78xx series) will survive such a situation without any problems. It is also possible for the input voltage of a voltage regulator to drop quicker than the output voltage—for example, when there is a protection circuit that shorts the input power supply voltage as a result of an overvoltage at the output.
If the output voltage of the regulator is more than 7 V higher than the input voltage, the emitter-base junction of the internal power transistor can break down and cause the transistor to fail.
You can use a shunt diode to prevent this condition (see Figure 2). This ensures that any higher voltage at the output of the regulator is shorted to the input.