It’s true that the regulator cannot really regulate without any load, but I’d still be very careful with this. You will blow up the arduino by applying overvoltage to the 5v pins. Have you checked for shorts and tried another regulator? Maybe put a 10k resistor across the output and check again?