Headlights blowing out

My headlights seem to be blowing out when starting the engine with the
headlights already on. I changed out the voltage regulator and checked the
grounds to the best of my ability, installed new headlight and tried to recreate
the same scenario. The lights blew out... they shouldn't be blowing out even
if left on. I am thinking it might be the solenoid but I don't know.  Can
someone give me advice.