I had to put water In my overflow tank and It has frozen, what do i do now?

My car overheated and Lost all antifreeze/Coolant. I had to put water in it, the temp
dropped ALOT and the water froze, I did add some coolant when I got home so it isn't all
Water but I can see where there is a layer of Ice in my overflow, the car starts but it
makes a weird noise. I don't want to drive it until I figure this out.