How do I get to Relic Castle on Pokemon White?

On Pokemon White it says to get to Relic Castle, I have to go all the way back to Route 4. I am on Route 8, so am I supposed to literally walk all the way back to Route 4 to get there? That's a really long way.