I can best explain this through an every day example. Imagine you have an mp3 player with shuffle mode on. When you choose "Forward", you have no idea which song will play next. Likewise, if you were to choose "Previous", then too, you will have no idea which song will play.
Switch off shuffle mode, then you get total predictability on which song will play and in which sequence. No matter which way you navigate, you will always have the songs playing in the exact same sequence.
Now assuming your mp3 player is not made that it can work in shuffle-on mode in one direction and shuffle-off mode in the other, either you will have total predictability or total uncertainty.
So if we expect to go back in time and find things exactly as we saw them while there, then we are in shuffle-off mode and hence we should have a completely predictable future, i.e. some one some where has already written a complete script and we just enact it without an iota of control.
But, if anything we know for certain it is that uncertainty is the hallmark of our Universe. We have no idea how life will turn out, much less what the next moment holds for us. There is no shuffle-off mode. So even if we were able to travel back in time, whatever that means, there is no way we can expect things to be what they were when we experienced them.
In a shuffle-on mode, "Forward" and "Previous" make no sense, no matter which "direction" you choose, you end up with a song that you cannot predict. "Direction" ceases to exist.
Ergo, time travel to the past, even if possible, makes no sense. And even if you were able to travel in any one "direction", there is no way you could trace back to the exact time and place you left from.
In conclusion, we just travel; there is neither a forward or previous, no future or past - we just perceive it that way. The songs keep playing and you have no idea where that song is placed in your playlist. All we can do is hope that our playlist has songs we like.
