I have given a lot of time and thought to time travel as it is presented in movies and television and there is one thing that stands out. In all of these representations of time travel someone goes back in time, changes the timeline, then comes back to a changed future. In that way "everything has changed". I am not convinced that time travel could or would actually be that way.
Think of time travel as a spiral instead of a straight line. Rather than going back then coming forward on the same line, you go back, change something then the future you move to is the one in which belongs to the changed past rather than the one you know. In that way, nothing is truly fixed, rather it is fixed as we know it or as what is our history but not universal.
This also presents a whole different problem that is often overlooked for the sake of simplicity. If you go back in time on this time line, then come back to a different one, unless in that one you also "Disappeared" you are entering a timeline where you exist at the same time. This may not be a true paradox, and may not strain the time space continuum but what it does do, in a very real way is create 2 of you. (Again, unless the you from the new time line also went back and changed things then moved to a different strand) This may not be as big of a problem as people think, but who knows.
Think of it this way. In timeline X you were born, had all of your life experiences up to and including the movement through spacetime. The fact that you went back at all creates all kinds of problems, (See butterfly effect) When you move forward from that point, you are moving forward into a timeline we will call Y. To that end the question of becoming your own grandfather is definitely possible as you could be the grandfather of someone who you are in timeline Y having been grandfathered by your own grandfather in timeline X. There is of course another point to timeline Y you may not or never have existed at all. (Say back to the future without fixing the relationship with mom) Here is the thing, You are made up of millions and millions of genes, all of which were dependent on which sperm fertilized what egg. It is possible that in that time line you never existed, or you were a completely different genetic person based on the randomness of which sperm fertilized the egg. Even barring that, if you are the same genetic person, there is nothing that says that all of your experiences would have brought you to the exact same point or to be the same person you are today. In timeline X you get beat up in 5th grade and do nothing about it, in timeline Y you fight the bully back, thus changing a part of who you are. Or even more subtle, In one you are beat up and you are depressed, and it changes your whole outlook on life from that point forward, in the other you are beat up and you take it defiantly in the future. even this perspective shift could change the outcome of your life, so much so that when you change the past in the shared time, and you move forward and meet yourself in timeline Y you are not the same person at all.
Here is the point. Everything we do has an impact, whether we want it to or not. In Bradbury's "A Sound of Thunder" we are lead to believe that it is the dead butterfly that changes the future, but even your presence in the past would change the past. For the time you are there, even if you were able not to meet someone you shouldn't have, you are there, breathing the air, which changes the amount of carbon dioxide on the planet for that time, that may not seem important but it could be due to the larger implications.
If we are to ever move through time, we will need to do so without interacting with it to avoid any movement from one timestream to another. Want to see the declaration of independence signed? You will need to do so by observing it but not physically being in the room. The added advantage this has is that if you are moving consciousness, rather than moving a body through space time it will take less energy to do so, as well as not having any impact on the past. To make time travel work we must find a way to be non participant observers rather than going back and being active participants.