The Trails application for iPhone makes GPX tracking and mapping super-easy and at a very very low cost.
This full-featured GPS application by German programmer Felix Lamouroux does many things that turnkey GPS hardware will do but at a super-bargain price of $1.99. It lets uses record their tracks (including support for stopping and starting) and then export the GPX file to a Mac or PC. In addition, Trails can import an existing file for you to use in the field or for reference.
Of course, it displays the track and your position on the Google road, satellite or topographical maps.
The author provides instructions on how to geotag photos using Trails.
Trails user Dirk Osada offers an interesting post: he tested the iPhone app against the $235 Garmin Edge 305, which was mounted on his bike. He said (in German) that depending on the route the devices were on par. However, at times, the iPhone provided better accuracy than the dedicated hardware.
For example, if you look where the green arrow is pointing in the first image in the article, the Edge futzes by 15 meters the positions when the bike was parked. Same difference with the second photo showing the part of the trip in the forest.
In the second picture one can see both devices with clear view to the sky providing possibly equal results. As soon as it goes, however, into the woods, the exactness of the Edge strongly decreases. [My editing of a machine translation (advised by my high school German classes).]