The problem with 3D is that the position of your eye has to match the position of the camera when recording. Your eyes can be in a LOT of spots depending on how you turn your head. Head-tracking increases this range even more. It's pretty much impossible to record something like that. You'd have to record the movie with hundreds of cameras at the same time, if not more.
Another approach would be recording with just a few cameras and using the differences between the images to recreate a 3D scene. Storing and playback of the video is still going to be tricky but I could imagine that this approach works.