Colorizing the Prokudin-Gorskii photo collection
The Prokudin-Gorskii photo collection is a series of photographs from an era before color photography had been invented. Prokudin-Gorskii, in all his insight however knew there might be a time when color photos were the norm, so he set off to photograph his Russian homeland in full color. Armed with red, green, and blue colored filters, he systematically took three shots per scene (each time covering his lens with a different filter). The final photographic plates are formed with the three images stacked vertically.
I attempted to programmatically separate and compile those images.
Armed with the Python Imaging Library and NumPy, I set off on this task. After importing the image, I removed the white borders that existed along their periphery, split the channels apart. I then ran an edge-finding filter over them (after a cursory Gaussian blur) in an attempt to remove the varying brightnesses from the equation, left from the three-color process.
The next step was to align the images, since they were misaligned between frames, again thanks to the three-color/three-photo process. I noticed that most of the photographer's subjects were composed in the center of the frame, and the rest tended to be in the nodes 1/3 of the way from each corner (see: rule of thirds). Knowing this meant I wouldn't have to iterate over the entire image to try to align them — instead, I could focus on those foci.
I used the Sum of Squared Differences to calculate how well each channel matched with the red channel, iterating over every possible combination of movements within a 15x15 pixel square centered around each of the foci. I picked the best match, and kept it.
The low-resolution images worked perfectly for this method, but running large files through that process turned out to be quite time-consuming since the offset distances were greater.
I developed a scheme that used an image pyramid, or a series of images increasing in resolution, to try to find the offset. Unfortunately, after an exceedingly prolonged battle, I could not get this method to work reliably.
My rule-of-thirds focus method also occasional backfired on images where there were large areas with no edges. On those, I had to tell the program to ignore certain focus points lest it get a wrong reading.