From the formulation of linear perspective in 15th century Italian art to 19th century stereoscopes and the 3D glasses of the 1950s, the quest for the third axis in a flat picture remains a hot pursuit. Tricking our eyes to "see" a three-dimensional object where it isn't provides not only great amusement but also important insights into the visual processing mechanisms of the brain. The sudden popularity of autostereograms, also known as stereograms (and sold commercially as "Magic Eye" pictures), grew out of several decades of work by a number of researchers.
I became intrigued with stereograms and investigated ways to create my own. The basic techniques are fairly simple; in fact, you could theoretically create a crude stereogram with a typewriter or even a pencil and paper. However, computers make stereogramming a practical exercise. A number of free and commercial programs to generate stereograms exist (see the FAQ at http://www.ccc.nottingham.ac.uk/~etzpc/nz/sirds.html and the Usenet group alt.3d). Having recently begun programming in Java™, I decided to create an interactive stereogram applet to learn about both Java and stereograms. First I'll give a brief introduction to stereograms and then I'll discuss my Java implementation.
A number of variations on stereogram making exist, but here I followed the approach of W. A. Steer (see http://www.ucl.ac.uk/~zcapl31/stereo.html). Figure 1 shows light rays received by the eyes from points on two lines at different depths. The pupils must separate slightly more to look at point A than at point B; that is, the rays arrive slightly closer to parallel from the more distant point. The brain attributes this to a difference in depth.
Suppose I placed line S of colored pixels between the scene and the observer so quickly that the pupils remain separated exactly as before to receive rays from point A or B. Could I color the pixels such that I fool the observer into believing the points remain unchanged? Well, I would need a hologram to reproduce the same physical wave front, but I can at least reproduce a similar depth image. The key idea involves correlating the separation distances between the rays where they cross the stereogram line S.
In Figure 2, the distance of separation a exceeds that of separation b since point A lies deeper than point B. Figure 3 shows pixels (their size highly exaggerated) at the crossing points for A and B with yellow and red coloring. The brain will perceive a yellow point at A and a red point at the closer point B. If you create similar correlated pairs of points all across line S, you can produce the impression of two lines of differing depths.
The pixel pairs receive random color values and the lines will appear multicolored as indicated in Figure 4. (Simple black and white dots work fine but create somewhat dull pictures.) The randomization of dot color and position inspired the more rigorous name of Single Image Random Dot Stereograms, or SIRDS for short.
The basic method of creating a stereogram starts with initializing the pixels to random colors. Moving left to right, the pixel at the calculated separation distance to the right of its left partner receives the same color as the left pixel. Repeating for each horizontal line builds the full 3D image. The resulting stereogram displays a somewhat grainy texture. A nice alternative employs an image to initialize the pixels instead of random colors. This results in a stereogram that displays repeated strips of the image where the strip width reflects the separation width of the deepest level of the scene. Many commercial stereograms use a background image that corresponds in theme to the 3D scene.
I said above that after inserting the stereogram, the eyes must remain focused on the distant points. This is the tough part of seeing the 3D effect. The brain automatically tries to focus instead on points in the stereogram plane. Training to defocus and see through the stereogram takes some practice-some people never quite get the hang of it.
Figure 5 shows the basic parameters involved in Steer's stereogram algorithm. The featureZ value is the depth of points in the object across each horizontal pixel line. The stereo separation variable sep gives the distance between the pixel partners corresponding to the given depth point. Then sep derives from the depth value and the observer's eye distance (obsDist) and separation (eyeSep) with the formula:
sep = (eyeSep * featureZ) /( obsDist + featureZ)
The algorithm proceeds as follows (the calculations assume pixel units for all variables):
A hidden problem can arise when one surface lies close and below another as in Figure 6. Here the left pixel belongs to two pairs: one corresponding to a depth point closer to the observer than the other. To solve this problem, the program will accept only the shortest link; that is, it biases to the closest point to ensure that near objects obscure far ones. The right pixel of the wide pair receives a different color than the left pixel to break the link.
The depths of the objects cannot range arbitrarily. Steer recommends limiting the range of depths such that the stereo separation varies by a factor not greater than two. A higher factor can lead to artifacts, or false images. Here, the default maximum depth equals the distance to the observer. The user can manipulate the minimum depth with the sepfactor that equals the ratio of the smallest stereo separation to the maximum. Solving the equation above for the minimum depth gives:
minDepth = (sepfactor * maxDepth* obsDist) /
((1-sepfactor)*maxDepth + obsDist)
By creating a Java stereogram program, I intended to learn more not only about stereogramming but also about image manipulation within the Java applet environment. I needed to obtain images, get their pixels, manipulate the pixel information, and then reload the pixels and display the resulting images. Java's multithread processing, for example, can cause tricky situations. A program can't make assumptions like thinking that an image's loading has finished when in fact it could still be proceeding in parallel with other threads.
My stereoMaker applet, shown in Figure 7, lets you make stereogram images of simple text input. In other words, the letters will appear above a lower flat plane. The user can choose to make stereograms initialized to randomly colored pixels or from one of a predefined list of images. Several options allow you to experiment with the algorithm parameters, the size of the letters, and so on. Two or three types of colors for the random pixel images can also be selected.
Figure 8 contains the Java code for StereoMaker. The StereoMaker class carries out the stereogram creation; the StereoCanvas class provides a framed canvas on which to display the stereogram, and two control classes-StereoControlsHi and StereoControlsLo-provide two lines of input controls. In addition to framing the stereogram, StereoCanvas can display a message when the stereogram processing is in progress. The controls pass their values to StereoMaker directly by accessing its public variables.
The StereoMaker class inherits the applet class and implements the Runnable thread interface. The applet includes the usual methods to build and maintain a threaded applet: init, start, stop, restart, handleEvent, and run. The init method creates the canvas and controls objects and lays them out. A MediaTracker object will monitor the image loading. The last thing it does is load a default backplane image in case the user chooses an image-type stereogram. The run method checks on the backplane image and other setup information before calling the stereogram-making methods.
The core stereogram-making routines include makeDepthImage, makeStereoRDS, and makeStereoFromImage. I will briefly outline what they do.
The depth image provides the coordinates of the scene to render in 3D. Here, the blue component of the 24-bit RGB pixel provides the height value in a range of 0-255. Method makeDepthImage initially creates a totally black image and graphics double buffer (for faster processing). It uses the FontMetrics class to obtain the information needed to
center and draw the input text on the image in blue. The
program then displays the image. Once Java has done most of the work of creating the image with the letters, I use
the PixelGrabber class to get the pixels for the image. The height array information is then available to the stereogram routines via the depthMap and Height methods. They
take care of translating height to depth and of scaling the depth values.
The makeStereoRDS method carries out basic RDS stereogram processing. The pairing of pixels as described above occurs with the link arrays linkL and linkR. An outer loop cycles through each horizontal line of the stereogram. The inner loop cycles through each point X in the line, first obtaining the corresponding depth value at X from the depth map. It the calculates the stereo separation sep value for point X and creates the pixel pair by linking the pixels at X±(sep/2). Finally, the color values are placed into the pixel array. After returning to the run function, the newImage method will create an image with the pixel array and pass it to StereoCanvas to display.
The makeStereoFromImage method follows many of the same techniques as makeStereoRDS. One difference is that it obviously uses a strip of an image as its base pattern. The method getBackplaneImage loads the selected image and getBackplaneInfo obtains its pixel array. The getBackplanePixel method then provides the pixel at a given position.
The stereogram will use a strip of the image of width equal to the maximum stereo separation. I could begin with the image strip at the left, but it will become increasingly distorted by the pairing of pixels along the horizontal lines. So to give a more balanced look, the stereogramming begins in the center and works outward to the left and right. Further, to improve the resolution, an oversampling technique treats the stereogram as initially larger by a factor oversample than the true image size. At the end the colors are averaged over every oversample number of pixels to obtain the final image.
As an exercise in learning Java and the basics of stereograms, I consider this little applet a success. I'm certainly happy to discover so many ready-made classes that greatly simplify image handling, such as PixelGrabber, which provides the pixels in RGB arrays. A few aspects were somewhat disappointing. I wanted an option to allow entry of any arbitrary URL for the image stereograms. This would allow the user to experiment with the innumerable images available on the Web. However, the Java security system only allows the applet to load URLs from its home site. For similar reasons, the user cannot save a stereogram to a file. (A screen capture can save the image, although a graphics program will be needed for cropping.) A standalone version would overcome these problems, of course, but that reduces some of the charm of Java and Internet downloaded programming.
A number of upgrades to the stereogramming applet come to mind. For example, the applet currently shows only two depths: the text and base planes. A standalone version could allow the use of depth images made from other sources. A nice addition to the applet would provide a second, separate window with a drawing pad and a palette of colors representing different depths. The user could draw multilayer depth images while occasionally pressing a button to transform the current image to a stereogram to monitor the work in progress. In all, it's not hard to generate a great-looking applet that actually does something.