Title: Virtual Visual Cortex
Artist: Rodolfo Antonio Salido Benítez (A10969987)
TA: Stephanie Sherman
Completion Date: March 17, 2015
Place of Creation: Photoshop, Matlab, Illustrator. (image processing)
Technique: Complex image processing and graphic design.
Material: Input photograph. Original file
My project consists of a visual representation of the information integration process of visual input in humans as modeled by computers. I presented this process as a multi staged process that intends to extract data in specific modalities from the input image. I originally intended to create a diagram explaining the biological integration mechanisms behind our vision. However, after many hours of research, I found that understanding of the information integration processes within our brain is very limited. I searched for methods to develop visual representations of the different visual integration pathways with very limited success. Thus, I chose 2 of the pathways that I thought could generate an appealing visual representation: the Gabor filter model for Cortical Simple Cells visual information integration (which effectively explains center-surround antagonism in Retinal Ganglion Cells and Orientation Tuning in Simple Cells) and a simplified color discrepancy model (which represents the sensitivity of different cones in our retina to different wavelengths of light – colors-).
The visual representations were generated by different computational algorithms in MatLab. The input image was first “Gabor filtered” under parameters that selected for an effective edge identification (relative to the image resolution) and different edge orientations (0º, 30º, 60º, 90º, 120º, 150º). This process closely resembles biological mechanisms (prefered orientation in simple cells studies by David Hubel and Torsten Wiesel). The output images [(A) in the diagram] were then filtered by non-maxima suppression and “added” on top of each other to generate an “edge detection” image. The simplified color perception visualization was generated by softening the original image with Gaussian blur. The purpose of this was to decrease contrast in the image generated by fluctuating light intensity (darks against lights) as these image characteristics are not part of color perception. The blurred image was then colored filtered through 3 modalities: green, red, and blue. This colors correspond to the different peak absorption wavelengths of opsins expressed in retinal cones (“color receptors”). This model is a simplified color perception model because it ignores color opponency mechanisms (mechanisms that detect color contrast when both colors have the same light intensity). The two images in the color perception visualization [(B) in the diagram] correspond to a gray scale visualization of color content (Red, Green, and Blue) within the image and a color visualization of the 3 different color channels. The final image [(C) in the diagram] is a visualization of the computational model of human visual perception used in this project. It is a result of an “addition” of the six different orientation channels and the 3 different color channels.
The design of the diagram intended to make good use of space. However, after further analysing it, I realized that the hierarchical order it was presented didn’t closely resemble human visual integration (orientation tuning involves “higher order” neurons while the simplified color perception involves “lower order” receptors and neurons). I wanted it to resemble a textbook diagram because my initial intention was to provide an effective visual aid for the understanding of Orientation Tuning ( I disliked diagram used in my neuroscience textbook ). The complexity of my “idealized” diagram was far outside my reach as of now (it involved a decent amount of computational image processing knowledge) yet I’m very pleased with the diagram of Orientation Tuning I developed. I opted for a diagram with “depth” in an effort to effectively include several different visual integration “steps”.
My original intention for this project changed drastically. At first, I wanted to construct an image that closely resembled reality based of biologically inspired computational analysis of image features. I wanted to “piece together” a photo from different characteristics individually mathematically extracted from an input image. However, after days of research, I encountered various limitations. Thus, I shifted my focus from trying to emulate human vision to trying to bring attention to “computational vision”. Through this “new” project I wanted to bring attention to several different issues:
Computers have very limited image processing capabilities in comparison to humans, however, advances in computational models for visual integration actively fuel breakthroughs in neuroscience. The limited image processing capabilities is only one example of many different tasks that computers fail on in comparison to humans. This highlights the different capabilities of humans and computers. This should encourage thought concerning the social fear of machines completely replacing humans in ordinary life.
The main purpose of the “final” image in my diagram was to provide an “idea” of how computers can process visual information. This image shows how machines can process the world that surrounds them differently than we do. Object, site, and face recognition processes differ greatly from humans to computers. This demonstrates how differently we integrate and communicate with our environment in comparison to machines.
Finally, my goal of creating a better visual teaching aid demonstrates how heavily we rely on visual input not only to understand our physical surrounding but even to try and comprehend abstract ideas. This realization reinforces the idea of “Visualism” brought forth by Johannes Fabian.