Designer & Developer: Jieliang (Rodger) Luo
Concept Development: Jieliang (Rodger) Luo, Sam Green
AI Visualization, Digital Prints | 2018
Mixed Vision conceptually explores differences and connections between human and computer vision. The artwork visualizes how a machine sees a painting and mixes the visualization in the original painting. Patterns in the original painting continue and mutate in the visualization. For the first iteration, we chose five paintings: Irises by Vincent Van Gogh, Mona Lisa and The Last Supper by Leonardo da Vinci, Bottles and Glasses by Pablo Picasso, and Suprematism by Kazimir Malevich.
Convolutional neural networks have improved computer vision such that machines can recognize, or "classify", thousands of objects. However, the classification mechanism is still mysterious. In this artwork, we used back-propagation to create saliency maps of the paintings. Saliency maps are a visualization to highlight areas in an image responsible for a given classification. We divided the painting into hundreds of square patches and input each of those to AlexNet for classification. Each painting is titled by the top three recognized classes from the patches. Saliency maps were then generated, colored, and reassembled based on their location in the original painting. Thus, audiences can see many details that a conventional saliency map visualization cannot reveal.