Introduction to Perception

Continuing on Julian's reply, it's interesting to consider that one of the current popular systems for perception based machine learning tasks are neural networks. "Deep learning" is, specifically, trying to emulate our different levels of perceptual processing. So in the case of a visual perception task, deep learning is trying to make the case that the first layer in the neural network will do edge detection, the second would do shape / object recognition, etc. However, in practice this seems a lot more difficult to realize. I wonder if this relates to the rich feedback loop we, as humans have, versus the simple correct / incorrect feedback used for training an supervised learning system. By using this rich feedback, it might be the case that we are able to better exercise top-down control on our perception process, enabling more complete learning. In the question regarding top-down processing, prior knowledge is just one component of this top-down processing. There are other factors, such as context (as noted by Julian) that are also a part of this processing. I personally think that what is missing from machine learning is a richer feedback loop enabling context decisions that could really take machine learning to the next step. How to realize that is another question

Tufte builds on this work with the nice idea that most of the elements of a graph or chart should just be above the noticeability threshold. Only key elements should have greater noticeability. This leads to some very bland charts that are mostly white and light gray, but I think more designers should heed his dictum, and not just for design of charts. When creating interfaces, most designer's first instinct is to highlight everything. All the features are equally important because if they weren't, why would we engineer them in the first place? As a result, the user has no visual cues for what is actually important for their primary task and what is not. By understanding that noticeability is a design dimension one can manipulate to direct users' attentions, designers can be more conscious in their design. I think this is a big benefit of this kind of research that designers alone would not have been able to come up or express this precisely.

Obviously, I don't think we actually want to design systems that let the blind see and the deaf hear. We don't want to do this because it is an impossible goal and really in the realm of medical research, not HCI. I think the better question for HCI is to understand what makes a blind or deaf persons perceptive system different than a seeing or hearing person. The more we can understand the unique facets of their perceptive system, the more we can design technology that builds upon their strengths and fixes their weaknesses. By viewing the problem through this lens, I think we'll have many more ideas of assistive technology and not be limited by the goal of trying to make the blind or deaf identical to sighted/hearing.

Psychedelic drugs will tend to induce alternative processing of comparable sensory inputs. By disrupting the perception-cognition chain of processing, alternative cognitive concepts and emotions may be triggered. Over some time window, these alternative activations of concepts may trigger analogy mechanisms which lead to more creative connections between concepts that normal perceptual processing would not activate. This conceptual activation goes beyond just thinking about two concepts simultaneously. Having both concepts activated simultaneously and superimposed over an experience might also trigger other cognitive mechanisms beyond analogy in order to reduce the cognitive dissonance generated by seemingly separate concepts merging. Likewise, unconscious conceptual blending mechanisms may manifest the in the actual delusions, which results in unconscious analogical reasoning or possibly triggering conscious analogical reasoning. This mechanism leverages the human mind to blend and connect concepts in ways that aren't easily accessible by the conscious mind (yet). One challenge seems to be the many latent dimensions that concepts may be connected that are activated by these drugs are computationally hard to discover or select between in inducing an artificial delusion. Likewise, blending two concepts together into a "realistic" output concept is not a computationally easy task in terms of marrying specific subspaces of concepts into a functional wholistic new output. This is an interesting space computationally both from a replicating the human ability, and in trying to provide access to this ability to the conscious mind. 

I was thinking a lot about visual perception and art while reading this chapter. For example, art/paintings often play with our visual perceptions, and it's interesting how much we can understand and view as 3d and realistic even when much of the image is not photo realistic. For example, this paper uses an Edward Hopper painting, and I think Hopper has this really brilliant way of capturing key ways in which we perceive light and depth that makes his paintings feel super realistic/authentic, even though they often include a lot of visual elements that aren't really 'realistic'.

To branch off Mary Beth's discussion of DeepDream and increasingly abstract art-- actually I tried DeepDream with Francis Bacon's Three Studies for Figures at the Base of a Crucifixion (https://goo.gl/UxTlXz), and the results weren't really interesting-- I think the original seemed more surreal. The DeepDream version basically just added some 90s-looking streaks to the painting. Then I tried it with the photo of the house my new apartment will be in, and this time it worked more as expected, but I have to say I didn't find it all that compelling. It just kinda stuck eyes and faces where you might imagine eyes and faces could fit into. So like, I get why people would want to try LSD and the like. My brother has been particularly interested in sensory deprivation chambers for the past year. I'd characterize myself as someone who is interested in the experience of/effects pareidolia but afraid of long term (drugs/LSD) and short term (freaking out and/or accidentally getting stuck in a sensory deprivation chamber for longer than intended) effects. So given this, I'd be interested in the potential of applying algorithmic pareidolia to virtual reality experiences to be able to replicate, for example, the experience of taking LSD.

But in terms of its creativity enhancing potential: do experiences of pareidolia really enhance creativity, or does it just seem this way because fewer people naturally or through drugs experience pareidolia, so the things people come up with after pareidolia are relatively unique perceptions of the world? If everyone can do it, is it more just like, eh, same old-same old, another house with a weird animal face or whatever? (And can extend this idea to whatever non-visual forms of creativity pareidolia might inspire, as well).

RE: What are the implications of a "just noticeable difference" on the design of interactive systems? Any interesting connections to your research? [credit: Chris MacLellan]

I support the point Alex makes previously. From my limited knowledge in designing interfaces (specifically dashboards), users' attention and perception can be manipulated (in a good way) using colors, contrast and other elements of display and visualization. For example, in a dashboard that displays different types of data, using the same color/light colors creates a consistency. If you want to create "just a noticeable difference" you can mark something red or use another form/shape for contrast (example below).

Judy asked "Beyond psychedelic drugs and isolation chambers, is any creative act top-down processing? Am I relying on prior knowledge to create something new?"

and Joselyn wrote "....I'm a bit skeptical that what's happening in deprivation rooms and while using LSD is strongly related to amplified top-down processing. It would seem that an entirely different process altogether, or a combination of processes is engaged when the mind is in that form of an altered state. I could see that the argument is that in top-down processing, the behaviors/perceptions of a person are determined by expectations rather than actual information, like a person who is tripping. I don't know, it seems like a tenuous theory to me, but I'd like to explore this theory tomorrow during class."

In both of these cases, I'd tend to assume that a combination of top-down and bottom-up processing (with an emphasis on top-down processing) would be a reasonable high-level characterization. So I'd really like to hear counterarguments! I agree that this would be interesting to explore in class.