Thread on Visual Attention and Semantic Centers

Author: @meekaale on Twitter

Full Title: Looking Very Cursorily A...

Category: #tweets

URL: https://twitter.com/meekaale/status/1346017719896956928

Highlights from January 11th, 2021.

looking very cursorily at what researchers in visual attention are up to this study from 2019 finds that the human gaze in everyday situations is not well captured by purely visual models, that our eyes are always guided by the "semantic" field https://t.co/80yeiaZ19Z

so here is the eye-tracking results in the top row, and below are various predictive models middle row: models based on just image processing by contrast, etc bottom row: models based on "meaning maps" where volunteers rate small pixel patches on a meaningness scale https://t.co/kNhkOUsgq2

the researchers have an overall project to prove that "image saliency is a relatively poor predictor of where people look in real-world scenes, and that it is actually scene semantics that guide attention"

they say image saliency models were theorized from results of experiments based on geometric images; there's a big difference between "the semantically impoverished experimental stimuli that originally informed image saliency models, and semantically rich, real-world scenes"

regarding these theories: "Critically, the singleton stimuli in these studies lacked any semantic content." a very scientific move: theorize how visual perception works by analyzing human behavior when faced with completely meaningless images cc @meaningness @literalbanana

then later research can find, stunningly, that people tend to look at what actually somehow MATTERS: "a number of recent studies indicate that scene semantics are the primary factor guiding attention in real-world scenes"

but I'm interested in this also because I wonder if Christopher Alexander's proto-scientific theory of Centers has been brought to empiricism in studies of visual attention

and there is also a question for me about how to see Alexander's Centers as both visual and semantic he presents his "theory" sometimes in a mode of almost pure geometry but clearly he is attuned to how Centers are never purely geometric but always enmeshed in meaningness

so I imagine an eye-tracking study of a visually complex scene with lots of contrasting items and geometric Centers and somewhat off-center, in the midst of everything, is a pregnant woman gently holding her hands around her exposed belly

our attention in this scene can never be grasped with an image-based model of pixel matrix convolutions or whatever the pregnant belly is a center in many ways, in Alexander's terms it is "profound"