Low Levels of Recognisability Patterns Experiments. A Case Study on Cognitive Ergonomics Research.

Role — UXR, HCI Researcher, Cognitive Ergonomist, Teacher

Year — 2014-2018

Designing cognitive & HCI experiments for researching pattern and colour recognition. The project ended in application to AI-assisted diagnostics for neuropsychiatry, understanding emergent properties as Artificial Recognition in Human-Software Research areas.

  • Goals — Understanding how Low Recognoscibility Patterns (LRPs) can be applied to AI-assisted diagnostics in pryshiatry, epidiagnostics and artificial recognition. Transfer knowledge through courses, building and expanding the experiments’s design in several input lectures.
  • Challenges — Time lapses. Transference of almost unidentifiable, qualitative and mainly personal experiences from subjects with language comprehension and speaking disorders (clinical cohorts) or language expression limitations (children cohorts).
  • Learnings — LRPs open a new form of interfield link among cognitive ergonomics, ethnographic studies of experiences, neuropsychiatry + diagnostics, and AI research.

Overview of the Project.

Disclaimer — What follows is part of a four-year project that started as a visual experiment around 2014, and transformed into a postgraduate experience research applied to experimental ergonomics in 2016, entitled ‘Designing Experiments for Researching Pattern & Colour Recognition with Children Presenting Early Signs of Autism’. The project ended in application to the PhD course of 2018 while investigating on Artificial Intelligence Assisted Diagnostics for Neuropsychiatry, held at the ECyT National Institute for the Study of Science & Technology, in collaboration with the School of Medicine at UCM.

Experimenting with Low Levels of Recognisability.

Patterns surround us, make things clearer, simpler, easier even though their meanings can be abstract. We humans recognise patterns by interpreting variations amongst motifs, by identifying differences and similarities in certain elements of a given collection of tokens that share a familiar taste. These characteristics get such elements repeated as stamps on a letter, framing the boundaries of how we understand, manage and express them to others.

These experiments talk about a unique sort of pattern, one which appears to hide itself from our identification, patterns difficult to get recognised straightforward, formations that are not designed with a previous idea in mind, but that emerge from repeating specific variables of shape and colour and that, when presented to humans, appear to convey a sensation of direction, of transformation, of alignment, of spatial clench or broadening. We called these sneaky compositions Low Recognisability Patterns (LRPs), and this group of experiments wanted to test how useful they are for Clinical Ergonomics (especially in diagnostics), and for understanding emergent properties in Artificial Intelligence (AI) agents, like that of Artificial Recognition in Human-Software Research areas.

Working with LRPs.

LRPs are, in this case, clusters of shapes and colours that work as a standard pattern, like a stamp or a motif would do, however they do so under low levels of recognisability. That means that subjects presented with LRP images or videos will most frequently have the sensation of seeking for a pattern of characteristics, and finding whatsoever close to these two variables, mainly:

1, Persistence or Change in the order of elements of a given composition, and
2, Iteration (repetition) of such order in a different space

Interpreting those variables in a composition means that we seek for a pattern: we create an imaginary collection of tokens that tries to characterise what we see as an ordered group of coloured shapes that repeats itself in space and time.

Below — Example of AI-LRP showing emergent white void rivulets

LRPs are emergent by definition, like the alignments of these dots created by software in the example, and the white, void spaces that compose, like rivulets, the distance among the rows of shapes. LRPs can also be how the entire process of iterations shifts from lighter to darker, its direction, its movement, etc.

Below — Example of LRP showing a letter Y-form, pink

LRPs are not a one-guess type of pattern: as they are not designed with preconception, multiple answers can be delivered. Here the most guessed pattern is a pink ‘Y’ kind of shape. Another possibility: the pattern is darkers/blues on the left, brighter/greens on the right.

When we seek for how subjects reacted to low levels of recognisability, we can just scale the complexity of those two parameters:

1, Multiplying order: complicating the motif to stamp, and
2, Shifting angle, direction, number and timing of repetitions in the iteration sequence

If the subjects interpret the orders and iterations exposed to them, they are said to be recognising the pattern: they recognise the differences, transitions and changes applied to the given collection of shapes and colours.

Below — Example of AI-LRP showing 3 black dots shifting directions

Here the pattern comes by isolating background colours and protagonist shapes: for example, here some black dots form an ordered square that persists in all the sequence, shaping one of the generally answered LRPs.

When we abstract the previous composition, the pattern automatically gets more and more difficult to recognise.

Below — Example of AI-LRP showing 4 black dots abstracted

How the Experiments Work.

The research strategy is mainly visual, qualitative, descriptive and comparative by design. The subject watches the LRPs, interprets them, and gives a feedback trying to answer what is the pattern, where it does appear, and how it can be found.

Steps:

1, Presenting the subjects what to seek for: emergent patterns of colour and shape dispositions, finding the differences and transitions if given, characterising what changes,
2, Exposing the subjects to iterating compositions: dynamic (videos/gifs), static (images),
3, Register how subjects react and what they describe.

Experimenting with Humans and Machines:

The experiments wanted to use modern tech to find whether children presenting autistic signs could be diagnosed at an early stage by comparing results with a control population. The hypothesis comes with these children expected to rapidly identify LRPs, get their attention abducted by them, working regardless they would be able to express or not they point the pattern out. One feature why such assumption proved to be interesting with assessing kids’s recognition skills is that interpreting LRPs is independent of cultural conditions: we don’t need to have background information to recognise them, just to explain them to others (by use of language, for example). This means that non-invasive tech, like neural-reading applied to children, can be used to understand whether they are recognising or not those patterns, even though they cannot express it textually. Experiments can also be used to map how visual, interpretational and spatial skills evolve in during growth.

Phase II of the experiment (to be developed for 2021) will consist in trying AI agents to interpret those LRPs. Software made the pictures by iterations, but can software emergently advance low recognisability too?

Cohorts:

1, A control group not diagnosed within the Autism Spectrum
2, Group presenting early signs of autism

Measurement:

1, Eye-movement tracking
2, Neural firing rates mapping
3, Neuro-glial glucose and chemosignalling techniques
4, Qualitative assessment of their textual explanations and experiences

What the Experiment Delivers: Interpretations & Previsions.

1 — Ethnographically

LRPs are to be found culturally independent. One will not need any cultural background to understand them, just for explicating them, and externalising a response, not for recognising that the pattern exists.

An explanation for this comes with the interpretation that the neural conditions for recognisability of LRPs are very primitive: evolutionarily speaking, organisms exposed to them would have been able to afford value onto those patterns in a very primitive stage of their adaptive processes.

Interpreting LRPs would be prior to the social expression of such patterns, like their linguistic explanation, description, definition, extrapolation, copy…

2 —Neurally

Signal frequency will show high rates in frontal, temporal and occipital lobes (related with visual and contextual information processing), the affective-connective cingular structures (related with emotional values within the given experience), and the caudate nucleus (related with the sense of will in movement), especially with dynamic, non-static, LRPs. In addition, oxygen and glucose rates will come higher than in default mode too, presenting a high metabolic demand of neuro-glial activity. These events together can be interpreted as for LRPs being very difficult to identify, characterise, assess and interpret, of which the neural analysis would come with higher costs than in common recognisable patterns.

3 — Diagnostically

Different populations of children presenting early signs of the Autism Spectrum would be able to be characterised in differential diagnosis (one of the boundaries in neuropsychiatry in current times) through these tests given the answers delivered in comparison with contrast cohorts. This factor can be used, through AI assisted epidiagnostics modelling, for better understanding the specific variations in the emergent cognitive skills of perception and interpretation dynamics that these subjects expose.

These experiments can be used to forecast further clinical behaviour once cumulative data rendered specific profiles, ‘styles of LRPs interpretation’ related with different groups of people. Key questions here appear:

1, Will autistic population, expected to react faster than controls to LRPs, perform as such in all the different manifestations of the spectrum?
2, Can we assume distinctive styles of interpretation in the different clinical collectives within the Autism Spectrum?
3, Can this be a marker for differential diagnostics?

4 — In application to Human-Software Research in Cognitive Ergonomics

Results of subsequent experiments will present alternative frameworks for characterising assessment values on AI agents, which can also serve to better modelling the logical processes of emergent pattern interpretation in humans.