Technology
campusreview.com.au
they can use diverse resources they learn
in school to start to overcome those.
How accurate is the skin conductance
headset? Is it widely used in science?
The one we demonstrated in the
video is the Emotiv headset. It’s used
in game controlling and a number of
humancomputer interaction studies and
efforts in controlling between humans
and computers. There are higher-end
versions of this which essentially are EEG
systems that allow us to look at seizures
and various cognitive states and brainwaves
… in office settings or clinical practice, but
we’re starting to see the ability with the
Emotiv headset – and the new generation
of wireless EEG sensors that are portable
or wearable and then deployable – a
new capacity to understand the nature of
individuals and small teams, in real time, in
social settings. That’s the kind of context
we’re looking at.
One of the capabilities of the Holodeck
project is, how do we integrate the
emerging and existing technologies to
create both the next generation visualisation
with, for example, AR, VR or haptic feedback
– these many different types of modalities
– and also the human dynamic ones
where we can sense to a greater extent
what’s going on both in the cognitive and
emotional states of the learner and their
family members in our health practices?
Is there a way to protect against the fact
that people know they’re wearing a headset
and therefore may behave differently and
adversely affect your findings?
Sure. I mean there’s a number of different
effects. There’s the novelty effect: this is
new and I’m paying particular attention.
Or there’s the performance effect, similar
to what you’re suggesting: as you put this
on you’re going to ‘perform’.
However, as we see the capacity to make
these things more and more of an everyday
experience where this will just be part of
the simulation, these effects will start to
go away, and you have to be present and
capable day in and day out. Eventually you
can imagine having the full capacity of
these types of instruments throughout our
everyday lives.
Members of our Holodeck team are
looking at scenarios called ‘future reality’,
where you have AR experiences and are
wearing physiological sensors throughout
everyday experience, and what does the
world seem like and what are our everyday
interactions? What’s possible in that
future reality?
I fully appreciate that currently there
could be limitations or biases, but that’s
all part of the process of the research and
development where we start to explore
what is happening and how do we make
the best guesses of what the future’s going
to be like and what we’re able to do in
terms of teaching and learning.
What are the potential applications of this
technology?
The Holodeck is a multifaceted instrument
and our team is across domains ranging
from scientific exploration, scientists
and mathematicians looking at, say,
system modelling or designers building
architecture, artists using the instrument
for collaborative dance and performance,
and then human-computer interaction
scientists looking at the future of smart
homes and assistive environments or
patient care environments.
The modality here, where we’re looking
at the physiological sensors and the
simulation, is one in which we’re looking
for the interpersonal and team dynamics
in the training scenario. But the larger story
of the capacity of an instrument like this is
to start to look at individuals throughout
everyday experiences and start to attend to
their needs.
To tend to the specific question of
depression or emotional needs, we’re
already starting to see correlations between
people’s online activity and their use or
physical activity in terms of accelerometry
from a social mobile phone or their
interactions with specific mobile apps that
are designed to attend to emotional states
such as depression, and to some extent give
some empowerment, mediation and control
that starts to help people in these states
overcome these conditions of depression.
What I would say is that there’s a
beginning of an ability to gather that
information. In this specific project, that
hasn’t been our focus, but the instrument
would certainly, in the longer time frame,
become more and more end-user capable.
With the Holodeck project, we’re asking:
how do you create a world-class instrument
that has these advanced features, and
then how do you create open-source
toolkits that allow others to acquire diverse
components of the instrument and set
those up based on the work we’ve done?
That’s part of our dissemination and
broadening capacity of the instrument to
give some capacity or start to give other
investigators and then ultimately end-users
these abilities.
I assume that eventually you aim to roll this
out in a lot of nursing and teaching areas,
but could we see this technology used
to track and improve the performance of
professional nurses?
You could certainly do a number of these
things and look at everyday experience in
the professional settings, but one of the
concerns of tracking is the Big Brother
effect of monitoring in work environments.
You have to be cautious and understand the
buy-in and who has access to the datasets.
We already see these things when people
get mobile phones or laptops from their
employers which have diverse technologies
to track the individual employee. So we’re
starting to see part of the conversation
around privacy and ownership in those
contexts. Of course, you multiply
that when you get into the more, say,
intimate elements of the brainwaves, skin
conductance, heart rate and respiration.
But we are proposing these as features of
an educational system that can be turned
on and off and controlled by the students.
So those are all parts of the way you
advance work in this area.
I’m not sure how long this work has been
going on, but are you seeing any findings
already?
We’re just at the beginning phase of the
simulation work, so setting up the capacity
to do this is a case study finding in the
ability to stream the signals and acquire the
information. In terms of larger and more
generalisable findings, we’re not at that
stage yet.
The process of developing the Holodeck
involves the development of the instrument
and then the use of the instrument, and that
goes back and forth in terms of an iterative
cycle. It’s advance the development, use
it, advance the development, use it, and
so on. So you see the gaps in the use and
then you can start to feed that back into the
needs for the development.
In any development, before you can find
findings, you build your system to a point of
robustness. You do some pilot studies and
then use that to inform the type of research
and the larger-scale empirical studies that
would produce the more robust findings. ■
27