Applied Coaching Research Journal Research Journal 3 | Page 27

APPLIED COACHING RESEARCH JOURNAL 2019, Vol. 3
Evaluating a CoP
Alongside the planning of the logistics of running
a CoP, planning for evaluating the CoP took place.
Although three separate CoPs were established, our
focus for the evaluation was on the PE and school
sport group as this was identified as the CoP which
would attract the largest number of staff. CoPs
were a new concept and way of learning to many of
the coaches involved. Thus, it was anticipated that
any shift in behaviour would take time. Therefore,
a longitudinal approach was adopted and the
evaluation was planned to run over a 12-month
period. In addition, the evaluation was designed
to reflect the social nature of a CoP. Rather than
relying on common, more traditional methods of
collecting insight (eg via surveys and interviews),
we wanted to be more innovative. Equally, we felt
that the most appropriate way to monitor social
behaviour was to observe the CoP in action. An
observation tool was then developed to ensure
that any observations that were carried out were
focused and consistent.
Community indicators framework
To develop our observation framework, we drew
upon previous work by Galley, Conole and Alevizou
(2012), who developed a community indicators
framework to help them observe and support
the communities using Cloudworks – a social
network platform for education professionals. They
were interested in assessing social behaviour by
capturing activity patterns and uses across the
platform. Galley and colleagues focused on four key
aspects of community experiences:
1) Participation – the ways in which individuals
engage in activity
2) Cohesion – the ties between individuals and the
community as a whole
3) Identity – how individuals perceive the
community and their place within it
4) Creative capability – the ability of the community
to create shared artefacts, knowledge and
understanding.
The community indicators framework was adapted
and simplified to provide an appropriate tool to
guide observations of the CoP (Figure 1). The key
things we were interested in capturing were:
1) Engagement – who joins in the conversations and
flashpoints of interest
2) Types of interactions – what are members
sharing (ideas, resources, experiences)
3) Support – are conversations enquiring and
supportive, willingness to listen and help
4) Relationships – what relationships are
developing?
Frequency: How many attend?
Engagement Types of Interactions (record examples)
Who (how many) joins in conversations? Informational (eg, sharing resources, links)
Examples of flashpoints of interest and
engagement Practical (ie, sharing practice/experience, producing
shared artefacts eg, definitions, proposals)
Are people motivated to return? Do members share ideas, experiences, issues or
challenges (e.g. personal issues/experiences)?
Support Relationships
Are conversations enquiring/supportive? What relationships are developing?
Is there a willingness to listen, help and learn
from others? (ie, an interest in others' views?) Is there a sense of community? (what does this look
like?)
Figure 1: Adapted community indicators framework for observing the CoP
27