Military Review English Edition May-June 2014 | Page 21

ROBOTIC WARFARE

assigned to which individuals and to which nonhuman components. Such practices might be promulgated through job descriptions, instruction manuals,
ethical codes, observation of past practices, training
before taking on a role, and so on. Backward-looking
responsibility involves practices of tracing back what
happened and identifying what went wrong. When
a failure occurs, humans will seek out the cause of
the failure, and humans operating in the system will
be asked to account for their behavior. Backwardlooking responsibility generally relies on, or at least
presumes something about, forward-looking responsibility. That is, to understand what went wrong or
what is to blame, we have to understand how tasks
and responsibilities were assigned.
The extent to which individuals operating in the
system are perceived to have responsibility or feel
themselves in a position of responsibility is not simply
a matter of tasks being delegated. It also depends on
how responsibility practices convey expectations
about the duties and obligations of the humans and
hold actors accountable for performing or failing to
perform as expected. Whether someone is considered
responsible depends, as well, on evolving notions of
what it means to be in control and able to think about
the consequences of certain actions.
In a given system, adoption of a new technology
may lead to negotiations about changes to existing
responsibility practices, creation of entirely new practices, or both. Established practices may not accommodate the changes produced from introducing the
new technology, e.g., changes in activities, attitudes,
and relationships between people. The real-time
stream of data that current UAVs produce is a good
example here. The role of pilots has changed insofar
as they now monitor video images and continuously
communicate with others who have access to the
same data and images (e.g., the sensor operator, the
mission intelligence coordinator, and the data analysts
miles away in an information fusion center). This has
transformed the way targeting decisions are made,
compared to manned operations. Decision making
has become more shared and less compartmentalized. As a result, established ideas about what various
human actors are supposed to do and what they have
to account for have had to be adjusted.23
New norms and rules have to be established to
govern the activities a new technology makes possible. Duties and obligations have to be reevaluated
MILITARY REVIEW

May-June 2014

and redefined. Mechanisms for evaluating actions and
holding others to account have to be adjusted or created. This will also be the case for future autonomous
technologies. Regardless of how machine autonomy
is interpreted, whether someone is responsible for the
behavior of the system will not only depend on what

Human responsibility
can best be understood as
constituted through a set of
responsibility practices.
the machine can and cannot do, it will also depend
on the practices that prevail in the context.
Shared values and principles may shape the
establishment of new practices. In the case of UAVs,
organizational values and national and international
laws provide a moral framework for and set limits
on the new activities these technologies enable. Take
the principle of distinction, a key principle in international law that states civilians should be distinguished
from combatants. This principle is intertwined with
established responsibility practices within military
organizations, as they are part of their routines, protocols, and procedures.
Yet, these shared values and principles are subject
to negotiation. Achieving an interpretation of them
may be challenging because of the introduction of
new technologies and also because of social, political, and economic developments. The current debates
about the use of drones provide a pertinent example.
One contentious issue is that, according to anonymous
government officials, the U.S. government regards all
military-age males killed in a drone strike as combatants unless proven otherwise.24 Such a controversial
and broad interpretation of a key principle of the law
of war affects responsibility practices significantly, at
least in the sense that soldiers involved in deploying
drones are held to a certain standard of responsibility
for harm to noncombatants.
Responsibility practices are continuously negotiated and renegotiated. This can often be seen when
something goes wrong with a new technology, and
investigators trace back the cause of the failure.
19