Automated Systems and the Need for Change

This chapter's foray into the agency conditions underlying the continued possibility of socio-moral change is meant to flag a methodological problem that has so far been overlooked by computer scientists and lawyers alike. To truly take on board the challenges raised by our evolving socio-moral stances is not just a case of developing systems that are capable of dynamically (and intelligently ) updating their ‘utility function’ in light of perceived changes. One also needs to consider the effect those very systems will have on the processes that lead to such changes. In the absence of a radical shift in the design choices that preside over the way those systems call for interaction with us, lazy normative animals, that effect will be dramatic, to the point of possibly undermining the very possibility of human-triggered change.