6.4: Irresolvable Paternalism
kill-condition
ClaimIf steering toward long-term values during short-term maladaptive behavior cannot be operationally distinguished from paternalistic control, the caring orientation creates the very principal-agent problem it claims to solve.
If "steer toward long-term values during short-term maladaptive behavior" cannot be operationally distinguished from paternalistic control regardless of how the machine is specified, then the caring orientation creates the very principal-agent problem it claims to solve. The resolution (making departures visible vs. controlling outcomes) must hold in practice, not just in principle.