2. Your digital reputation may affect your opportunities. Facebook posts may affect your chances of getting or losing a job, bad friends may affect the rate of your loan, etc. These effects are independent of whether the data is good or bad.
3. People start changing their behaviour to get better scores which have disparate outcomes. Social Cooling describes the negative side effects of trying to be reputable online. Some of the negative effects are:
a) Conformity – you may hesitate to click on a link because you fear being tracked. This is self-censoring, which has a chilling effect. You fear choosing freely.
b) Risk-aversion – When physicians are scored, those who try to help sicker patients have lower scores than those who avoid such patients because sicker patients have higher mortality rates.
c) Social rigidity – Our digital reputations limit our will to protest. For instance, Chinese citizens have begun to get “social credit scores,” which score how well-behaved they are. Such social pressure is a powerful form of control.
4. As your weaknesses are mapped, you become increasingly transparent. This leads to self-censorship, conformity, risk-aversion, and social rigidity becoming normal. No longer is data a matter of simple credit scores.
Yes, we've had credit ratings before. But this is a whole new scale, with an incredible level of integration, automation and accessibility.
Are we becoming more well behaved, but less human? What does it mean to be free in a world where surveillance is the dominant business model?
Are we undermining our creative economy? In a creative economy the people who dare to be different are our greatest resource.
Will this impact our ability to evolve as a society? Yesterday’s fight for equality by a minority is today’s widely accepted norm. But will minority views still flourish?