It may seem like something out of a science fiction movie, yet an artificial intelligence system recently played a key role in the overthrow of a democratically elected government. The Netherlands’ government chose to resign in January 2021. The incident that led to their departure began in 2012, when a self-learning algorithm was used to look for evidence of benefits fraud. The tax office hoped that the system would detect possible fraud instances, which would subsequently be assessed by humans to eliminate any inaccuracies.
According to IEEE Spectrum, the algorithm developed bias tendencies, disproportionately classifying people from lower-income families, immigrants, and ethnic minorities as having committed fraud. Many of the false flags were approved by the people responsible with reviewing the cases, which were maybe overly trusting of the algorithm and ordered that money be repaid. The new system’s (and the humans in government who trusted it) implications cannot be understated. Thousands of individuals were forced into poverty and others committed suicide as a result of massive tax bills that occasionally reached six figures.
Chermaine Leysner thought the government had made a mistake by requesting her to repay the child benefit she had received since 2008 until she got a bill for €100,000. “I was trying like crazy to offer my children some good things to eat or purchase sweets,” Leysner told Politico. “However, there were instances when my little son had to go to school with a hole in his shoe.” She divorced her children’s father due to the stress of the bill and other personal situations. She wasn’t by alone.
The Ministry of Justice and Security Statistics commissioned an estimate of how many children were taken into custody after a false fraud alert. They discovered that the plan had removed over 1,100 children from their families, placed them in care, and kept them apart for years. The invoices, which were incorrectly issued to 30,000 parents, affected about 70,000 children. Because the projection was based on data after 2015, the value might be much higher.
Compensation of at least €30,000 will be paid to victims of the algorithm and the people who permitted its faulty assumptions, but it will not be enough to save the government from the scandal, dubbed “kinderopvangtoeslagaffaire” in the Netherlands. “Innocent individuals have been criminalized, and their lives have been wrecked,” Prime Minister Mark Rutte told reporters as his administration agreed to resign in January as a result of the scandal. “This is where the buck ends.”
