In the Netherlands, families can claim an allowance towards the cost of childcare. Over several years, the Dutch Tax and Customs Administration treated large numbers of parents as fraudsters, and demanded that they repay their allowances, in many cases amounting to tens of thousands of euros.1
On 15 January 2021, Prime Minister Mark Rutte’s government resigned over the scandal, known in Dutch as the toeslagenaffaire.1
What happened
A parliamentary inquiry described the affair as an “unprecedented injustice”. According to reporting at the time, approximately 10,000 families were forced to repay tens of thousands of euros of subsidies after being wrongly accused of fraud. Some parents were racially profiled.1 Announcing his government’s resignation, Rutte said: “The rule of law must protect its citizens from an all-powerful government, and here that’s gone terribly wrong.”1
The role of data and algorithms
The scandal had many causes, including harsh laws and an institutional culture of suspicion. Data and automated risk assessment were part of it.
In October 2021, Amnesty International reported that the tax authority’s risk classification model for childcare benefit had included “Dutch citizenship: yes/no” as a category, and that non-Dutch nationals received higher risk scores. Amnesty found that the authority had not assessed the human rights risks and impacts before using the model, and described the outcome as racial profiling.2
In December 2021, the Dutch Data Protection Authority fined the tax authority €2.75 million. It found that the authority had kept track of which childcare allowance recipients had dual nationality and had processed that data for years in an “unlawful, discriminatory, and therefore improper manner”. The data should have been deleted in January 2014, but was still held in 2018.3
In April 2022, the regulator imposed a further €3.7 million fine relating to the authority’s fraud signalling system, a list used to prepare risk models. According to the authority’s own investigation, staff had been instructed to partly base the risk of fraud on characteristics such as nationality and appearance.4
Why it failed
Suspicion was built into the system. Risk scoring treated characteristics such as nationality as signals of fraud. Once a family was flagged, the presumption ran against them.
Errors were enforced at scale. Flags turned into demands for repayment, and the safeguards that should have caught mistakes did not do so in time. An inquiry does not describe an affair as an “unprecedented injustice” when the system for correcting errors is working.
Nobody owned the whole picture. Responsibility was spread across legislators, ministers, officials and systems, so that no one stopped the harm until it had become a national scandal.
Lessons for leaders
- Never let protected characteristics become risk factors by default. Using nationality or similar data to target scrutiny needs explicit legal and ethical justification, and usually will not have it.
- Assess human rights impacts before deployment, not after. Amnesty’s central criticism was that this had not been done.
- Make every adverse decision explainable and challengeable. People flagged by a system must be able to find out why and get a human review, quickly.
Questions to ask in your organisation
- Which personal characteristics feed into our risk scores, and could any of them act as a proxy for nationality or ethnicity?
- When our systems flag someone, how easily can that person find out why and challenge it?
- Have we carried out a human rights or equality impact assessment of our risk models, and when was it last reviewed?
Sources
-
Al Jazeera, “Dutch PM Rutte and his government quit over child welfare scandal”, 15 January 2021. Link ↩↩↩↩↩↩
-
DutchNews.nl, “Tax office algorithm led to racial profiling: Amnesty International”, 26 October 2021, reporting Amnesty International’s Xenophobic Machines. Link ↩
-
NL Times, “Tax office fined €2.75 million for privacy violations in benefits scandal”, 8 December 2021. Link ↩↩
-
Clifford Chance, “Dutch government fraud scandal leads to record-breaking GDPR fine”, 26 April 2022. Link ↩↩
Infotactica