Late last year, the European Union agreed on a draft of the AI Act. It will undergo final approval this spring, and will go into effect in 2026. Judging by the gap between the go-live and enforcement dates with past regulation such as GDPR and the EU Copyright Directive, that means we’re looking at 2028+ until we start to see enforcement and fines. Which is a life time when it comes to AI (the AI act went through lots of revision to incorporate foundation models after the 2023 release of chatGPT).

Either way, I was very interested to see the applications of AI that are outright banned due to “the potential threat to citizens’ rights and democracy”:

Let’s review them one by one:

biometric categorisation systems that use sensitive characteristics (e.g. political, religious, philosophical beliefs, sexual orientation, race)

Good one.

untargeted scraping of facial images from the internet or CCTV footage to create facial recognition databases

The keyword I don’t love is “untargeted”, which will make it possible to scrape facial images so long as there is a justification. I support very heavy limitations on applications involving facial recognition, including collection of the necessary data.

emotion recognition in the workplace and educational institutions

This is an interesting one. As a student at MIT, I would walk through a super long hallway called the Infinite Corridor most days. Up high along the corridor sat a huge screen that detected faces (I’m using past tense because I’m pretty sure they wouldn’t do this anymore), inferred an emotion, and super-imposed an emoji representation of that emotion on top of each face. I tend to have a “resting unhappy face” (if you will) and I would always get a sad face plastered on top of my body (mostly not accurate, so I found it mildly annoying).

This was 2010 and earlier, and at the time zero people were concerned that this emotion detection was unethical. It did feel “off” to me, but not hugely so.

So, should that application be banned? I don’t have a good answer, my thinking on this might be stuck in 2010 when I viewed it as a not-too-harmful research project.

social scoring based on social behaviour or personal characteristics

Tons of companies do this (for fraud prevension and other uses) and the ban will have a large impact.

AI systems that manipulate human behaviour to circumvent their free will

AI used to exploit the vulnerabilities of people (due to their age, disability, social or economic situation).

These seem extremely broad. Is this a limiation on social media? I’m pretty sure that Instagrams’s algorithms manipulate me to keep on scrolling and scrolling. Is that a circumvention of my free will?

The intention must be more narrow; according to this article the ban targets the cognitive behavioral manipulation of people, for example, “voice-activated toys that encourage dangerous behaviour in children”.

It’s super interested to see the start of regulation for AI, I’ll be following all of this closely.