AI incidents in HR piling up
That WIld West situation was not sustainable. The last few years have shown real examples of AI going wrong in HR. One case involved Amazon, where an automated CV screening system began to downgrade women because it learned from historical hiring data dominated by male candidates. Another example came from HireVue, a company that used video analysis in job interviews. Their system interpreted facial patterns and micro expressions in ways that had no proven link to job performance, leading to serious concerns about fairness and prompting the removal of the technology.
These incidents were not caused by bad intentions, but by a lack of rules, oversight and understanding of how AI behaves.
Introducing the EU AI ACT
Because of these risks Europe introduced the EU AI Act. For HR this law is important. Many AI tools used in hiring, screening, assessment or workforce management fall under a category that the law calls high risk. This does not mean the tools are dangerous by definition. It means they can influence someone’s job, income or future, and therefore require more care, transparency and human involvement.
The challenge is that many HR professionals are not aware of this shift. They do not always know what they can use safely, what requires extra steps or what is not allowed. Others simply look away and hope it will not affect them. That is why this article explains the essentials. It gives every HR professional a clear basis: what the EU AI Act means, what high risk really stands for, what HR teams must do and how they can work with AI in a safe and responsible way.
A new regulated reality for HR
The EU AI Act is the first comprehensive law that regulates how AI may be designed, purchased and used. HR sits at the heart of this development because AI in the employee lifecycle processes personal and often sensitive data. When AI influences decisions about people, the law requires transparency. Applicants and employees must be told when AI supports decisions that affect them. HR teams must be able to explain how a system works at a general level, why it is used and how it is monitored.
Human involvement becomes essential. Important decisions cannot rely fully on automated systems. HR teams must supervise outcomes and be able to override them when needed. The regulation also expects organisations to document how AI systems behave, how risks are managed and how fairness is monitored over time.
Alongside the EU AI Act the GDPR remains fully applicable. Whenever AI processes personal data, organisations must determine the legal basis, explain the purpose, protect the data and respect the rights of individuals. Workers and applicants can request insight into the data used about them, ask for corrections and demand human review if an automated decision has a significant effect.
What high risk AI means for HR
High risk in the EU AI Act refers to the impact an AI system can have on people, not to the technology itself.
AI that plays a role in hiring, screening, assessment, promotion or workforce allocation is considered high risk because it can shape someone’s career. This category comes with stricter expectations. Organisations must document the purpose of the system, describe the data used, test for quality and fairness and monitor performance throughout the lifecycle. Logs must be kept so that decisions can be traced if questions arise.
For HR this means very concrete things. If a tool helps decide which applicants move forward, how performance is interpreted or which employees are selected for new roles, the organisation must treat the system with extra care. That includes:
- human oversight
- training for supervisors
- transparent communication
- clear processes for spotting and correcting errors.
How the GDPR shapes AI in HR
The GDPR continues to define how personal data may be used in AI systems. In HR the most common legal bases are legitimate interest or performance of a contract. Consent is rarely valid because employees are not in an equal position to freely agree. Many HR related AI projects require a Data Protection Impact Assessment. This assessment maps how data flows through the system, which risks exist and how the organisation mitigates them. It is a practical tool that helps teams think more clearly about design choices and their impact on individuals.
Working responsibly with HR data
HR data contains personal and sometimes sensitive information. Job history, performance notes, training records, demographic details and communication patterns require careful protection. Encryption secures data when stored or transmitted. Pseudonymisation reduces direct identifiability. Access to raw data should be limited to trained staff. Clear internal procedures help prevent mistakes.
When working with external vendors HR teams must understand how data is handled. They need clarity about training data, storage locations, retention periods, log management and incident procedures. Contracts must describe who is responsible for what, how audits can be performed and how AI outputs can be explained when individuals ask questions.
Fairness and explainability in HR AI
AI systems must be fair. HR teams should consider how training data was collected, whether it reflects the current workforce and how often outcomes are checked for imbalance. Historical patterns can lead to hidden bias, so continuous monitoring is important. Explainability matters just as much. HR professionals and candidates should be able to understand why a system produced a certain outcome. A tool that influences careers must be able to provide meaningful reasoning rather than opaque conclusions.
Security matters even more in HR
AI introduces new security concerns. Manipulated inputs, compromised data, unstable models or leakage of sensitive information can cause harm. HR data is sensitive by nature, so strong protection is necessary. Monitoring activity, limiting access and keeping systems updated reduce exposure. Organisations should also have a clear procedure for identifying errors, escalating issues and correcting decisions when needed.
Choosing the right tools and vendors
Selecting AI tools for HR is not only a matter of functionality. HR teams must understand how a system was trained, how fairness is checked, how often updates occur, where data is stored and how incidents are handled. The distinction between controller and processor determines responsibility under the GDPR. Contracts should make clear how errors are managed, how audits work and how data is protected throughout the lifecycle.
Human oversight remains essential
AI can automate routine tasks such as administrative CV parsing, scheduling or generating standard reports. These activities carry lower risk and offer immediate benefits. But decisions that influence careers always require humans. Supervision by trained HR staff ensures that AI supports decisions rather than replaces judgement. Humans must be able to correct outcomes and question results when something does not feel right.
Governance and continuous monitoring
Once AI is introduced in HR the organisation must monitor how it behaves. Accuracy, fairness, data quality and the need for human intervention show whether a system remains reliable. Documenting updates and evaluations supports compliance and good practice. Clear internal roles help: an AI owner in HR, a technical model owner, a data protection officer, a security contact and representation from legal and worker groups.
Helping employees understand AI
Trust grows when organisations communicate openly. Employees, unions and works councils should be informed about how AI is used, what it does, what it cannot do and how privacy is protected. HR teams need training so they can interpret outputs, detect unusual patterns and know how to escalate concerns.
Where HR teams can start
A practical first step is to map which AI use cases already exist in the organisation and which new ones are being considered. Classifying them by impact helps determine where more structure is required. Low risk use cases can be introduced quickly. Higher impact use cases benefit from controlled pilots with close monitoring. Completing Data Protection Impact Assessments, reviewing vendors and building clear governance processes gives HR teams the confidence to work with AI in a safe and responsible way.
Watch the webinar
Our CEO and Co-founder Jos Gheerardyn was invited by Starfish Consultancy for a webinar on AI in HR. Watch the entire session ‘How can you apply AI in HR in a safe and compliant way?’ that inspired us to write this article.
