EU AI Act

Must it be indicated during a call that it is an AI system?

Currently, there is no legal obligation to explicitly identify calls as AI-powered. However, this will change with the entry into force of the EU AI Act on August 2, 2026. From that point on, it must be clear and unmistakable that end customers are interacting with an AI and not a human.

In practice, this means: At the latest during the first interaction (i.e., directly at the start of a call), it must be clearly conveyed to end customers that they are interacting with an AI. An audible notice ("You are speaking with a digital assistant" or "This is an AI-powered call") is sufficient. It's crucial that there's no risk of confusion with a real person for the average informed and attentive end customer.

Many telli customers are already voluntarily implementing this transparency measure to build trust with their end customers and meet legal requirements early.

Which use cases are permitted without issues under the EU AI Act?

In principle, all AI applications that do not pose a high risk to the fundamental rights or safety of individuals are permitted, for example:

• AI for general customer communication, e.g., scheduling appointments, reminders, or status updates.

• AI as a supporting solution for customer service employees, without making independent decisions about individuals.

• Standardized surveys without evaluating sensitive personal characteristics (e.g., simple appointment questions, availability inquiries in application processes).

Which use cases are explicitly prohibited according to the EU AI Act?

AI systems that potentially manipulate, deceive, or impair the fundamental rights of individuals are prohibited, specifically:

• Systems for "social scoring".

• AI systems that make or support decisions based on discriminatory or sensitive characteristics (e.g., ethnic origin, political views, health status).

• AI applications that intentionally employ manipulative or deceptive behavior.

Which use cases are considered "high-risk" according to the EU AI Act?

"High-risk AI" primarily includes applications that can have a significant impact on the lives of individuals. Particularly relevant here are:

• Biometric categorization and emotion recognition: e.g., AI systems that categorize individuals based on sensitive attributes or specifically recognize emotions to make decisions or control user behavior.

• Employment and recruitment procedures: e.g., automated evaluation of applicants by analyzing their responses or emotional reactions during an interview.

• Workplace relationships and employee evaluations: AI used for performance tracking, evaluation, or monitoring of employees that influences labor law consequences (e.g., dismissals, promotions).

• Access to essential private and public services: AI that influences credit allocation, insurance, or other fundamental services.

Important: AI systems that perform profiling of natural persons (e.g., automated assessment of behavior, emotions, or language) are always considered high-risk.

Under what conditions can these high-risk AI use cases still be deployed?

An AI system that formally falls under the high-risk category could still be classified as non-high-risk if the following exceptions are met (in accordance with Art. 6 Para. 3 AI Regulation):

• The AI only performs a narrowly defined procedural task, without making relevant decisions independently or having a significant impact on the person.

• The AI merely serves to improve or complement a human decision, but does not make its own autonomous decision.

• The AI only recognizes decision patterns or deviations, without replacing or directly influencing human evaluations.

• The AI only performs a preparatory task for a later human evaluation, e.g., it only collects structured information that is then evaluated by a human.

Important: Even when using an exception, an AI automatically remains high-risk if it performs direct profiling or makes relevant decisions about natural persons independently.

Practical Example: Is the use of telli in applicant management (e.g., for initial telephone pre-screening) to be classified as high-risk?

In principle, AI systems in applicant management are considered high-risk. However, telli can utilize an exception here under the following conditions:

• telli asks applicants exclusively standardized, general questions, without evaluating sensitive personal characteristics (e.g., emotions, communication style).

• telli does not make its own decisions about applicants, but only collects structured answers and makes them available to the company for human evaluation.

• The final decision regarding applicants is always made by a human, not the AI.

If these conditions are met, telli would generally not be classified as a high-risk AI for this use case.

Note: The FAQ answers are based on the current legal situation and assessments (March 2025). As the EU Commission will publish detailed guidelines by February 2026 at the latest, some details listed here may change in the future.