The European Union Artificial Intelligence Act (EU AI Act) is one of the first — and now one of the most mature — cross-sector AI regulations by a major regulator anywhere in the world.

The EU AI Act was originally proposed in 2021 and was approved in May 2024 after three years of drafts, amendments, and public consultation. The goals of the EU AI Act are to:

• Regulate AI capabilities according to risk.
• Standardize “high-risk” AI development and deployment.
• Formally designate accountability and liability to AI “providers” (organizations that develop AI tools) and "deployers" (people or organizations that deploy an AI system in a professional capacity, not individual end users).
• Set rules for general purpose AI models, or tools that can perform multiple functions, including those they weren't originally designed to perform.

The EU AI Act covers AI applications across all sectors including healthcare, medtech, life sciences, and pharmacology, but does not establish distinct rules for each sector.

AI policy has been introduced or proposed over the past three years in places such as California,¹ Colorado,² China,³ and Canada,⁴ although none of these policies currently reach the scale and global impact of the EU AI Act. Business leaders and regulators worldwide are paying close attention to the EU AI Act and its repercussions for two reasons.

Firstly, because the EU AI Act was proposed in 2021 and drafting and negotiation involved every member state, the EU AI Act is more mature than most other AI regulations. As such, many leaders view it as a precursor to future regulations in other markets. Healthcare and technology leaders are watching the EU AI Act in case it meaningfully influences or informs global healthcare technology regulation, much like the EU’s General Data Protection Regulation (GDPR) and Medical Device Regulation (MDR) have done in the past.

Second, the EU AI Act applies to AI products developed outside the EU that are utilized within its borders, meaning the regulation will affect companies or products that operate globally.

The EU AI Act broadly defines AI as encompassing everything from machine learning to large language models (LLMs), a type of AI that has been trained on vast amounts of text to understand existing content and generate original content. The EU AI Act assigns AI applications one of four risk categories:⁵

1. Unacceptable risk applications are entirely banned. These are applications that contradict the EU’s values for human equality, freedom, dignity, democracy, and the rule of law.
2. High risk applications are subject to specific legal and compliance requirements because they pose harm to health, safety, and human rights. Many healthcare tools will likely fit in this category.
3. Limited risk applications are those that interact directly with humans and therefore need transparency verification.
4. Minimal risk applications are all other applications that are largely left unregulated.

The EU AI Act entered into force on August 2, 2024, with bans for prohibited systems applied from February 2, 2025. Compliance for high-risk AI systems and most other systems will apply from August 2, 2026.

EU AI Act holds both AI developers, called “providers,” and AI users, called “deployers,” liable for breaching regulatory requirements. For example, if a hospital develops or uses an in-house, high-risk AI tool that doesn’t meet the compliance requirements set out under the EU AI Act, then that organization will be penalized. If a hospital deploys a product that a vendor provides, and that product does not comply with regulation, both the hospital and the vendor will be subject to noncompliance penalties. The current noncompliance penalty can reach €35 million or 7% of annual gross revenue, whichever is higher, and applies to both developers and providers who breach the EU AI Act’s terms.

The magnitude and formality of this liability has added urgency for AI providers and deployers across the EU to address specific AI challenges such as governance, safety and ethics, and vendor relationships.

How does the EU AI Act affect high-risk AI tools in healthcare?

Healthcare data is highly vulnerable because it often includes sensitive information such as medical histories, diagnoses, and financial details. Improper AI use within the sector can lead to potentially harmful effects including data privacy breaches, outcome or process biases, and inaccurate, unreliable, or even harmful recommendations. Because of this, the EU AI Act is likely to consider many healthcare AI applications as high-risk.⁶

The EU AI Act subjects high-risk AI tools to additional layers of scrutiny and regulation.⁷ AI providers, whether external vendors or healthcare organizations, must follow a five-step process when developing and employing high-risk tools:⁸

1. AI provider develops a high-risk AI tool.
2. The provider or a third-party organization must conduct a conformity assessment. This assessment involves ensuring the tool’s quality management aligns with the EU AI Act, reviewing and confirming technical documentation meets the EU AI Act’s requirements, and verifying consistency between the tool’s documentation and design and post-market monitoring systems. This conformity assessment verifies the AI tool's compliance with the EU AI Act's seven requirements: risk management, data governance, technical documentation, record-keeping, transparency and provision of information, human oversight, as well as accuracy, robustness, and cybersecurity.
3. After successfully passing the conformity assessment, the provider must register the AI tool in an EU database before deploying or marketing it.
4. The AI provider must create a signed EU declaration of conformity for the AI tool and affix a CE marking to demonstrate compliance with EU standards.
5. The AI tool can be used or put on the market. If product lifecycle changes violate the EU AI Act's requirements, the provider must stop using the tool or remove it from the market and revisit step 2. AI providers and deployers must conduct post-market monitoring and continuously ensure human oversight of AI tools.

The European Commission established the AI Office to monitor, supervise, and ensure compliance with the EU AI Act across the member countries. The AI Office will employ over 140 staff including experts in technology, administration, law, policy, and finance. The AI office will also consist of five units that reflect its mandate, such as the “regulation and compliance” unit.

A common question that has arisen since the EU AI Act proposal is, “How does the EU AI Act affect LLMs?” This question is not surprising. In healthcare, AI developers have already trained LLMs on extensive literature, research, and clinical datasets to understand and solve pervasive medical or operational challenges. LLM investment is substantial and it’s still growing. The global LLM market is set to reach a value of $260 million by 2030, with an 80% compound annual growth rate (CAGR).⁹ Many healthcare organizations around the world have also increased their spend on LLMs — according to one Gradient Flow study, 56% of U.S. healthcare providers increased their LLM budget by 10 to 100% across 2024.¹⁰

LLMs are at risk of noncompliance, but developers are finding ways to make products comply

In the EU, the AI Act and AI Office treat GPAI models like LLMs as high-risk tools. The tools are therefore exposed to heightened regulatory scrutiny, and AI developers must provide their training datasets to the AI office. But LLMs are trained on enormous datasets that are difficult to verify and explain. The potential opacity of these datasets could therefore preclude regulatory approval.

One representative example is Ada Health,¹¹ an LLM-driven, symptom checking chatbot app from Germany. Ada is certified under the EU Medical Device Regulation, which uses similar regulatory instruments to the EU AI Act, with both using risk-based regulation. Ada Health obtained this certification two years early by adopting strict quality principles, such as using retrieval augmentation generation to keep its outputs free from hallucinations (false or misleading output as a result of processing issues or incomplete training data). This means the AI references an authoritative and verified knowledge base outside of its training datasets before it generates a final response. The tool also explains its outputs to the end user and doesn’t sell any data or track cookies. Early-approved tools like Ada potentially chart a path for future healthcare LLM adoption.

AI opportunities in healthcare are multiplying as tools promise to provide powerful solutions to evergreen challenges like unsustainable clinical workloads, inefficient scheduling, and drug discovery. But AI use in the healthcare sector — which uses vulnerable patient data and has the potential for real-life consequences for staff and patients — has thus far been largely unregulated globally. Governments are increasingly under public pressure to regulate AI and ensure safe, equitable, controllable AI utilization across healthcare.

The EU AI Act provides governments around the world with a blueprint that they can use to inform their own regulations. Further, the EU AI Act’s rollout can offer “on the ground” healthcare leaders a glimpse at the signals and implications to prepare for, should any reader’s home jurisdiction follow a similar policy approach to regulating AI.

The three major implications of the EU AI Act that AI providers and deployers should monitor as they prepare for future regulations in their own jurisdictions are:

1. Additional financial exposure
2. Inflexible compliance processes
3. Increased market and public scrutiny

Implication 1: Additional financial exposure

The EU AI Act exposes AI providers and deployers to additional legal, quality control, and administrative costs beyond the noncompliance penalties previously discussed. For example, compliance processes and conformity assessments for high-risk AI tools cost providers an additional €9,500 to €14,500 per tool.¹² This figure excludes external legal advice, audit, or consultancy fees. Additionally, high-risk AI providers must have a quality management system in place, costing as much as €330,000 to establish.¹² Even without additional compliance expenses, training an existing healthcare AI-based app costs an average of €38,000, and developing a comprehensive, custom tool can far exceed €100,000.¹³ These costs significantly impact the investment (and return on investment) calculation for both AI providers and buyers.

Implication 2: Inflexible compliance processes

Although 69% of physicians think AI may help workflow efficiency,¹⁴ deploying high-risk healthcare AI tools in a tightly regulated environment may limit the benefit these tools have on workflow improvements. With many AI tools now classified as high-risk or limited-risk, AI providers and deployers must undertake workload-intensive compliance checks, conformity assessments, and publish training data to confirm their legality. The EU AI Act also adds these additional regulatory layers to AI tools under development, which delays their market entry and introduces financial risks for tools requiring redevelopment due to noncompliance. AI deployers must also establish new levels of AI governance levels and processes to ensure the tools they purchase meet EU AI Act standards and maintain safe, ethical use with comprehensive human oversight.

Implication 3: Increased market and public scrutiny

Noncompliance with the EU AI Act can severely damage an AI provider and deployer’s reputation. Regulation breaches in healthcare will breed mistrust from the public, especially with 60%¹⁵ of patients already feeling uncomfortable with healthcare providers using AI in their treatment. Bigger picture, 62%¹⁶ of patients ranked hospital reputation as the most important determinant for selecting a healthcare provider, so providers must comply with regulations to avoid eroding confidence or losing patient loyalty.

On the developer side, AI market confidence is highly volatile and AI providers risk rapid valuation drops if users discover biases or errors in their AI tools — Alphabet lost $100 billion in a single day in 2023 after the public identified an error with their AI chatbot in a promotional video.¹⁷ The volatility that affects market confidence also applies to regulation breaches.

Overall, the EU AI Act and regulations that follow from it are poised to amplify the public and market-facing risks that AI tools and users currently face.

The complexity and rapid development of the AI market often outpaces healthcare organizations’ abilities to keep up, challenging them to determine how to react to or prepare for AI regulation. Below are conversations both AI providers and deployers should be having to set their AI tools up for potential regulatory compliance regardless of jurisdiction.

• Audit your exiting AI capabilities to ensure they comply with upcoming regulations or follow a certified ISO standard (a standard published by the International Organization for Standardization) that aligns with existing regulations elsewhere.
• Establish a multistakeholder AI governance structure that stringently controls AI use and procurement and engages all levels of leadership, staff, and end users.
• Commit to a standardized process to ensure safe, equitable, and responsible AI development and use across your organization.
• Create a standardized vendor/customer evaluation process to ensure all tools you buy or sell comply with regulations.
• Engage patients and staff in every step of the AI development and implementation journey, ensuring full process and use case transparency.