What is agentic AI in procurement, and how is it different from a chatbot?

Agentic AI performs defined tasks autonomously - supplier search, risk assessment, purchase order routing, contract drafting within set rules and without waiting for a new prompt each time. A chatbot answers questions. An agentic AI system takes actions. That operational difference is also a legal difference: actions create commitments, and commitments create liability.

Can an AI agent legally commit my company to a purchase order?

Yes. If an AI agent submits a purchase order through your procurement system and that order is confirmed, whether by a human click or by automated workflow - the company is bound. The fact that the AI generated the order is not a defense. Current law treats AI as a tool, not a person, which means the company deploying it carries full responsibility for what it does.

What happens if our AI procurement software selects the wrong supplier?

The company is liable for the outcome. If the selection criteria were biased, discriminatory, or not aligned with competition law, the exposure compounds beyond a commercial dispute into potential regulatory action. This is why documented, published, and legally reviewed decision criteria are not optional for procurement AI systems.

Can we feed supplier contracts and NDAs into ChatGPT or other public models for analysis?

You can, but you should not do so without understanding the risk. Uploading supplier contracts, pricing data, or NDA-protected information to a public model may violate the NDAs you have in place, expose personal data of supplier representatives to GDPR obligations, and create cross-border data transfer issues if the model runs on infrastructure outside the EU. The safer architecture is a private, isolated environment where data does not leave your control.

Is a Human-in-the-Loop setup enough to protect us legally?

Only if the human oversight is meaningful. A buyer clicking approve on 200 AI-generated orders per day without reviewing them does not qualify as meaningful oversight under the EU AI Act. Courts will treat the approval as valid, which means the company is bound, and the employee may face separate liability for negligence. Approval thresholds scaled to order value and actual review capacity are what make HITL a real governance control.

When does the EU AI Act apply to our procurement AI system?

It depends on how the system is classified. Prohibited AI practices have been subject to enforcement since February 2025. GPAI obligations applied from August 2025. If your system evaluates supplier credibility, assesses creditworthiness, or touches employment-adjacent processes, it likely sits in the high-risk category and high-risk obligations take effect in August 2026. That timeline is closer than most teams currently have it in their planning.

Who is liable when an AI procurement system makes a mistake - us, the vendor, or the model provider?

Primarily you. Foundation model providers disclaim liability in their standard terms. System integrators carry responsibility for how they implemented and configured the system, but that only helps if your vendor contract explicitly allocates those obligations. Without negotiated performance warranties, indemnities for incorrect outputs, and clear liability allocation, the deploying company absorbs the risk by default.

What should a CPO demand from an AI vendor before deployment?

Data isolation and a contractual prohibition on using your data for model training. Audit logs tied to source documents, not just system-level logs. Published and legally reviewed decision criteria. Human approval gates calibrated to order value. Exit rights that do not trap you in a multi-year dependency. And the ability to stop the AI if it breaches defined compliance thresholds. If a vendor says their system is too complex to audit, that is a risk signal, not a feature.

Home
Blog
Agentic AI in Procurement: The Legal Limits When AI Starts Acting

AI in Procurement

Agentic AI in Procurement: The Legal Limits When AI Starts Acting

Denis Rasulev

May 28, 2026

Agentic AI in Procurement Podcast with Denis Rasulev, Kateryna Velychko, and Oleg Pacholčuk.

Denis Rasulev at Digicode Europe brings together two practicing lawyers to examine the legal side of autonomous procurement AI. Kateryna Velychko (Code & Code Legal, IT law and AI compliance) and Oleg Pacholčuk (AK Pacholčuk, commercial and contract law) cover corporate liability when AI acts without direct human instruction, EU AI Act timelines, GDPR exposure from public AI models, and what belongs in every vendor contract before agentic AI touches a binding transaction.

When your AI agent makes a €100,000 mistake and a human clicks approve without reading
it – the company is still bound.
“The algorithm did it” is not a legal defense. It never will be.
The EU AI Act is not a future deadline. Parts of it apply now. High-risk obligations land in August 2026.
What trusted agentic AI tools in procurement actually need to include and what to put in the vendor contract.

Watch the Full Podcast

The Shift Nobody Prepared Their Legal Team For

Most procurement teams have spent the last two years getting comfortable with generative AI – tools that summarize RFPs, draft supplier emails, pull contract summaries on demand. That comfort has been, for the most part, earned. The tools work.

Agentic AI is a different category entirely. These systems do not wait for a prompt. They perceive a condition, reason through it, and act. In a procurement context, that means identifying a supplier shortfall and sourcing alternatives. Flagging a price deviation in an incoming invoice and routing it for dispute. Drafting a purchase order and submitting it through a workflow. The range of agentic AI use cases in procurement is expanding fast, but the legal framework governing it has not kept pace.

That gap is exactly where this episode starts. The companies that get agentic AI for procurement right in the next eighteen months will not just be more efficient – they will be structurally harder to compete with.

There Is No “The Algorithm Did It” Defense

This is the point everything else in the conversation builds on: current civil and commercial law does not recognize AI as a legal entity. The AI cannot be liable. The company deploying it can, and is.

More than that – in commercial law, a businessperson is treated as a professional who is presumed to understand the tools they use. Deploying a procurement agentic AI system does not lower the bar. If anything, it raises the expectation that whoever authorized the deployment understood its operational limits and governance requirements.

What this means in practice: if an AI agent hallucinates a discount, accepts an indemnity clause your legal team would never approve, or commits to a delivery schedule your operations cannot meet – the company is bound. The fact that no human intended that outcome is not relevant.

The EU AI Act Timeline Is Not What Most Companies Think It Is

The timeline is more immediate than most procurement teams currently have it. A large number of procurement teams are treating the EU AI Act as a future problem. It is not.

Here is what the actual timeline looks like:

Prohibited AI practices: obligations in effect since February 2025
General-purpose AI (GPAI) governance: applicable from August 2025
High-risk AI system obligations: August 2026, which is the next deadline most procurement teams will hit

The high-risk classification is where most organizations underestimate their exposure. If your AI system evaluates suppliers, scores creditworthiness, or touches any employment-adjacent process: it likely sits in the high-risk category. That triggers documentation requirements, transparency obligations, oversight mandates, and technical robustness standards that need to be built in before go-live, not added later.

For anyone currently evaluating trusted agentic AI tools in procurement, August 2026 is not a long runway.

The Black Box Problem Is a Legal Problem

Procurement directors use the term “black box” a lot. What they mean is: the system made a decision and I cannot tell you why. Operationally, that is a problem. Legally, it is a different order of risk entirely.

Under the EU AI Act, high-risk systems are required to maintain logs sufficient for post-deployment audit. This draws an important distinction that does not appear enough in the conversation around AI explainability: a system producing a technical rationale is not the same as a company having a defensible decision.

Defensibility means the company can demonstrate, in a regulatory hearing or a supplier dispute, that the decision criteria were lawful, published, aligned with competition rules, and applied consistently. That is a higher bar than “the model provided a confidence score.”

The decision criteria the AI operates on must be set by a human, made explicit, and documented. If the AI selects Supplier A over Supplier B and the reason cannot be articulated in terms a court or regulator would accept – the company has a problem, regardless of how technically sophisticated the model is.

Feeding Sensitive Procurement Data into Public Models Is a GDPR Risk

This is the section of the episode that tends to produce the most uncomfortable recognition in the room when it is played to procurement or legal teams.

First, feeding sensitive procurement data into a public AI model creates two distinct legal problems, and most organizations are walking into both. This directly violates the majority of NDAs that procurement teams have in place with their suppliers.

Second, that data almost always contains personal data of supplier representatives. Which means GDPR applies. And if the model infrastructure sits outside the EU or EEA, you have a cross-border transfer obligation on top of that.

The answer is not architecturally complex, but it requires deliberate choices from the start. And the vendor contract has to say so explicitly – not in a general terms document, but as a specific obligation.

His point on what a CPO should demand from any AI vendor is one of the most quotable moments in the episode:

And if we have an AI provider who claims that their solution is too complex for us to explain it, then they are essentially not selling you a solution, but liability.

– Oleg Pacholčuk, AK Pacholčuk

Human-in-the-Loop Is Not a Safety Net. It Is a Responsibility.

Consider the scenario: a buyer receives 200 AI-generated purchase orders per day. Their job is to click approve. One of those orders contains an error that costs the company €100,000.

Can the company argue that a human was in the loop?

The answer is precise. It will be treated in court as a human approval. The company is bound by the commitment. And there is a second consequence: the employee who clicked approve without reading now faces potential liability for negligence – the company can pursue that claim up to statutory limits. Both things are true at the same time.

The EU AI Act uses the phrase “meaningful human oversight” for high-risk systems. Meaningful means the reviewer had enough time, enough context, and actual authority to stop the action. A buyer processing 200 orders a day with a one-click interface does not meet that standard. The human is present in form but not in function. That distinction has a name in compliance: automation bias.

The practical fix is straightforward in principle: approval thresholds scaled to order value and risk level. Below a defined amount, automatic processing. Above it, genuine review. But the threshold has to be set intentionally, documented, and enforced – not left to convention.

If we cannot audit the AI, we should not deploy it. First should come transparency and only then efficiency.

– Kateryna Velychko, Code & Code Legal

The Liability Chain Most Companies Have Not Mapped

When something goes wrong with an agentic AI procurement system, there are typically four parties in the frame: the deploying company, the AI vendor (system integrator), the foundation model provider, and sometimes a separate implementation partner.

The foundation model providers – OpenAI, Anthropic, Google, Microsoft – disclaim liability in their standard terms. Comprehensively. That is not a criticism; it reflects how these systems are built. But it means the risk does not disappear, it moves down the chain to the system integrator and, ultimately, to the deploying company.

From a contract law standpoint, there are specific obligations that belong in every AI vendor agreement: push for performance warranties tied to defined SLAs, indemnities that cover IP infringement and hallucination-driven outputs, clear allocation of what happens when the system makes an incorrect decision, and exit rights that do not require eighteen months of notice.

There is now a market for AI liability insurance. The question is whether it actually pays out: products from Munich Re, Armilla, Vouch and others are now real and increasingly structured. The caveat is important: insurers examine whether the company fulfilled its regulatory obligations before paying out. If meaningful human oversight was absent, that is gross negligence. Gross negligence voids coverage.

What to Actually Do on Monday Morning

The practical advice converges on a diagnostic question to start with: do you know how your AI tool makes decisions?

Not in general terms, specifically. Is the logic documented, and is it defensible against the procurement criteria your company has published? From there, three steps:

Involve legal and compliance before deployment: not as a checkpoint at the end, but as architects of the process from the start.
Test the audit trail before you test the AI’s intelligence. If you cannot trace why a decision was made, the system is not ready to go live regardless of how well it performs on other metrics.
Run an AI risk assessment to understand where your system sits in the EU AI Act’s risk pyramid before August 2026 obligations land.

For organizations working through what responsible deployment of agentic ai in procurement looks like structurally, the AI-Powered Procurement overview covers the architecture decisions that govern compliance, auditability, and data isolation in practice.

At Digicode, we build procurement AI systems with isolated environments, audit logs tied to source documents, and human approval gates at every legally binding step because those are not optional features, they are what makes the architecture defensible. If your team is working through these same questions, mydigicode.com is where that conversation starts.

See what a defensible, audit-ready procurement AI architecture actually looks like in practice

Book Your 30-Minute Expert Session

FAQ

What is agentic AI in procurement, and how is it different from a chatbot?

Agentic AI performs defined tasks autonomously – supplier search, risk assessment, purchase order routing, contract drafting within set rules and without waiting for a new prompt each time. A chatbot answers questions. An agentic AI system takes actions. That operational difference is also a legal difference: actions create commitments, and commitments create liability.
Can an AI agent legally commit my company to a purchase order?

Yes. If an AI agent submits a purchase order through your procurement system and that order is confirmed, whether by a human click or by automated workflow – the company is bound. The fact that the AI generated the order is not a defense. Current law treats AI as a tool, not a person, which means the company deploying it carries full responsibility for what it does.
What happens if our AI procurement software selects the wrong supplier?

The company is liable for the outcome. If the selection criteria were biased, discriminatory, or not aligned with competition law, the exposure compounds beyond a commercial dispute into potential regulatory action. This is why documented, published, and legally reviewed decision criteria are not optional for procurement AI systems.
Can we feed supplier contracts and NDAs into ChatGPT or other public models for analysis?

You can, but you should not do so without understanding the risk. Uploading supplier contracts, pricing data, or NDA-protected information to a public model may violate the NDAs you have in place, expose personal data of supplier representatives to GDPR obligations, and create cross-border data transfer issues if the model runs on infrastructure outside the EU. The safer architecture is a private, isolated environment where data does not leave your control.
Is a Human-in-the-Loop setup enough to protect us legally?

Only if the human oversight is meaningful. A buyer clicking approve on 200 AI-generated orders per day without reviewing them does not qualify as meaningful oversight under the EU AI Act. Courts will treat the approval as valid, which means the company is bound, and the employee may face separate liability for negligence. Approval thresholds scaled to order value and actual review capacity are what make HITL a real governance control.
When does the EU AI Act apply to our procurement AI system?

It depends on how the system is classified. Prohibited AI practices have been subject to enforcement since February 2025. GPAI obligations applied from August 2025. If your system evaluates supplier credibility, assesses creditworthiness, or touches employment-adjacent processes, it likely sits in the high-risk category and high-risk obligations take effect in August 2026. That timeline is closer than most teams currently have it in their planning.
Who is liable when an AI procurement system makes a mistake – us, the vendor, or the model provider?

Primarily you. Foundation model providers disclaim liability in their standard terms. System integrators carry responsibility for how they implemented and configured the system, but that only helps if your vendor contract explicitly allocates those obligations. Without negotiated performance warranties, indemnities for incorrect outputs, and clear liability allocation, the deploying company absorbs the risk by default.
What should a CPO demand from an AI vendor before deployment?

Data isolation and a contractual prohibition on using your data for model training. Audit logs tied to source documents, not just system-level logs. Published and legally reviewed decision criteria. Human approval gates calibrated to order value. Exit rights that do not trap you in a multi-year dependency. And the ability to stop the AI if it breaches defined compliance thresholds. If a vendor says their system is too complex to audit, that is a risk signal, not a feature.