Evals are the new PRD, Expedia’s AI chief tells VB Transform 2026

Summary

Expedia's AI chief advocates for evaluations replacing PRDs, embedding security early. He emphasizes specialized agents over AGI and user control in bookings. The article highlights risks of automated evaluations and the need for risk-calibrated governance, warning of AI-driven threats.

Key Takeaways

1. The new PRD are the evals, Xavi Amatriain, Expedia Group’s first chief AI and data officer, told the VB Transform 2026 audience last week in Menlo Park.
2. Sixty-six percent of the 157 enterprises surveyed already permit some production deployment without human review or are building toward it within the next 12 months, yet only 5% fully trust the automated evaluations that would make that decision.
3. We don’t want the agent to book the hotel or to buy you a plane ticket for you. That’s something that the user has to have the agency. And the agent can recommend, can suggest, can discuss with you, but you’re gonna have to hit that click. And that’s non-negotiable.

Statement Breakdown

• Claimed Facts: 50% of statements the article presents as facts
• Opinions: 40% of statements classified as editorial or subjective
• Claims: 10% of statements surfaced for additional reader evaluation

Credibility & Bias Reasoning

Credibility assessment: The article presents insights from an industry expert and supports claims with survey data. However, it relies heavily on a single source's perspective and lacks direct counterarguments or independent verification of all statements.

Bias assessment: AI-Centric Innovation Advocacy. The article strongly advocates for a specific approach to AI development and governance, emphasizing the views of Expedia's AI chief. It frames these ideas as the future, potentially downplaying alternative perspectives or challenges.

Note: This article offers expert opinions on AI development. While informative, consider the strong advocacy for specific approaches and seek diverse viewpoints for a comprehensive understanding.

Credibility flag: Expert Insights, Future-Focused

Claimed Facts (9)

• This provides verifiable background information about the speaker's professional history.
• This presents specific data from a survey, presented as factual findings.
• This statistic from the VB Pulse research offers a concrete example of evaluation failures.
• This describes Expedia's stated AI governance framework.
• This explains a specific mechanism within Expedia's AI governance process.
• This details Expedia's approach to building AI systems.
• This statement describes the dynamic nature of the travel industry, providing context for AI application.
• This statistic from a separate survey provides data on AI security incidents.
• This statistic indicates industry trends in AI security tooling adoption.

Opinions (9)

• This is a declarative statement of a new paradigm, presented as a strong opinion or assertion by the speaker.
• This explains the speaker's interpretation of how evals function in product development, framing it as the standard approach.
• This is a forward-looking statement about the future of coding, presented as a strong prediction.
• This is a subjective assessment of the impact of guardrails, presented as a negative consequence.
• This expresses a personal preference and rationale for a specific approach to governance.
• This is a statement of personal belief and skepticism regarding a specific AI concept.
• This presents an alternative perspective on AI architecture, framed as a superior approach.
• This is a strong assertion about the primary focus of AI development, emphasizing system design over individual components.
• This is a predictive statement about future security threats, presented as a strong likelihood.

Claims (5)

• While guardrails can have drawbacks, stating they universally make systems 'worse off' is an oversimplification and potentially dubious without further qualification.
• Labeling guardrails as a 'necessary evil' is a strong, potentially biased framing that suggests an inherent negative quality rather than a tool with trade-offs.
• This statement acknowledges disagreement but doesn't elaborate on the counterarguments, leaving the reader to infer their validity and potentially downplaying their significance.
• The claim of being able to 'almost automate the whole cycle' of feedback loops is a strong assertion that might overlook significant human oversight and complex edge cases.
• While AI-driven attacks are a concern, stating they will be the 'next attackers' is a speculative and potentially alarmist prediction.

Key Sources

• Xavi Amatriain — Expedia Group’s Chief AI and Data Officer
• VentureBeat’s VB Pulse research — Research arm of VentureBeat
• Louis Columbus — Author, VentureBeat
• VentureBeat’s June Pulse survey — Research arm of VentureBeat