94% CAGR: What the Inference Boom means for your AI costs | Vamshi Ambati
Siddhartha Ahluwalia Questions Compute Accessibility
Siddhartha Ahluwalia questions the notion that building AI applications has become cheap, pointing out that access to the necessary compute power is still limited and controlled by a few providers. He suggests that as models improve and consume more tokens, costs could increase, limiting accessibility.
Vamshi Ambati: Inference Explained Through Analogy
Inference is akin to the human brain processing inputs and making decisions after a lifetime of learning (training). For LLMs, it's the forward pass computation that predicts the next word based on learned parameters, a process that becomes more expensive with larger, more capable models.
Vamshi Ambati: The Overhyped Agents
AI agents are currently overhyped in the short term, with excessive buzz around their immediate transformative potential. While they will likely become integral to daily work in the next five years, the current hype suggests a premature expectation of companies being run entirely by agents without human oversight.
Vamshi Ambati's Accidental Entrepreneurship
Vamshi Ambati's entrepreneurial journey began accidentally, driven by a desire to test his potential and apply data science to impactful areas like healthcare. This led him to quit his job and immerse himself in a hospital for three months to gain firsthand understanding, a move he now views as 'stupid' but essential for his growth.
Healthcare's Resistance to Tech Adoption
Healthcare's perceived slowness in adopting technology stems from a complex ecosystem involving providers, patients, pharma, and payers, each with distinct incentives and regulatory frameworks. Understanding the roles of entities like the Chief Administrative Officer and the nuances of revenue cycles is crucial for effective tech integration, which is often misunderstood by outsiders.
From Services to Product: The Predera Pivot
After initial success in healthcare services, Vamshi Ambati pivoted Predera to a product-only company in 2019, focusing on MLOps and later LLMOps. This transition, though challenging, was driven by identifying common patterns across verticals and optimizing for product value, culminating in an exit strategy that valued the LLMOps platform significantly.
Vamshi Ambati: The Forward-Deployed Advantage
Founders should consider a 'forward-deployed' approach, embedding closely with customers to deeply understand problems before building solutions. This method, exemplified by Palantir's model, builds trust and authenticity, allowing for tailored solutions that address specific enterprise needs, rather than generic product offerings.
Landing Enterprise Customers: Trust and Tenacity
Securing large enterprise clients requires building trust through authenticity and demonstrating expertise in specific problem areas, rather than relying on broad pitches. Persistence, like the nine-month effort to land Walmart, is crucial, especially when approaching top-tier companies with a well-defined, high-value offering.
Vamshi Ambati: The Sales Hustle
Closing enterprise deals requires more than just technical expertise; it demands relentless follow-up, keen observation of client pain points, and a willingness to adapt sales strategies. Ambati emphasizes that learning from failed deals is more instructive than celebrating wins, highlighting the importance of self-criticism in developing sales muscles. This iterative process of understanding and adapting is crucial for long-term success in the services firm.
