Watching America Run Away With AI - Alistair Pullen (Cosine AI)

1. Alistair Pullen: The Sovereign AI Mandate

Timestamp: 00:00:00 to 00:04:02 - watch this moment on skim

Cosine AI has secured a mandate to build the UK's first sovereign LLM, leveraging compute resources on the Isambard supercomputer. This initiative was partly spurred by the ban on their previous model, Fable, highlighting the geopolitical importance of domestic AI capabilities. The company aims to achieve this ambitious vision with significantly less capital than US-based AI giants by focusing on an 'inference company' model.

Significance (High): This point establishes the core mission of Cosine AI and frames the UK's AI development within a geopolitical context. It sets the stage for the discussion on how smaller entities can compete in the AI race.

Sources in support: Alistair Pullen (CEO and Co-founder of Cosine AI)

Neutral sources: Tim Scarfe (Host)

2. The 'Inference Company' Model vs. Billions

Timestamp: 00:04:02 to 00:07:19 - watch this moment on skim

Pullen argues that an 'inference company' model, which licenses technology rather than hosting models and charging per token, requires millions, not billions, to compete. This approach avoids the massive inference costs faced by companies like Anthropic, allowing Cosine to focus resources on training and development using national compute allocations like Isambard.

Significance (High): This presents a compelling economic argument for challenging AI incumbents. It suggests a strategic pivot that could democratize access to advanced AI capabilities by reducing the capital expenditure required for deployment.

Sources in support: Alistair Pullen (CEO and Co-founder of Cosine AI)

Neutral sources: Tim Scarfe (Host)

3. Open Models Lagging Frontier Performance

Timestamp: 00:07:40 to 00:11:01 - watch this moment on skim

Pullen suggests that open-weight models from companies like Mistral and DeepSeek are not yet competitive with frontier closed-source models. He attributes this gap to architectural choices, particularly the trade-off between total parameter count and active parameters, and the data strategies employed by leading labs.

Significance (Medium): This assertion challenges the current state of open-source AI, implying that significant advancements in architecture and data curation by major labs create a performance chasm that open models struggle to bridge.

Sources in support: Alistair Pullen (CEO and Co-founder of Cosine AI)

Neutral sources: Tim Scarfe (Host)

4. Active Parameters and Data: The Real Differentiators

Timestamp: 00:14:59 to 00:18:54 - watch this moment on skim

The discussion highlights that active parameter count, rather than just total parameters, significantly impacts model performance. Furthermore, while pre-training corpora might be similar across large models, post-training data curation and the scale of Reinforcement Learning (RL) are critical differentiators for frontier models, enabling better generalization and alignment.

Significance (High): This point demystifies AI performance, shifting focus from sheer size to more nuanced factors like active parameters and sophisticated data strategies. It suggests that true AI advancement lies in refinement and targeted training, not just raw scale.

Sources in support: Alistair Pullen (CEO and Co-founder of Cosine AI)

Neutral sources: Tim Scarfe (Host)

5. Alistair Pullen on Combating 'Slop' in AI Code

Timestamp: 00:19:48 to 00:23:21 - watch this moment on skim

Pullen identifies 'slop' – code that passes tests but is inefficient, overly complex, or poorly structured – as a major issue in AI-generated software. He advocates for rewarding the 'process' of code generation, not just the final output, and using real-world user interaction data (trajectories) to train models on how users actually prompt and refine code, leading to more robust and maintainable solutions.

Significance (High): This addresses a critical practical problem in AI-assisted development. By focusing on the quality of the development process and learning from user interactions, Cosine aims to produce AI that generates genuinely useful and well-engineered code, not just functional but messy solutions.

Sources in support: Alistair Pullen (CEO and Co-founder of Cosine AI)

Neutral sources: Tim Scarfe (Host)

6. Alistair Pullen: The 'Fable' Ban and Sovereign AI

Timestamp: 00:23:24 to 00:27:36 - watch this moment on skim

The development of Cosine's Fable model was directly spurred by US export controls on advanced AI, creating a 'sovereign mandate' for the UK to build its own capable coding AI. This geopolitical necessity became the founding case for a UK-based, sovereign model trained on national supercomputing resources like Isambard-AI. The core belief is that an inference-focused company, rather than a training-first lab, can compete effectively with significantly less capital, provided it has access to national compute allocations and a collaborative feedback loop. This strategic positioning aims to circumvent restrictive export policies and foster domestic AI capabilities. The journey underscores the tension between global AI advancement and national security concerns, driving innovation within specific geopolitical boundaries. Ultimately, the goal is to establish a robust, independent AI ecosystem.

Significance (High): This point establishes the foundational motivation for Cosine's work, framing it as a response to external restrictions and a strategic move towards national AI sovereignty. It highlights the geopolitical underpinnings of AI development and the potential for such pressures to drive innovation.

Sources in support: Alistair Pullen (CEO and Co-founder of Cosine AI)

Neutral sources: Tim Scarfe (Host)

7. The Epistemic Gap and Verifiable Rewards

Timestamp: 00:28:00 to 00:29:56 - watch this moment on skim

A significant challenge in AI development, particularly outside of coding and mathematics, is the 'epistemic problem' – the difficulty in determining true correctness or falsehood. While coding offers verifiable rewards through execution feedback, many domains lack such clear metrics. Alistair Pullen acknowledges this gap, noting that even with advanced AI, we don't always 'know whether it was correct or not.' He suggests that while static compilation for verification isn't feasible in all fields, bringing everything into distribution and using human judges or well-prompted LLMs can approximate correctness. However, he posits that for many problem sets, enumerating and ensuring model proficiency at train time remains crucial, as true generalization across all tasks is elusive. This highlights the ongoing need for robust evaluation methods beyond simple accuracy metrics.

Significance (High): This point exposes a core limitation in AI's ability to truly 'understand' or verify its outputs, especially in subjective or complex domains. It underscores the ongoing reliance on human judgment and the difficulty of achieving genuine, domain-agnostic intelligence.

Sources in support: Alistair Pullen (CEO and Co-founder of Cosine AI)

Neutral sources: Tim Scarfe (Host)

8. Runtime Proof: Code Review as Execution

Timestamp: 00:29:56 to 00:31:36 - watch this moment on skim

Pullen proposes reframing code review as 'runtime proof,' where an agent's generated code is not just statically analyzed but actively executed in a sandboxed environment to verify its behavior and security. This method aims to catch subtle bugs and vulnerabilities that traditional reviews might miss, ensuring the generated code is truly reliable.

Significance (High): This innovative approach to code validation could significantly enhance the trustworthiness of AI-generated code. By treating execution as the ultimate proof, it moves beyond theoretical correctness to practical, demonstrable reliability.

Sources in support: Alistair Pullen (CEO and Co-founder of Cosine AI)

Neutral sources: Tim Scarfe (Host)

9. Agentic Harnesses: Evolving Importance and Sub-Agent Orchestration

Timestamp: 00:37:43 to 00:41:41 - watch this moment on skim

Alistair Pullen initially suggests that agentic harnesses are becoming less important as models improve, capable of performing tasks with just bash commands. However, he quickly pivots, emphasizing that harness engineering remains critical, particularly for efficiency and managing complexity. He highlights the game-changing impact of sub-agents, arguing that complex problems are too large for a single LLM pass. By decomposing problems into smaller, agentic subtasks, each with a clear specification, context rot is avoided. Cosine's 'Swarm' system represents an extreme of this, orchestrating hundreds of sub-agents hierarchically to tackle highly complex projects like building a mechanical watch compiler, a task beyond the capability of even top-tier models in a single pass. This approach allows for greater specialization and parallel processing, overcoming the limitations of monolithic agent execution.

Significance (High): This discussion reveals a nuanced view on the role of agentic harnesses, moving from a potential obsolescence to a critical component for managing complex AI tasks. The 'Swarm' system exemplifies how sophisticated orchestration can unlock capabilities previously unattainable by individual models.

Sources in support: Alistair Pullen (CEO and Co-founder of Cosine AI)

Neutral sources: Tim Scarfe (Host)

10. Memory: The Unsolved Hack in AI

Timestamp: 00:45:15 to 00:47:29 - watch this moment on skim

Alistair Pullen describes memory as a 'very hard' problem in AI, likening current approaches to 'hacks' rather than elegant solutions. Existing methods, often relying on VectorDBs or embedded knowledge snippets, suffer from agents' difficulty in knowing when to query, whether information is useful enough to store, and how to keep it updated. He recounts instances where outdated memory led agents astray, causing them to deviate from optimal paths. While Cosine is exploring continual learning to integrate memory into the latent space, Pullen acknowledges this is also challenging. He suggests that memory being a tool, rather than an intrinsic part of the model, makes it difficult to manage, especially during RL training where its vast surface area presents significant risks for reward manipulation.

Significance (High): This candid admission highlights a critical bottleneck in AI development. The difficulty in implementing effective, reliable memory systems suggests that true long-term reasoning and contextual understanding in AI remain significant hurdles, impacting the AI's ability to learn and adapt consistently.

Sources in support: Alistair Pullen (CEO and Co-founder of Cosine AI)

Neutral sources: Tim Scarfe (Host)

11. Alistair Pullen: The Fable Ban as a Catalyst

Timestamp: 00:47:00 to 00:49:18 - watch this moment on skim

The US ban on exporting advanced AI models like Fable inadvertently created a powerful impetus for the UK to develop its own sovereign AI capabilities. This geopolitical pressure, rather than being a hindrance, has become a foundational case for Cosine AI's mission and has galvanized support for national AI initiatives.

Significance (High): This framing positions geopolitical restrictions as a strategic advantage, highlighting the growing importance of national AI sovereignty. It suggests a shift in the global AI landscape where regional capabilities are becoming paramount.

Sources in support: Alistair Pullen (CEO and Co-founder of Cosine AI)

Neutral sources: Tim Scarfe (Host)

12. Cosine AI: The Inference-First Economic Model

Timestamp: 00:49:18 to 00:50:52 - watch this moment on skim

Pullen argues that an 'inference company' model, rather than a training-first approach, can compete effectively with billions-dollar labs using millions, national compute allocations, and consortium feedback loops. This economic bet suggests a more accessible path to advanced AI development.

Significance (High): This challenges the conventional wisdom that massive capital is the sole determinant of AI success, proposing a more efficient and potentially democratized model for AI development. It implies that strategic resource allocation and collaborative efforts can yield competitive results.

Sources in support: Alistair Pullen (CEO and Co-founder of Cosine AI)

Neutral sources: Tim Scarfe (Host)

13. Rewarding Process Over Outcome for Trustworthy Agents

Timestamp: 00:50:52 to 00:52:31 - watch this moment on skim

To combat 'slop' and enhance AI agent trustworthiness, Pullen advocates for rewarding the process an agent takes, not just the final answer. This can be achieved through methods like reframing code review as runtime proof, where the agent must demonstrate its solution in a controlled environment.

Significance (High): This represents a significant paradigm shift in AI evaluation, moving beyond simple output validation to assessing the reasoning and methodology. It addresses a critical gap in current AI capabilities, aiming for more reliable and understandable AI behavior.

Sources in support: Alistair Pullen (CEO and Co-founder of Cosine AI)

Neutral sources: Tim Scarfe (Host)

14. US Export Controls: An Accidental Gift?

Timestamp: 00:53:09 to 00:53:59 - watch this moment on skim

Pullen views US export controls on advanced AI technology as an 'accidental gift' that has spurred the development of sovereign AI initiatives in the UK and elsewhere. While posing challenges, these restrictions have forced a focus on building domestic capabilities, potentially leading to a more distributed and resilient global AI ecosystem, albeit with a potential supply-chain sting in the tail.

Significance (High): This provocative framing suggests that geopolitical restrictions, often seen as barriers, can paradoxically foster innovation and self-sufficiency. It highlights the strategic implications of AI development and the potential for global shifts in technological leadership.

Sources in support: Alistair Pullen (CEO and Co-founder of Cosine AI)

Neutral sources: Tim Scarfe (Host)

CEO Promoting Own Company's Vision (Medium severity)

Type: Commercial

Alistair Pullen, as CEO of Cosine AI, is advocating for the viability and importance of their sovereign AI approach and the 'inference company' model. This commercial interest inherently biases his presentation towards highlighting the strengths and potential of his company's strategy.

Significance: Pullen's position as CEO means his narrative is intrinsically tied to the success of Cosine AI. While he presents technical arguments, the audience must consider that his primary goal is to promote his company's vision and technology, potentially downplaying alternative approaches or challenges that could hinder Cosine's growth.

Commercial Interest in AI Development (Medium severity)

Type: Commercial

Alistair Pullen, as CEO of Cosine, has a direct commercial interest in promoting their AI models and technologies, such as Fable and Swarm. This financial stake could influence their presentation of the technology's capabilities and benefits.

Significance: The audience must consider that Pullen's advocacy for specific AI approaches and architectures, like their 'inference-centric' model and 'Swarm' system, is intrinsically tied to Cosine's business success. This doesn't invalidate his points but warrants scrutiny for potential overstatement of advantages or downplaying of limitations.

CEO Promoting Sovereign AI (High severity)

Type: Commercial

Alistair Pullen, as CEO of Cosine AI, is directly promoting the development and adoption of sovereign AI models, which is central to his company's business model.

Significance: This direct commercial interest means Pullen's arguments for the necessity and viability of sovereign AI, especially in the UK, must be viewed through the lens of his company's strategic goals. The audience is left to question whether the emphasis on sovereign AI is driven purely by technical merit or by a calculated business strategy to capitalize on geopolitical tensions.