At Novita, we organized our SF Tech Week Panel around a question we hear often from developers: what does it take to move from an open model to a dependable AI product? We wanted to make space for a practical conversation about the layers between a model checkpoint and a working product, and to bring the people building those layers into the same room.
On October 7, we brought that conversation to AWS Builder Loft in downtown San Francisco for Open Source AI Stack: Models, Inference, and Agent Harness. More than 400 people joined us, making the panel one of the venue’s top three most-attended events of the year. The turnout brought together developers, researchers, infrastructure teams, founders, and AI enthusiasts who were looking for more than a high-level overview of the open AI ecosystem.
Panelists and attendees gather during the SF Tech Week Panel.
Why We Brought These Topics Together
The reason for the panel was simple: open AI products are shaped by much more than models alone. The developer experience also depends on how models are evaluated, served, connected to tools, and operated in production. Those decisions are closely connected, but they are often discussed separately.
We wanted to put those layers into one conversation. That meant inviting people who approach the stack from different angles, including model and post-training work, inference benchmarking, agent systems, AI developer tooling, and the infrastructure that connects them.
The result felt practical from the start. People came to hear from the panel, but they also came to compare notes with the builders around them. During the Q&A and networking that followed, the same questions kept resurfacing: how should teams think about performance, reliability, and cost, and which choices help an AI system hold up beyond a demo?
The People Behind the Discussion
We were grateful to welcome guests who represented several parts of that ecosystem. The event program and our public speaker materials confirmed the following roles:
- Akshay Deshpande, Member of Technical Staff at Artificial Analysis, where he leads inference benchmarking across model providers.
- Sam Herring, ML Engineering Lead at Nous Research, with a focus on post-training.
- Walter Korman, Tech Lead at Vercel for the AI Gateway and AI SDK products.
- Mao Cheng, Member of Technical Staff at RadixArk.
Panelists discuss the open AI stack in front of a live audience.
Four Connected Technical Themes
As the conversation moved between speakers and audience questions, four themes kept connecting back to one another.
Open models
Open models give developers more room to inspect, adapt, and build. The checkpoint, however, is only the beginning. Teams still need evaluation, serving, safety practices, and a path from experimentation to a product that people can rely on.
Inference
Inference is where model capability meets the constraints of an application. Latency, throughput, cost, and reliability all influence what developers can offer users, which is why inference benchmarking and model evaluation belong in the same conversation as model development.
Agents
Agents make the connections between models, tools, and runtime systems visible. A useful agent needs more than a capable model; it also needs tool access, state management, guardrails, and a way to recover when a multi-step workflow goes wrong.
AI infrastructure
Infrastructure determines whether the rest of the stack can operate consistently. It supports the compute, routing, observability, and deployment workflows that allow teams to test ideas and keep improving them.
We did not want these topics to become four separate talks. Their value comes from the handoffs between them: how an open model is evaluated, how it is served, how it becomes part of an agent workflow, and how infrastructure keeps the whole system dependable.
A speaker addresses the audience during the SF Tech Week Panel.
A Broader Open AI Community
The panel was one part of a broader ecosystem gathering. We were glad to see participation and support from Artificial Analysis, Nous Research, RadixArk, Vercel, Command Code, Kylon, MiniMax, TinyFish, Zed, and Zoowork, alongside the developers and community members who joined us in person.
Each organization approaches the open AI stack differently. Some work on research and model development; others focus on evaluation, developer tools, agents, infrastructure, or products built on top. Bringing those perspectives together made the subject feel less abstract. It gave people a chance to talk about how the pieces fit together in practice.
What We Took Away
Our biggest takeaway was the community’s appetite for technical discussion. A room of more than 400 people is meaningful on its own, but the questions and conversations mattered just as much.
The developers we met wanted to understand the trade-offs behind the systems they use: how inference performance is measured, how agents can be made safer, how open models can move into production, and which infrastructure choices will hold up as applications grow. No single company or layer can answer those questions alone.
That is why we see events like this as part of our work at Novita. We build infrastructure for developers, and we also want to help create conversations that make the ecosystem easier to understand and more connected.
If you are building with open models, inference, or agents, explore the Novita AI model library and try the models that fit your next project.
Thank You
We want to thank Akshay Deshpande, Sam Herring, Walter Korman, and Mao Cheng for accepting our invitation and sharing their experience with the community. We are also grateful to our partners and exhibitors, to AWS Builder Loft for hosting us, and to every person who spent the evening asking questions, exchanging ideas, and helping make the event possible.
The evening reminded us that the open AI stack is being shaped by people working across many layers, not by any single part of the ecosystem. We look forward to continuing that conversation with the developers and builders who joined us in San Francisco.
For more event context, read the AWS Builder Center event listing and our event recap on X. You can also read Novita’s speaker highlights for Akshay Deshpande, Sam Herring, and Walter Korman, alongside Novita’s speaker highlight for Mao Cheng.
