Why Specialized AI Could Beat The God Model

[00:00:00] You saw the SpaceX S1. It's like, oh, $30 trillion. It's like, what is the world GDP? 100 trillion. >> Both Stripe and Open Router really want lots of new companies in the world. We don't want everyone to be a part of one giant company. When the models get more intelligent, the risk actually will continue to get higher. And yet, no one new is taking responsibility. [music] We're going to slowly realize how good we've had it with like deterministic [music] code. remember the days when computers did exactly what we told them to do. >> Are we going to prevent the [music] models from deceiving users during training runs predictably? Will like a model that's big enough and powerful enough suddenly stop deception and stop sandbagging. [music] Um, welcome to Asz podcast. We're here with Amjad of Replet and and Alex of Open Router. And this is the first podcast that Alex has done since the acquisition. So, we're really excited to have both of you. >> Mhm. Thank you. Excited to be here. >> Alex, let's let's start with that [00:01:01] actually if you can briefly um share um obviously, you know, massive acquisition. Uh Amjad is an investor. We're also of course an investor. Um the biggest shareholder, but yeah, who's who's counting? Um um Alex, why don't you give us a little bit of the backstory like how does acquisition like that even uh even happen? Like do you get a DM from Patrick one day? Uh, you know what? Uh, what what can you share? >> Hey, how much for an overrouter? [laughter] >> I had talked to Will Gabbrick um the like strike president a long time ago, a couple years ago when we were doing our series A and uh and we just we stayed in touch. we had like a lot of like Stripe work streams going on with various teams at Stripe. Um so there were always sort of like things that we were doing with Stripe. um we presented at Stripe [00:02:01] sessions uh and and and so it always kind of they always kind of felt close and then yeah just in in July I believe um they reached out wanted to chat met met like both of them in person and uh and it kind of just progressed from there fairly quickly. Um they're very efficient and you know they were very like founder friendly about the experience. Um I was really impressed with the whole thing. >> Did it did it did you want to sell like did it even cross your mind before they shout? >> No we were not thinking about that at all. Um I you know did really respect the company do really respect it today and you know of possible acquisition options for us. Um it was I think my top choice and [00:03:03] uh so it was like an interesting idea. Uh and you know as we kind of like fleshed out the reasons why it would make sense for both companies it got more and more interesting uh it was really clear how like aligned they were with us kind of having [clears throat] autonomy over the brand and roadmap and product and keeping open router doing what it's already doing just much faster with um much more with like a like a much more serious gotomarket plan and um and then you know some better together stories between the two products uh and the two the two companies and and then culturally there were just I I think very like in terms of like mission and values and building like a neutral trusted platform that businesses can depend on and scale on top of that's also really developer friendly with the best possible developer experience for [00:04:04] you know to encourage new companies to emerge. um you know that that alignment was there and there was a bigger picture kind of alignment too where like you know both stripe and open router really want lots of new companies in the world you know we don't want everyone to be a part of one giant company we want to like create really good incentives and uh and really sort of easy streamlined workflows for people to start new companies and grow them successfully and you know make both lifestyle and venturebacked businesses on top of really good like reliable and price efficient infrastructure and and like a marketplace that works. Um and you know I really want that future [laughter] and Stripe demonstrated that they've been wanting it and building towards it for like many many many years. Um and in many ways like payments and inference [00:05:04] are going to blend together for companies of the future. >> I'm I'm curious uh I understand why Stripe's incentive is to have you know much more vibrant startup ecosystem. I understand the kind of moral argument and why you would want that. But why is that good for open router? Like is your model for open router is that it's like a network effects business? Is it is it like a network? >> Well, for us, I think the the there's like a couple different problems Open Router solves. one is, you know, allowing you to build a company that uses AI um or like augments intelligence with unique data and and other services uh without model lock in, without vendor lock in, >> allowing you to kind of like be on the PTO frontier continuously as [00:06:05] the ecosystem grows it. And um to do that it it's a lot of work because there's all kinds of little lock in that appears. Um we also uh you know I I like we want companies to feel like the the the best way that they can like add more than just prompts on top of a single model. there's like a lot more to building um like unique intelligence and I think a big component of that is neurodeiversity. you really need the the power of multiple models that are trained in different ways, including some of your own >> um to do more than chat GPT or Claude would on the task if like someone who is thinking about buying from you as a potential customer is wondering, well, what if I just use the model directly? [00:07:05] [snorts] um like how do you really show that you are significantly better and able to build a business that that matters? Um and I think a lot of it will involve neurodeiversity and blending like powers and and good data from multiple models. Um another component is is like helping people get really good cost efficiency. Like there are a lot of businesses that just cannot do not emerge until they become cost effective and like creating an environment where we can help drive down costs by building an efficient market uh is like crucial to making that happen. Otherwise like you know why lower my prices as a as a provider? um we have a captive market and uh and like you know I think that's like a key point of marketplaces that was just [00:08:05] totally missing from AI before we showed up. There was just one player open AI. It could have been like a uh a very strange world. I'm not saying that we did all the work. [laughter] Of course not. But like helping people like choose new models and explore new models and learn like what makes um you know a closed source or openw weight model uh like actually good at your task involves like seeing what the whole ecosystem is doing and learning automatically like LLMs are not things where you can just enumerate all the features on a web page. It's impossible. you you have to see how they're being used to know what they're good at. >> So, you started the company three years ago. I'm I'm curious what is surprised you the most about sort of the evolution of open and closed source models to the present um as it relates to model performance or just people's uh [00:09:05] perspective on the variety of models they have available to them. >> People have been more open-minded than I thought they would be towards openweight models. Like typically there's like a there's a lot of brand especially an enterprise typically there's a lot of like brand trust where like an enterprises you enterprises in general are like oh I'm you know I don't really know how to tell the difference between these things so I'm just going to like you know buy the one that all the other credible enterprises are buying and and there's a lot of like enterprise um lock in with like a mentality like that. Um, and that just didn't happen that much where they It did happen like a bit, but uh we saw a lot of enterprises uh want to explore new models. It it's it's simply like it it was m it was very much good for a marketplace. um [00:10:07] enterprises wanted to diversify outside of just the proprietary frontier model labs um both for like cost reasons and for differentiation reasons. They wanted to like own their intelligence so they could um you know I think like one keep their their talent like have like an internal AI practice like AI is just like a huge strategy topic. It's not like you go to your board and you're like, "Oh, yeah. Um, you know, we fixed the AI problem, [snorts] uh, quarter complete." Like the your board is like asking you every month like what, you know, what's what's next for the AI internal AI team and all like every single enterprise, it now has just this, you know, internal AI team that they're developing where they they need a strategy behind it. um it's not just a like, you know, we check the feature off, we like set up the database and we're done. [snorts] Um [00:11:09] and so I think that that uh dynamic has resulted in just like a desire to explore and diversify um and a desire to kind of like figure out how to reduce costs and figure out how to do benchmarks for the first time in the company's life. I have been like surprised there hasn't been more benchmarking like more companies creating more benchmarks. it's starting to happen and I think like eventually we'll see a lot more of them um to demonstrate like oh yeah like this thing is better than using you know claude direct but um like I think that's going to be like a bigger focus for this internal AI group at every company um evals and I think like Jud's been doing that a lot at Replet for example like you guys have done a lot of cost per task research um you guys have And you know, you you've made like a doom loop rescue. You've kind of been like experimenting with new ways of using agents like doom [00:12:10] loop rescue and and uh like helping bring those to developers. Um so like more of more of that kind of research I think is going to pop up internally everywhere. [snorts] Um for all of those reasons. Yeah, I I think um [clears throat] uh Satcha uh CEO of Microsoft has been very uh very you know preient on this and also very articulate on um why companies need to own their intelligence ultimately in the same way that you know we had dot companies and then every company became an internet company like every company employs people that know how to like build websites and be on the internet. Similarly with with software, every company has software engineers. Every company needs some AI practice, AI capability and that will compound over time the knowledge, the intelligence inside the company, the the the use case [00:13:11] model fits like which models actually work for them, how do they save save money, they need sort of that independence. The other thing that um I think Alex Karp of um Palanteer has been talking about is that there's like a risk that when you work closely with the foundation model companies is that they're going to move into your business and we've seen that you know with Figma visavic we've seen that with like now Harvey and OpenAI and you know it's it's really hard to partner with them and because you know they're they see the world as their potential market Like I mean when when they talk to investors they're like you know you saw the SpaceX S1 it's like oh $30 trillion it's like what is the world GDP 100 trillion and so there is a sense in which these companies are different than other generation of companies it's it's harder to partner with them because their ambition is is is such that they want to they want to be they want to [00:14:11] subsume a big part of the economy. Um and increasingly like what we're thinking about at replet is kind of in the similar vein to what Alex have sort of innovated is uh replet is becoming more of an independence layer inside inside enterprises where we create a layer of interaction between um between you and the and the models and we get you the best token at the cheapest price. uh but also we um we also create uh an abstraction layer on top of the cloud as well because you know you should be able to deploy to AWS and Azure and you should be able to use data bricks and snowflake and so on. Um and so increasingly I think there is not just with AI with all of technology there needs to be more platforms that help companies gain gain independence. It seems like yeah what Open Router did for their segment you're you're doing for other other areas of the business. >> Yeah, [00:15:12] there was this this kind of reminds me of this tweet I saw the other day when somebody was like basically all companies are building the same thing now. Everybody's building an agent loop with like notifications and and context uh like third party uh connectors and context management and memory and uh >> sandbox >> uh sandboxes uh you know web you know agentic web search >> computer use >> and an always on agent on top of it and notifications and it's like this product is showing up everywhere. Yes. [snorts] >> Um, in a way, yeah, it is like showing up everywhere, but it it also kind of feels to me like these are just the new table stakes primitives. It's kind of like like a 2005 version of that tweet would be, oh, everybody's building the same thing. It's like a database, a users table, a sign-in page, a signup [00:16:14] page, a profile page, a logout page. like everything's the same. [laughter] It's like there's like a lot of differentiation really. It's just there's like a there's table stakes needs for AI just like there are table stakes needs for the web. >> Yeah. [snorts] And and I think um as well like inside the enterprise making these products actually do real work is still unsolved problem like you can use Muse in your personal life and connect it to your credit card and bank accounts and no one's connecting Muse to their enterprise data or even Grockbot and things like that. Um I think there's an even more emphasis on data sovereignty and and security. So we we spent the past you know year almost uh working on [00:17:15] making uh making making replet uh deployable on your own cloud basically on prem like bring your own cloud like two years ago I would have thought I would never do this uh because you know it's just like yeah the cloud is the future like software as service all of that uh but but now actually we've sort of like reverted a little bit back to a world where companies are a little bit more protective because there's so many ways in which data can leak like all these agents that people are using. I mean there's all these screenshots on Twitter. I don't know how true where Instinct is like or Muse is like mixing people's data starts to call you by a different name or something like that. Uh and so yes, the kind of consumer stuff is kind of obvious, but on the enterprise, there's still tremendous amount of work for the entire industry to do in order to actually get these things to be useful and productive at work. >> Do you are you doing any like do you have like a custom personal agent other [00:18:16] than Muse or Instinct that you use for like work stuff that you've been building? Yeah, I mean I I built something on Replet like a long time ago. Uh I started as like a sort of a CRM agent initially. That was the main problem that I had, but slowly like we added features to it and it's sort of like doing more and more things. But what's what's really interesting is that the more I connected Replet to my all my stuff, it sort of like started answering all the things for me. Um and so increasingly like the platform itself is like subsuming these sort of like these domain specific agents that I built. I do think that there there is some in some ways you want something that is sometimes like focused on one particular thing and you don't want it to be able to do everything. On the other hand once you have your entire company's context in one place it's really cool to join across totally different domains. like when I ask it a question, it can like [00:19:16] look at my sort of personal chat history, join it across the GitHub repo across Salesforce. And so it says like it will link like random things. It's like, oh, you met this guy like a year ago at a conference. I see it on your calendar and by the way, someone else from their team is in discussion with your sales team. and it creates all these different synergies and and when I go into a meeting I like I you know I I've connected a lot of different threads and and I'm I'm make much more progress on a deal or or something like that. Uh so so this is where it's sort of trending now. >> Well I I think I I I'm like a little bit I'll take the counter on that. I think that the the worst part about doing crossdain joins with your personal agent is that the the more work you give it to do, the more understanding of what's going on [00:20:17] you're sacrificing. And yet no one no one knew is taking responsibility for that sacrifice understanding. Like agents don't have any responsibility. you can't like you like if there's a fixed level of cortisol that the whole company can tolerate between everybody and uh you know you like de you you uh you like I I want to be like less stressed about some area if I'm going to be like sacrificing my understanding of it >> someone else needs to take the cortisol >> but the the agent doesn't >> like take on any of that responsibility Um, and like a universal agent that's doing all things, you know, I can't adjust how much I'm sacrificing in all the different things. It's it like points me a little bit towards, you know, maybe like down the road like the sub agents that people use will be [00:21:17] like very vertically focused. Maybe we have like a chief of staff type agent that, you know, coordinates between them. Um, but I feel like you do need like vertically focused agents where you're like, okay, like there this agent is more responsible psychologically for these things. Um, and like I want like quality checks that make sure it's doing those things correctly. And I it it doesn't need to focus on anything else. It just has one focus area. Um, I wonder if that's gonna like help people like at least get like a weird loose sense of responsibility on top of agents. >> Fascinating. So, so you're saying general agents create like a tragedy of commons of sorts? >> Kind kind of like I have a general agent that every day um looks for things that need me and tries to figure out what to do. And it's just like impossible to [00:22:21] improve this agent. [laughter] like I it's it's like every time I try to make an improvement, I end up like ignoring its output about a week later. Um it's it just feels like it doesn't really care about any of the like specific things it's diving into. Um, and uh, like kind of imagine having a chief of staff where they're very good at, you know, like drafting all of your replies across the whole organization. Um, and then compare that to something where you have like 10 chiefs of staff, each each as competent as that one chief of staff, but they're all responsible for like individual sectors of of what you of what makes up your life. like the latter I feel like gives you a way of tuning how much understanding you sacrifice >> compared to the the gain you get from [00:23:22] like basically I can like lean in more to the areas where the agent is failing for some areas and then have agents with very good competency like take over my understanding of other of other parts of my life. >> Yeah. Interesting. It's it's sort of like uh almost rediscovering, you know, um specialization, right? What's his name? The like the famous economist uh Adam um like >> Adam Smith. >> Adam Smith like with a pencil kind of thing where you know that that was like a huge realization for humanity that like special specialization is actually good. The problem is like we kind of like over specialized as like a civilization and I think over specialization is is um is oppressive in its its own ways. I mean I think we've you know um sort of like you know [00:24:23] there's the Marxist theory of alienation right the the idea is um uh because of over spec uh uh you know people are doing just focus on one thing uh they they do not see the fruits of their labor they don't actually know what their impact is on the larger organization or the product they're producing and therefore they actually kind of feel depressed and detached and and you're kind of acting like a machine and you're not actually fully fully human. And so m maybe there's like a bit of a reaction to that and and and I think with our agents we're like oh that there should be like one god god agent but in fact specialization is actually like really good for machines and that's like the the point that you're making and like humans should be general but like machines should be ultimately a lot more specialized. >> The problem with I think what I'm describing is that we don't know what good looks like. There hasn't been a system of specialized agents that feels as elegant as like chat GBT or Claude or [00:25:24] Muse. Uh where you're basically just talking to one thing only. It it's it's yet to be discovered like maybe like opening I just launched dots. Um I think they're kind of like experimenting in that direction. Gro like Grockbot I guess kind of counts that >> but but they're all they're all very general. Like I think the idea behind DOT is that it's like it's like uh it's like your digital double as at least that's what I understood it. >> Well, when I saw Grock bot, I don't know what what what's going to I don't know that much about dots yet, but uh they just came out, but Grock bot when it first came out, like the first use cases that I saw people talking about were, "Oh, wo, I can make two bots. One that knows my bank account, right, >> and one that knows my Twitter account." And the two bots like don't have the credentials from each other. Yeah. >> But they can talk to each other if like they need to get something done. Um there's no credential sharing though. [00:26:24] And that was like one sort of big unique thing I saw like a couple time like pop up a couple times that people seem to like. >> But it looks like Muse is but I wasn't sure. >> Muse and Instinct are >> have a much stronger product market fit than Grockpot. Um, and perhaps >> seems like that >> perhaps it is because you don't have to worry about creating these these domains, but but I think maybe personal agents are different than than work agents. And I think your kind of your critique of general agents is is more uh pertaining to to to work and to enterprise, which I I sort of agree with. And um you know there's also all sorts of uh data access considerations. I think as CEOs we can have general agents because we have admin access. Uh but you know you're for individual employees or certain teams they can't have like truly general you know fully [00:27:25] contextaw aware agents because there's there's access control control issues. So you'll have to kind of work on on something like specialization. Ultimately, I also think we need to figure out what is agent to agent communication look like. I don't think there is like good protocols around that just yet. I I don't think that agents are trained to handle that very well. I think we've seen it seems like the next generation of OpenAI models are trained to do agent collaboration because we've seen it in the hugging face hack where they start helping each other and sort of emerge naturally. But there's also needs to be some way in which like an agent can't convince another agent to kind of give it give it information that it shouldn't give it give it. like that there's there needs to be like data um isolation and and and and and proper ways in which [00:28:26] these agents communicate. You almost don't want to communicate them fully in natural language. Maybe that there's like some other DSL or protocol that that they need to follow. >> I really I I I think one of the cool potential applications of Jev and other decision models like it is going to be alignment. you know, checking to see if a tool call or like an agentto agent, you know, communication is aligned because there's just so many tool calls. Like you really need like a cheap fast model. Um, if you're going to block something like that, then uh like a really really fast decision model that just classifies and gives feedback on rejections um might be like a really good way to bridge the gap between agents and from agent to infrastructure too. So I haven't seen like and we have like a a little prototype that we're running [00:29:27] internally um at open router but um I kind of think a good I think it could be like an interesting alignment >> using it for policy enforcements. >> Yeah. Like imagine um uh looking at the system prompt and the current tool call being made and be like like you know is this aligned with the the system prompt of the original agent and with these like extra guidelines that maybe we didn't tell the agent about. For example, let's say you have a bunch of agents that are instructed to do um to like red team the like some some new product and they cannot should not uh be able to access the internet. Um and if they ever do, they should stop right away. Um but you might not want to like explain all of that to the agents doing the red teaming. you might want them to try to like break out of the sandbox um and act like bad actors. Like what [00:30:27] would a bad actor do? It wouldn't be like break out of the sand, you know, try to like break into this company and the moment you do stop. Don't do anything else. [laughter] Mhm. >> Um so like having another model more like you know use for [snorts] you know building a neurodeiverse system having another model like check uh every single tool call or every single assistant message to see if it's indeed aligned with something that wasn't in the system prompt I think makes sense. Um, and then having kind of like structural safeguards too, which is what I think Nvidia um, just launched with their like open agent safety. Uh, I think it was called open shell. Um, like I think companies are probably going to explore a combination of those. >> I I wonder another thing about specialization and um, sort of what you're talking about. There's a lot of talk of recursive self-improvement. Um there's something I don't think it's getting a lot of um sort of discussion [00:31:30] which is uh models training their replacements. It's sort of like you know you can think of it as a just in time compiler. So the way just in time compilers is um you know as you're executing dynamic code you know the the the interpreter realizes that there's an opportunity to optimize it will emit machine code on the fly and that's a lot more optimized. So you can imagine models like general models you're kind of doing something with Opus or some some of the Astra or some of the big models and they realize that the use case is is limited uh or you prompt them in some way or some other agent observing and realizes that use case is limited and I think you know general agents have all these flaws that you just talked about but also there's there's more potential for them to be harmful. there's more potential for them to go off the rails and sort of on the fly trains a model uh that that could be [00:32:30] it's a replacement but is like a lot more domain specific uh and therefore it is cheaper and also um you know less uh less vulnerable to prompt injections uh less less less harmful for for you know because it's less capable um And it's it's it's almost like a like you know some system that's training machine learning models for specific use cases as it's monitoring the entire system >> like would that specific use case be it involve um unstructured text generation or very like structured decision model like >> it could be unstructured text generation it could be decision models like even the case of of of Jav like if you have if you understand the the inputs ahead of time. Uh you could potentially like, you know, take an offtheshelf like Quen or something like that and like train it specifically for that policy. Like for that makes sense to do for cost [00:33:33] reasons. Um assuming that there aren't like really like the the model labs, the frontier model labs might make very lowcost models um that you can easily transition to >> safety. Safety as well, right? Oh yeah, I see your point. >> They're so capable. And so I think often times people are using these big foundation AGI like models to like it's like nuking a butterfly, right? It's like [laughter] they're very, you know, most of the times like a lot of the use cases even unstructured use cases don't need that capable model. >> I wish there were more public emails about this stuff like but a lot of the eval are private. you just can't see um whether like when the models get more intelligent the risk actually uh will like continue to get higher because I think there's also an argument to be made that alignment will get better and the models will like start to [00:34:37] you know avoid going off and you know hacking on their own um as we as they get smarter and better at alignment especially when it comes to agentto agent coordination like the the this is something Noam Brown um said on a podcast recently like as the agents have gotten smarter, they've gotten just better at coordinating. Um there uh there's they're there's still like and and you know it's unclear if they're going to like be harder to align than humans when they're when we get more and more of them. But if we can figure that problem out um then a smaller model like will it be harder to align? >> So so Eric asked earlier is is that do I think it's true that smarter models are are more aligned naturally or they're easier to to align? Well, I if you think back to the original sort of like [00:35:37] rationalist less wrong arguments for AI safety, there is this thing called the orthogonality thesis. The idea is that intelligence is orthogonal to ethics or morality or you know so on. Um I don't believe that's entirely true with with humans. I think people who are generally like more intelligent kind of more more educated tend to tend to not always tend to you be more considerate of animals for example um but uh but but in in in in machines uh I I think it could go the the other way because you know there's been quite a bit of studies on on on RL Well, showing like, you know, how reward hacking and deception they just like get better at it. And like the evals could be could be deceiving because uh the model [00:36:40] could be smart enough to to to know that it's getting evaled. I mean, we already know this. It's been shown that if you do a lot of monitoring of chain of thought, they start lying in their chain of thoughts. >> [snorts] >> Um, and so you add pressure almost on the chain of thought and that that kind of creates and and I think at some point for you to do proper alignment evals, you need to run it for like months, >> right? You need to run this thing for months on a like a really large, you know, goal or task in order for for it to to truly kind of figure out whether it's aligned or or or not. Yeah. I I I always struggle with this word alignment. It just feels like wrong for so many reasons. It's it's sort of like vague and and uh sort of like aligned to what whose values and and so it just doesn't make the conversation easier. I think in this [00:37:42] case I'm talking especially about deception like the model is actually deceiving its user. I mean I I mean maybe this kind of reduces to like are we going to solve the line, you know, are we going to prevent the models from deceiving users during training runs predictably with like you know better like like will will like a model that's big enough and powerful enough suddenly stop deception and stop sandbagging? Um and nobody knows the answer to that yet. So like at the point when that does you if that ever does happen though we might see kind of an interesting pressure for organizations to go towards the frontier like >> oh interesting >> you know all uh to have no you know basically no risk or or significantly less um [00:38:42] >> would they would they be willing to pay 10x to to get that that much? I mean, it probably depends on the like types of tasks they're trying to do. >> Like, you know, some just have way lower risk than others. Um, writing code uh that or doing like security research is the highest risk type of task today. And so you probably spend 10x to get a fully aligned model that can also find all the bugs or fully like anti-deceptive model that can also find all the bugs. One of the coolest things about decision models is that you fully control the structured output and and generally with structured output models in general like the the room for misbehavior is so much lower. you just have like defined tasks and only machines are like dealing with the outputs and it's not writing code um [00:39:43] that it can execute the those tasks feel like probably under represented in the ones that people talk about and in the things that enterprises are dealing with. Um so I I expect like enterprises to get a lot more interested in them. >> Yeah. I I feel like we're going to slowly realize how good we've had we've had it with like deterministic code. We're like, "Oh my god, remember the days when when computers did exactly what we told them to do?" And I think like things like Jev I think hint at like more more of a need for you know not only specialized models but models whose outwood domain is is more controllable and maybe you could do maybe you could do a lot more than than we thought you you'd need um you know by using like a bunch of [00:40:45] specialized models specialized output models. >> Have you guys done any workloads internally with it? >> You know, I've been training a lot of small models. Uh, I mean, I said this glip thing when it first came out because I I I was like sort of I gave this hacker news comment like comment which I felt disgusted with myself afterwards. But I I've been taking a lot of like Quen 8B and like um and like [clears throat] asking honestly like asking you know Fable and Opus and Austra to train a model. For example, I trained a cost estimator model uh internally so that when you put it in a prompt in replet, we know exactly how much it it will cost. And it basically emits a you know probability distribution over multiple buckets like if it is between five and $10 bucket A bucket B between you know 10 and $20 and like so I'm used to the training these classifiers by you know giving it different enums [00:41:46] essentially and and looking at the log props per enum. >> I've been doing it for for a couple of years. I trained a chatbot to play by just doing that. So I I'm I'm already sort of pilled on like sort of decision models and specialized models. So it wasn't that big moment for me. But I understand that like a found like a true foundation model that's fully promptable uh is like is like amazing user exper amazing developer experience and you can like do a bunch of stuff with it without training a model from scratch. But if you have a data, if you work at a place where you you have the data, we have so much data at Replet. Like I ended up training a lot of specialized classification models pretty easily. >> Yeah. Like I definitely it it also feels like less model debt. Like something that I I still hear from companies is that they're like worried about fine-tuning models for like unstructured outputs because you're just like always you got to redo it again in like two months and everybody just feels the [00:42:47] model the weight of the model debt. But like a very bespoke classifier that's trained with like proprietary data, >> you just I I feel like people won't think it's behind constantly and it might just like last longer. >> Yeah. >> Um it's just you you you don't have to worry about its ability to speak a new language or write Rust or, you know, do anything that like the the LLMs are being like evaluated on. You know the use cases so you can like build it more. And I I I it seems like an easy thing for enterprises to build themselves and uh and it and actually like not regret. >> Yeah. Speaking of Rust, actually like a good analogy is when the world got super excited about dynam dynamic languages. Um like if you think back to the 90s, everyone was writing in Java and C++ things like that. And then like Python, JavaScript, Ruby just like took over the internet and everyone was like ah this is how you build startups really quickly. this is and you built Stripe, a financial organization on Ruby. I was [00:43:48] like, how crazy is that? And and we built Facebook, you know, using using PHP. And then everyone was like, oh Like we're running into all these really bad bugs. It's like freaking slow. Uh so like let's go in and add types. Okay, let's add a JIT compiler. Let's And you end up sort of reinventing everything. And then Rust came out. I was like, "Okay, I guess we can use Rust for for a lot of things we would otherwise be using JavaScript and Python." And my prediction is that the same cycle will happen here where we're using these AGI like models for all these different use cases and then everyone's going to wake up and be like, "Oh my god, this is like so wasteful, so risky for no reason." Um, and and there's it's got to be so much easier. We're actually adding that capability on Replet, but I think it's going to be everywhere. it's it's going to be so much easier to like to go on a site like upload a CSV file and get a special model that does one thing. Uh and um you [00:44:50] know that that goes back to your thesis about open router this like neuro neurodeiversity which I like really fundamentally believe in a lot more and Eric and I had like discussions a lot about like AGI and whether we we're truly on a path to AGI or whether it's even desirable to to to get there. And I think the future is is a lot more diversity >> outside of code review, which was like I think the first time I saw people get really serious about using like different model families to doublech checkck the results of their main model. um the uh these like fusion models like I I like the research has been getting has been like kind of slow for years on like doing mixture of of models and composite models, but like things have been speeding up from my like view of the research and and I mean now we see like a bunch of AI [00:45:52] agent labs Um like we launched a fusion uh tool, a fusion model and Cognition launched one and like they do reduce cost um follow and and allow like you know a wider breadth of ideas to be searched like our initial launch was focused on deep research. The thinking is that like if all these model apps are training on different sources of data like why not pull from all of them and uh and this like resulted in basically fable level quality at 2x lower cost. >> We just published results actually just just today about about that. We we we showed like a deep sweethe through through combination different things including the harness but but also you might think of it as a fusion type thing um where it's like um you know frontier [00:46:52] level at like 40 to 50% of the cost >> and uh which models does it use? I think it's it changes over time, but one thing that has been interesting is I think OpenAI added this feature that allows you to save um [clears throat] the computation uh like uh across different model families. So you can like also across different um effort levels. Mhm. >> So, uh, like you wouldn't do you wouldn't miss the cash if you change the effort. Don't quote me on this. I think it's also across different models, which is hard to fathom how. I might be wrong though, [laughter] but I I need to double check that. But I think, you know, staying within the OpenAI family has like added a lot of efficiencies, but in the past we've done it with other with other with other models as well [00:47:52] because cache is is like one of the big like being cash aware is like one of the biggest things when you're designing fusion models, routers, um sort of escalation models. Alex, this has been a great conversation. Thank you. podcast.