Open Source Wins, AGI Is Here, and Scorsese’s AI Toolkit with CEOs of Cerebras & Black Forest Labs

[00:00:00] We are in the race for super intelligence and uh Andrew Feldman is back uh and obviously CEO and founder of uh Cerebras doing inference chips pioneered the space had a successful IPO. We've talked about this a couple of times. We got to see each other in January at Davos. IPO happens. Uh the boys and I got to sit with you recently. >> That was fun >> at liquidity. >> That was really that was really fun. >> Had a great discussion with the boys. I wanted to deep dive with you about a couple of topics. The first one is the buildout of AI. We've never seen a build out like this since, you know, the Great Wall of China, >> right? Who knows? >> The pyramids. I mean, it feels like the amount of capital, time, and intelligent people on the planet dedicating themselves to the buildout of something. Um I I can't think of anything in our lifetimes with per perhaps uh you know before our lifetimes the war effort [00:01:00] >> right >> this is a mobilization and a scale that we read about we hear about but you're actually doing it you have customers who are building data centers and you're a key piece of that >> all app started with an $8 domain and no VC funding and became one of the largest ad platforms in the world now that same engine powers Appleven ads for e-commerce. Your ads run inside mobile games, reaching over a billion people with full screen, distraction-free attention. The platform finds buyers and optimizes for profit. You set the target, it does the rest. One cookware brand went from 4 million to $16 million, turned profitable, and is on pace for 80 million this year. Visit apploven.com/allin to launch your first campaign today. >> Maybe you could just enlighten us in 2026. What is Cerebras doing and what is happening with this buildout out in Texas? These are some gigantic gigantic [00:02:01] efforts. The the size and scope of what is being built, the physical size and scope. Usually when we talk about software or we talk about hardware, we're talking about chips or boxes and and they don't have the same sort of physical enormity. Right. >> Right. And what we're talking about now are data centers that are in the next several years going to use more power than the previous 50 years on Earth took. Wow. Right. We're talking about individual buildings the size of football fields that have more power coming into them than midsize cities. And they're being built they're being built across the US. They're being built in Canada. They're being built throughout the Nordics. are being built here in Paris and throughout France, in Europe, in the Middle East in nations that sort [snorts] of weren't front and center in anybody's mind previously. You know, Kazakhstan, Tajjikstan are building out Georgia building out data centers of size. Armenia, everybody's sort of focused. [00:03:01] >> Every country >> um and every state obviously in America feels they need to participate in this >> and the people who are buying the capacity, the open AI, anthropics, SpaceX, SpaceX AI, uh the Googles, they are insatiable right now. >> Yeah. and [clears throat] they're building how many years out when you talk to them? They were ordering chips from Cerebras before you were finished with the chips. They're putting orders in ahead of time. The irony is unlike many sort of exciting times in technology. They're trying to capture yesterday's demand, right? The demand is way outstripping our ability to build data centers and to fill them with hardware. >> All right? And so, you know, we have a $25 billion backlog, >> $25 billion backlog [snorts] >> and we we are not alone in that that that OpenAI uh Anthropic, you go through [00:04:02] this list of of uh Google wants more data centers, Microsoft wants more data centers, AWS wants more data centers, right? All of these players are not chasing sort of if you build it, they will come. They're chasing the demand is booked, >> right? [laughter] How do we keep them from leaving? Right. And and that that's extremely unusual. >> It's very unusual. And now we have people who are, you know, we have a term for a token maxing. >> Yeah. >> And there's a great debate. Is this actually creating value? I'm curious where you stand. You know, is it even possible that this much demand could be created if value did not exist? There is clearly massive value happening. >> Yeah. But there's also massive experimentation. >> Oh, for sure. You know, I you know what I I I liken this to when we first started with uh AWS and it was so good to get around your own IT organization, >> right? [00:05:02] >> That you told every engineer, yeah, go ahead, put on your credit card kite and sign up. >> Yeah. >> Right. And a lot of it was really useful and some of it was like, God, I wish we didn't do that. >> Yeah. And so for sure there's experimentation, but it doesn't mean that the net value isn't enormous. It it means some of it is going to go nowhere. >> And you know, it was the same. I remember when Costco opened up in in in in the Palo Alto area in 1988. And people used to shop Costco like they shop Safeway. They go down every aisle. >> Yes. >> And that's a horrible way to shop Costco because you end up with four things you didn't need and each was $22. >> Right. And as people got more sort of accustomed to it, you go to the back, you get the chicken, >> 18 cupcakes for the kids' birthday party, bang, you are strategic. And it's exactly the same. >> I think at first people opened up and said everybody as much tokens as you want. >> And the in enterprises, there's no open loop. We don't give sort of any resource [00:06:03] unconstrained to people. And now we're jumping on saying, "Whoa, all right. These guys should have as much as they need. They're enormously productive over here. We can use maybe an open- source model, maybe a cheaper model over here. And now we're sort of running like a business and we're really seeing a certain type of person emerge who knows how to deploy this technology. Systems thinking, >> yeah, >> which developers kind of have innately CEOs tend to be great strategists and understand systems. Uh but this the intelligence is getting so much better every step along the way that I'm watching individuals typically startup founders but also venture capitalists and associates who work at my venture firm. They start playing with the tool and then the tool starts playing with them. They start to go, "Oh, I haven't clearly defined what my goal is. I don't understand what a system is. I don't [00:07:03] under I've never heard about making a requirements document." And the software's like, "Do you have a requirements document? What's your goal?" [laughter] The AI starts telling people, "You're token maxing and you need to get a little more focused here." >> One of one of my colleagues 20 years ago, a really smart smart computer scientist said, "Computers really dumb. They do exactly what you tell them." >> Yeah. >> And at first, prompting was like that, right? You modified your prompt a little bit and it changed the answer >> dramatically. >> Dramatically. And increasingly it's understanding what your intent was. >> Right. Right. And if you if you have a chance to to to play with Fable or or 56 from from OpenAI, increasingly what you don't have to get the prompt just right. You don't have to be a prompt whisperer. Instead, you ask it and it says, "Well, here are some things and and by the way, maybe you wanted the chart to to to go two ways. You wanted a line and a bar." And it's like, "Well, that's exactly what I wanted. I didn't ask for it, but [00:08:03] that is better. And and so it's it's understanding intent and that's a huge leap which if we were sitting here two years ago, the idea >> we would never have been able to predict in a short 24 months that it would go from being a great summarizer researcher of web results, >> right, >> to actually understanding your intent and then providing a solution and abstracting it all from you. >> That's right. That's right. >> Which is a very weird thing. I don't know if you've played with the Hermes agent yet. >> Yeah. >> Have you played with it yet? >> I mean, I asked it just this morning. Uh, and I I was given a secret bit tensor project that has the new um ZAI's uh model 52 >> and they gave me >> GLM 52. Yeah. >> GLM 52. So, somebody in that Bit Tensor, I think you understand Bit Tensor, you've heard of it, the distributed >> um crypto project. And so they have all this extra capacity. I was a whisperer [00:09:04] told me probably some capacity in China that has free energy. Okay, fine. So they gave me unlimited capacity. So I started having to do some really crazy jobs where I was saying like every hour I want you to tell me what the trends in the world are that nobody else has identified yet. And you can do whatever you want to do that. But my goal is to be the smartest trend hunter in the world. And I watched what it was doing in the background and it started debating itself >> on where it should find the things. He said, "Well, we should probably go to Hacker News and Reddit." And then I was like, "Yeah, but there's also social media and trends tend to manifest on Instagram." >> That's a reasoning model. You were watching a reasoning model work out. >> Yeah. Isn't that interesting? I mean, that's amazing. And and it was collapsed. So, as a civilian, >> right, who doesn't hit the uncolapsed moment, and if you were using chatbt 3.5 or you were using 4.8, whatever it was and you haven't used this new level of reasoning and inference and unlimited compute essentially, [00:10:04] >> right? >> It opened my eyes just this morning of what a world >> of unlimited tokens might look like, >> right? >> Cuz unlimited tokens, I believe, means unlimited reasoning. It does. What does that mean? Yeah, it's uh I mean if you run these for 25 or 48 hours, you get amazing things now and what what if by using Cerebrus we were 15 times faster and then you ran it for 24 hours, >> right? A and you got weeks or [clears throat] months worth of thinking. >> Yeah. And I mean it it is uh it is extraordinary. And I I think one of the things is people like Ilia and Sam in the early days were saying this was coming. >> Right. >> Right. And I think when you look back you say to yourself, "Holy crap." Those guys saw it. >> Yeah. They were They could see around the corner. >> That's right. And the rest of us could see like what? I'm not sure. It's when we had Sam on Allin uh at one point he [00:11:06] uh and he said, "Oh, you know, I'd love to come on at some point." I said, "Sure, come on." And he he was talking about it. He said, >> you know, I said, "What's next?" He said, "Reasoning." I said, "Unpack that. What does it mean?" Well, >> understanding what your intent was just as you're saying and then figuring out a strategy and then maybe talking to other agents and other threads about like is this the right thing to do and vetting each other's work and I'm like wow we have come a long way from guess the next word >> right right fill the sentence in you know summarize this PDF >> now Cerebrus is at the center of this because this reasoning is inference >> this reasoning is inference and it's computationally intensive >> right >> right and So fast compute makes this sort of work fast and sort of tractable. It doesn't it by taking a huge amount of time to get a good answer. >> And so it's exactly the fact that that that this reasoning consumes a huge amount of tokens internally >> that allows a blisteringly fast machine [00:12:06] like ours. And I I brought one because I I'm never far without you know when one costs half a billion to make you you bring it everywhere with you. Uh we we were um we were tossing this back and forth at Davos. >> Uh what's the model number of this one? Th >> this was in the first eight or 10. Um >> so this has a special place. >> This has a special place. I mean my wife says it's like I'm a kid with a a dirt bike for his 8th birthday. It was in his bedroom at night. I I carry him with me. >> I I mean when you have um you know your your next party at the house, I highly recommend just a little or something or I >> I think it would be like a great fit. It would be a great bit if you had some. >> That's [laughter] right. Um, but what we're looking at here is the ability to do that reasoning at scale. And what is Moore's law for inference and for cerebrus? Do you have something internally you discuss as we're going to double this every x time [00:13:06] period? So all chips prior to us in the processor world followed Moore's law. >> Got it. >> And we broke >> doubling every 18 months. >> Doubling about every 18 months. >> Got it. >> Um and we crushed it with this chip >> and we've carved out a whole new trajectory. And uh my view is in the next 18 months we'll be way over 2x >> interesting. And so, uh, I I think that, uh, early in an architecture, you have room to to do much better than what was traditionally Morris law. Now, if you've got a 20-year-old architecture like the GPU, it's much harder, >> right? >> You you have to rely on things like smaller geometry, >> right? Going to the next fab node. But in a newer architecture, you have a huge amount of room still to to to learn about the the work that that is being presented and make optimizations that that give you huge gains. H how do you run the company? Like just being the CEO [00:14:07] now in the age of AI, [snorts] um you have $25 billion in demand. You have to you have to deploy at an just an incredible blistering pace. You have to hire people. You have to create a roadmap. I don't mean to give you a panic attack here. >> You have to keep up with somebody like OpenAI who's moving so unbelievably quickly. Yes. >> Right. And they're they're competitive. You got to keep up. >> Right. >> Right. Your hardware, your software, your deployments have to keep up with some of the fastest moving organizations in history. >> They're demanding customers. >> They are not they're not pushovers for sure. >> Yeah. A and also potentially competitors down the road. >> I look I I think there is so much demand right now that that that there is no silicon that will go unused, >> right? >> But why isn't OpenAI releasing Jalapeno? Why is Amazon making their own chips? You see this reoccurring trend? Is it a [00:15:09] way to let you know to let Jensen and Nvidia know, hey, we can do this, too. So we we need good pricing. Is it is it a little bit of a flex that way or is that the future that they're going to be in your business? >> No, I I I think nobody likes being dependent >> and I I think some of the lessons learned by the the hyperscalers of the x86 world is they were dependent on Intel. And uh some of the lessons learned by uh the GPU makers was they were dependent on a small number of hyperscalers. >> Yeah. >> And they wanted more customers and so they set about to to help fund these neoclouds. >> And so I I think mostly it's about uh an opportunity to control at least an important part of your destiny. >> Got it. >> And I I think that's a very reasonable thing. I think you don't have to sort of make the fastest chip. You you just can't be entirely dependent on other people's chips. [00:16:09] >> And that dependency has become a hot topic. Not sure if you caught the episodes over the last two weeks, but we've been talking over the last year about open source. I've been championing that a lot just because I was early into Open Claw and quickly started using Kimmy and was like, "Wait a second. I'm blowing out my claw tokens, but this Kimmy, I can't tell the difference. And then we started smart routing it and suddenly this open source started to figure out reasoning and the gap has >> suddenly closed this year. >> Well, I I you know, you you don't want to take your your Ferrari to the grocery store, >> right? You you you there there are times you want to drive your fun car. >> Yeah. >> Right. And there are times you want to throw the kids in and and don't worry if they're Cheerios on the floor. >> Minivan time, >> right? There's minivan time. And and and I think that as the sort of sophistication of the user grows, right, you're going to have hard problems and those are going to be frontier model [00:17:09] problems. They're going to be open AI problems. They're going to be anthropic problems. Going to be Gemini problems. And behind that, they're going to be a lot of ordinary problems. Right? I mean, if you think about a company, you know how much time is spent cutting things out of workday and getting it in a different cell for Yeah. >> Right. Think about >> the cutting and pasting economy is real. >> That's right. And and this doesn't need, right, gold medal math. No. >> What what this needs is sort of rock solid open-source capabilities. Yeah. And if you think about what I mean, well, we've been thinking a lot about it in DNA, but a huge amount of DNA, all right, is not invention, >> right? And you you may not need sort of the most sophisticated agents for this. And another card that's turned over recently is some folks uh maybe have concerns with the ambition of the frontier models and maybe sharing their data data leakage [00:18:11] and sovereignty of intelligence and they're saying hey our company is going to choose maybe we're in a regulated industry finance healthcare HIPPA you know FINRA all kinds of different regulations >> we need to have this on >> prem domestically and we'd liken an open-source version where we have uh a little bit more control. Yeah. And I I think >> are you seeing that now? >> We are seeing that for sure. And I I think OpenAI made a good call releasing OSS 12B some months back. That was a good open-source model. Um but I think in the US we need more domestic open source models. We need to give the world a choice, >> right? Right. If they want to run open source right now, it's OSS 12B or Chinese models. >> Nvidia has some in >> Nvidia has seen the same opportunity to push open- source models. I I I think giving them more power might might be sort of >> Well, I was about to that was you cut me off at the past like my understanding [00:19:12] was Jensen was like, "Hey, we we don't even want to talk about these open source models we have because our customers, >> right? >> We're now going to be competing with Sam, Daario, Elon, >> uh Sergey, like do we want to be in that position, >> right? >> So, but we do need some more champions here and it's open source so people can fork it." Um, but that puts you in a more neutral position. >> That's right. We we we run today, we run GLM, we run Kimmy, we run the Quen set of models and we run OpenAI's models, the closed source ones. We run models for say Galaxos Smith Klein, which they wrote and developed. Um, we run models for uh our partner uh in the UAE uh G42 and MBZ UAI. Yeah. Um that are are are their models that they designed. So we we have a a a wide variety. >> So sovereignty is a trend. >> Sovereignty is a trend. And I think uh the the government's actions with regard [00:20:12] to fable and uh uh 56 um where they said, "Oh, whoa, let's think and then we can act." >> Um I I think sort of particularly here in Europe was a bit of a wakeup call. And when you saw this going down, there's a layer of partisanship in our country right now. It's pretty fervent. Daario is pretty explicitly, you know, not part of this administration. They they've been very adversarial. Both sides have been have admitted that they're starting to work it out now. So, it's hard, I think, for us not being in the room with these parties to understand what's partisanship, what's games here. But do you believe that what they released was truly dangerous for cyber warfare, for cyber attacks, and that if you were to rate Daario's not communication, because he's a very effrovescent communicator, um I think is a diplomatic way to say it, um but to [00:21:14] have a scheduled rolled out release, >> right? We'll put aside the government's control of it, but do you think that is is a wise thing for us to do at this point? And do you think it's there was actually a major threat there? >> So what's interesting is I hadn't seen it before, >> right? And I I think you know if we just step back and say is it reasonable I don't know whe whether this was the right time but at a time >> that that that a model is sufficiently uh creative in its thinking that it poses a meaningful threat for the government to say we'd like you to roll it out in steps. >> Yeah. >> Now this doesn't seem unreasonable to me. >> Not at all. >> Right. I mean we we do this with powerful pharmaceuticals. >> Right. We we we'd like I mean we're certainly not encouraging seven years of trial and the amount of paperwork and all the garbage that has acred to the FDA, >> but with a powerful new technology, it certainly doesn't seem unreasonable to say, "Hey guys, um let's at least do [00:22:17] some red teaming at the government so we know our defenses can block this." >> Yeah. Have we checked Have we checked >> the infrastructure of the country like >> of the NSA? Have we checked the infrastructure up? [clears throat] Right. And can you give us two or 3 weeks to patch any obvious holes that are found? This doesn't seem to be >> an unreasonable thing for the government to ask. >> Right. We But we in this very polarized time >> put on top of it, well, oh my god, it's President Trump doing it. And then you have to think, well, what if it was President OA AOC or President anybody in between the the two extremes? >> I I think the polarization hurts a great deal. It hurts clear thinking, >> right? It hurts clear thinking and and and both sides are going to do some dumb things and some really smart things. >> Right. >> Right. And in fact, what I found is that the people in the government are are trying really hard. >> Um >> the rank and file, >> the rank and file are trying really hard and this is moving fast. And I I think [00:23:18] that that an ability to to set aside some some of the the polarization and say h how do we do this in a reasonable manner? I mean we we want Dario and Sam competing like crazy. It's awesome, right? It's good for the technology. It's good for it's good for entrepreneurs to see even with thousands of people this is what what you can continue to achieve. Right. >> Right. This is Google in the ass made them get sharper. Amazon got everybody got better because of that. We want that and we certainly don't want to become sort of a region where the first thing we want to do is regulate it. >> Right. >> Right. But as it gets more powerful and the industry really should do a better job of regulating itself perhaps and and it did seem like they were starting that process but then the communication was lacking maybe. >> Yeah. Uh, you know, I I think not only are they racing hard, but they're inventing this as they go too. [00:24:18] >> Yeah. Right. There's not a playbook. No. >> Right. They're inventing the we we say, "Oh, just put on guardrails." Well, they have to design the guard rails. >> Sure. >> Right. They the guardrails have an impact. Um, you know, one of the things that fast does is it makes the guard rails less painful. And so that that we we discovered that in the last six weeks. Yeah. is that the very guard rails can add time and make it feel slower and so fast ships like ours can really help that. But so they're racing against competition. They're racing against their own sense of greatness. >> Yeah. >> Right. Which is maybe even the biggest driver here. A and I think that they're in earnest trying to think about how I do the right thing. A and all of those are are are mixed in this bucket. and and sometimes you're on one side rather than the other. And >> yeah, and as you're saying, this is a first time, right? When we when 3.5 came out, it wasn't like when we're using chatbt 2.5, 3.5, it was taking down [00:25:20] networks, but in talking to Nicash uh from PaloAlto Networks, I asked him like, "Hey, well, how would you grade this?" And he said, "We put it against our software and we found bugs we were not aware of and >> it killed them." >> Yeah. He said we had to stop everything we're doing and do patches for 6 weeks, >> right? And and that's when you know, right? I mean, Nick leads, you know, maybe the leading security software firm, right? >> And when it finds in an hour, [laughter] right, tens of critical opens, you're like, whoa, this is a powerful tool >> and we need to think and and maybe you you show it to a a group first, right? May maybe you I I don't know what the right thing is but >> yeah I mean red teaming and uh we've always had just when you were releasing the new version of um an operating system you know when you have your iPhone you can say I want to be part of the beta. >> That's right. you know, right? And there's like two other baiters that you don't even get the chance to opt into as [00:26:21] consumers. And those ones are for security. Those ones are for, you know, making sure you don't lose your data or data. >> That's right. Disappear or leak or >> corruption. Any any number of these things. >> I think we can also know that that there will be a massive data leak. >> Of course, >> we we know this. Yeah. >> Right. And it's like uh Warren Buffett talked about the reinsurance industry that you know something bad's going to happen. You don't know when. >> Yeah. >> But you got to save up for it, right? You put money away for reinsurance, >> but there will be a tornado. There will be a massive earthquake. I mean, we we know this and we can do our best to plan, but there'll be a massive breach and they there'll be and we we have to steal ourselves in advance and we have to think about it and think about the right response at the time and sort of prepare ourselves for a future that is in specific unknown. But in general, we're pretty sure it's something's going to happen. >> Something [snorts] will happen and [00:27:21] >> yeah, it's typically a black swan, right? By definition, it's going to be something we didn't consider or a question we didn't know to ask, >> right? But but even knowing that there's some unknown unknowns is a useful place to start. >> Yeah. What are we not asking ourselves? >> That's right. >> With reasoning, the AI is going to be able to tell us, "Hey, schmuck humans." >> That's right. By the way, here's what you're not thinking about. This is now my closing sentence when I do my prompting is I need you to make me a prompt that will help me do this trend scouting for an example. And then I always say at the end um please check your work >> right and uh then tell me what I haven't considered in terms of my goals and uh give me ask me some questions every time you run the job and that has changed everything because it's like I checked my work by the way this was incorrect right >> and uh I'm wondering hey uh would you like me to also do this and some of the tools like perplexity do that automatically they give you your next three prompts but if you give it explicit instructions [00:28:22] My lord is it um good at that. >> So, you know, over the course of the last 10 years as I was raising money, I I thought one of the smarter questions I got at the end of a conversation where someone asked, "What was the smartest question you heard that that wasn't covered by what I asked?" >> It's incredible. right now. Now, that's somebody who's curious and thinking and humble and and trying to >> sort of use this to get a picture of of the space and and to the extent that you can ask the AI that >> and and and that it can sort of broaden your your view, you know, maybe what question should I have asked >> to to be an expert in this? What what what would a a PhD level uh uh questioner ask of this or a gold medal math? I mean I I think those are the sort of questions that you you you know you don't even know how to ask >> which you know if you start thinking about AGI and super intelligence um you know they're just definitions but [00:29:22] they're important definitions I think to kind of keep in mind because they're way points. >> That's right. And AGI I think I suspect you'll agree with me that we've hit it. we just haven't exactly deployed it fully. We we we have artificial general intelligence now. It feels like when we're talking about these reasoning moments and you know the the the ability for it to be as smart as any human but >> let's talk about >> by any definition we had 20 years ago we've hit it. >> Yes. >> Right. I mean if you think about oh there's a touring test blew it away. Yes. >> I mean, you think about that that any period of time sort of 10, 15, 20, 30, 40, 50 years ago, we we we any definition we would have previously put forward, right? >> We've blown past it. And so, which goes back to our previous point of like, do we know the questions to ask 20 years ago, science fiction authors, you know, had their say and we answered all their questions, >> right? >> If they were to look at this today, they'd be like, "Well, I'm out of [00:30:23] >> I'm out of questions. >> I'm out of questions. Sorry." And that's where sort of the the sort of listening to people who we who sound sometimes like they're on the fringe, >> right? When when when Ilia was talking eight or 10 years ago about the need for safety and then and you're like, "What?" And dead right. >> Yeah. >> Right. When when when Elon was talking about building rockets and driving the cost to to to near zero of of of a launch vehicle, you're like, "What?" And there it is. And now you can see and that's I think that's why it's really fun to be a technologist now. [snorts] >> Well, and with these tools specifically, you know, we're talking about building all these tools and then the tools are starting to build themselves in this recursive loop. >> That's right. >> We're kind of just starting to see people apply loops. Uh, in fact, loop maxing became when I was doing my trend, when I did my trend thing, it kept picking up loop looping and it kept [00:31:23] picking up the maxing stuff and it created a buzz word for me, loop maxing, right? And then it >> magically people started talking about loop maxing and I was like, "Wow, this is really weird." It anticipated that this would other humans would come up with this word. But talk a little bit about recursive and then the road to super intelligence. And do you have a way Andrew that you think about super intelligence and what [snorts] it will mean for humanity and how we will define it and how we'll experience it? Yeah. >> I I I think let's begin on on on loop maxing or sort of recursive learning. I I I think um I think what what what Sam and Ilia and then later Daario and and and Dana Dennis saw um six years ago or five years ago was that um powerful recursive gains are are exponential, >> right? you get better, you do it again. [00:32:23] And if you continue to get gain, the the the the slope of that curve is so steep. >> Yeah. >> And that um we're just beginning to see that now. >> You ask it a question, you learn from the results, you ask it to do it again, it the results get better and more information is added. Your answer gets better. You ask it to do again, it covers more material. And the these sort of loops are producing sort of not a little bit better answers but vastly better answers. >> Yeah. A >> and that is enormously powerful because we don't quite know where it ends, >> right? >> You keep throwing compute at it. I mean, how much better does the answer get? >> You know, we we run out of tokens or our budget or or or but but holy cow. I mean, when does the exponential stop or does the answer keep going up and up and up to the right? >> Yeah. And that's sort of an enormously interesting intellectual question right now. >> Yeah. Like when do we run out of [00:33:23] problems to solve and >> Well, that's right. And and when are are the the problems no longer sort of intellectual problems and they're now people problems? >> Yeah. >> Right. How to organize people to to get done what the AI asked for. >> Right. Right. I mean, as you know, in running your company, a lot of your problems aren't hard intellectual problems. They're people working together problems. >> Yeah. >> Right. And you >> motivation. >> Motivation. You spend a lot of time as a leader spraying WD40 on your team. >> Right. >> Right. It just so so friction is reduced and >> um how do we learn about those from AI? >> How do we get behavioral insight from from AI? And and I think that's some of the things the world models are going to bring us as they begin to watch human behavior. >> Yeah. We didn't even get to that. This is going to be for another interview. But um when these things jump off the screens, >> right, and they're in the real world and [00:34:23] the recursiveness starts, not trying to solve math problems and right, you know, humanity's most difficult ones, but hey, you know, there's an incredible world out here and here's the palace of Versailles, right? You're just like now we're like make me a new version of Salesforce and we're like hey you know what I'd like a palace of Versailles I've got a hundred acres somewhere out Texas or Nevada I'll just send a thousand optimistes out there make me the Palace of Versailles >> right >> sounds fantastical but the Palace of Versailles would seem fantastical to people who lived a thousand years before it >> and it was fantastical I think to the people who built it. >> Yeah. >> Right. Even to the builders, I I think they were awed >> at it as they built it. >> Yeah. They're compounding. They're compounding recursive learning. >> That's right. A >> and generations. We talked about you had a really such a great insight of in building this place, you had generations of masons. >> Yeah. I I think in in in all these large projects um often there were families uh [00:35:23] who were specialists who you and uh you you apprenticed under your father, your uncle and when you had a project that took 50 or 70 or 100 years, you might have three or four generations of the same family, right? The same stonemason family working on the same structure >> and passing on passing on the learnings, >> new innovations, >> right? which is what we've modeled with this new >> That's right. >> models and and what you're building in the infrastructure. >> It's pretty incredible when you think about it. >> Especially when we're sitting here and the and that's what I mean I I think the the problem with human learning is um it often moves it at the pace of a generation. uh and like uh elephants and other large mammals, we don't have generations but every 15 or 20 years. And if you want to move really quickly across generations, you want them happening more like Drosophili, like fruitfly. You want two [00:36:24] a day. >> Yeah. >> Right. Then and you see that in genetics, that's why we study them in genetics because learning encoded in the DNA, you can study over thousands of generations. >> And I I think that what we're getting is that equivalent in AI. We're getting sort of learning so quickly over the equivalent of thousands of generations. >> Yeah. Darwin would be >> in awe of this uh pace evolution. >> That's exactly right. You think about it as there was I remember when I was getting my psychology degree and they were teaching us about paradigms and I was like trying to understand how the paradigms shifted and the professor said to me uh Jason uh what you have to understand is paradigms don't die. >> They don't. >> People do. >> That's right. >> And that's how Freud [snorts] >> he and Thomas Freud and Skinner and Young like it took them dying. >> That's right. It took generation question it and that was 20 years [00:37:24] sometimes 40 years as their students maintained positions of leadership until someone said maybe we could do it differently and I I think what you're seeing is this iteration is a shortening of the the the intergeneration gap and the learning is so fast. It's uh always so great to talk to you because uh one it's just intellectually um uh so your your approach to it is so intellectually rigorous but also um with so much poom in the world. I I feel so good that you're such an optimist about this technology and you're building it >> with such thoughtfulness and it I think for people who are hearing these horror stories about AI and job loss and everything, they need to understand there are people like yourself who are building this in an incredibly thoughtful way and this is going to be a net benefit for humanity [snorts] that just is unimaginable. Yeah, >> we have a shot with this technology. So [00:38:26] not our children nor anyone they know dies of cancer. >> All right. I mean say it like that. There will be some dislocation in the economy. Sure. There will be there was dislocation when cars came and and and it was a bad deal to to to be a guy who shed horses, right? Or built carriages. >> But you got to also against that, you know, make your tea of the cons and the pros. >> Yeah. >> Right. There's a shot that our children, none of them nor their people they love will die cancer. And that that's one >> thing that we can work on wi with this technology and we will have great purchase on [snorts] >> and and I think you begin listing those and then it's a more thoughtful discussion. >> Yeah. Unlimited energy, unlimited calories, unlimited knowledge, unlimited education, unlimited housing >> and how we do it. We imagine imagine sort of we know how to teach children and we don't do it right. Aristotle was a tutor to Alexander the Great. Socrates was his tutor. We know that if you give a child a tutor and the tutor modifies [00:39:27] the teaching for the child, they learn better. >> That's not how we do teaching classes. Factory farming. >> That's right. We we teach to some sort of mid-level. Imagine if we beat built agents that that taught children for their way of learning. >> Right. >> Right. And here's a way we we've been doing it the same way for a thousand years. And during that entire time, we knew how to do it better and we chose not to. >> And and here's a way we can do it. Put that on the pro side. And so as long as we're sort of thoughtfully and fairly writing the good and the bad, I I think it'll come out. >> You got to get out there, Andrew, and keep communicating your version of the world because some people see around the corner and they get a little nervous and >> Okay, fair enough. But I think the ledger, as you describe it, >> is heavily weighted towards abundance. >> I think it'll create abundance for sure. >> Massive abundance. >> Andrew, pleasure. Always a pleasure to see you. I'll see you in six months for our checkup. That'll be great. >> Industries, capital, and intelligence are converging into a single [00:40:28] interconnected system, and the infrastructure behind it needs to evolve just as quickly. NASDAQ was built for this moment, powering more than 135 marketplaces and regulators globally and connecting capital to companies shaping the future. As the innovation economy accelerates, connectivity becomes the critical asset. NASDAQ is the leading technology platform that makes it possible and scalable. Learn more at nasdaq.com. Robin Rombach is uh the co-founder and CEO of Black Forest Labs. You are uh based in Germany in Black Forest which is a city >> in Germany. >> A mountain range actually. >> A mountain range. >> Yes. >> Where you grew up. >> Where I grew up. Yes. >> And uh you are working on open- source image and video models. You worked at stable diffusion for a little bit. >> That's correct. >> Cut your teeth on that. and uh you're known for the open source model flux and maybe also for some closed source models. Tell us about the business of Black Forest Labs. What is the business [00:41:29] and what is the goal? >> 100%. Uh one uh quick addition, we are based in uh the Black Forest. It's a town called Frybook and in San Francisco. We have like >> in San Francisco of course. Yeah. >> Um we >> you're splitting your time or >> uh I'm splitting my time to a certain degree. Um we um yeah started a company 2 years ago. Um me and my co-owners as you said like we've worked on stable diffusion in the past. Before that we invented like an algorithm called latent diffusion which is basically like the fundamental algorithm behind all of like generative models that are being deployed for image generation, video generation, even like physical AI now. >> Yeah. Um it basically makes use of this principle that uh you can compress natural data such as images, such as video, such as audio into a much more like efficient representation and then train a transformer model on that. And um I mean this is the stuff why like you know like JPEG uh MP3 and all of that [00:42:30] works. And we basically translated that into like a neural uh algorithm uh a few years ago when we were still like PhD students in uh in Munich actually. And uh then build on like on top of that we build stable diffusion and then um on top of that uh yeah uh the generative models that we are developing today and of course like the technology has advanced um but we are now tackling I would say models that are really made for understanding like the whole world around us multimodal visual models uh pre-trained on images audio data at the same time and we are now like entering a new paradigm which is combining that with uh something that's called action prediction such that you can actually use the same model to make images to make videos to make audio and to predict actions which means you can ultimately deploy it on a robot in the real world. >> Wow. So from the image to the video, the [00:43:32] audio and then eventually the real world with robotics and a real world model because if you can make the image you and you can train the model that means by default you understand the world. In order to make a video of the world you have to understand the world. Yeah. [snorts] >> And the objects in >> I I think that's Yeah. I think that's like a a really good like way to think about it. Yeah, it's like it's like an intuitive way uh to interact with the world, right? Like I I would say there's like these like complimentary forms of intelligence ultimately. There's like intuitive intelligence and then there's like a deep reasoning layer. Now ultimately you need for like a kind of like complete form you need both um and you need them to interact and I think like we've been approaching it more from like the intuitive side um images is like a very natural way to approach this whole field because it's not as computationally intensive as let's say video right [snorts] but now yeah I think like we're combining it it's converging into like a [00:44:32] a multimodal model and yeah we see like exactly like pre-training on videos gives like implicit understanding of the physics of interactions with the real world and then you can get stuff like action prediction like robotics out of the same model. And with these models and the training there kind of um been a limitation in creating videos and creating images where the criticism of generative AI is it's a bit of a slot machine. I give a prompt, it gives me something back. But how did it come up with that the training data, but you know, maybe I want a different style? Maybe I want u a different color. Maybe I want a different uh you know, aesthetic. >> Yep. >> Has that how does that problem get solved? And do you actually understand what's happening when the image is being made under the hood? Yeah. [00:45:33] Yeah, I I think like ultimately it's about like exposing as many like manipulation layers as possible to like I don't know like a user or developer that builds on top of this model, right? And um I think like we've seen that in the past with like in the past uh image models they basically started from simple text to image systems, right? Then they've expanded into a text plus image to image systems which means you could suddenly take an image like a real image or generated image um and iterate on that based on a text font like edit it um modify it right and then this expanded into taking multiple images um and a text font and combining them in a in a semantic way and producing new content. And the same principle now applies to video. And I think now it becomes actually even more interesting when like all of these like modalities are actually combined inputs and outputs of the same model. >> So let's talk about video. There's an announcement that you're [00:46:33] working with the greatest director of all time or living director Martin Scorsesei. We'll talk about that in a second. Yeah. >> Fantastic. Uh but in a movie uh this promise of being able to make a movie in which the camera angle uh the sound could be something that a Martin Scorsesei would be proud to release to his fans. How close are we? And maybe tell us a little bit about this partnership, the technology being able to make an actual movie like Good Fellas or a scene from Good Fellas, uh, versus where it is today where you can make interesting five or 10 second clips and then maybe people struggle making 10 of them and then they use some post- editing software to put them together, but you immediately understand this is not that. It's not a movie. It's AI slop. It's [00:47:35] cluji. It's doesn't pass the uncanny valley. Well, I think it's important and that's at least like the view that we have is that um these AI models, they are a medium, right? They we don't want to set like any way of how they are supposed to be used. We don't want to tell anyone especially not someone like Martin Kazi how how is he supposed to use these models? Like he is one of the like greatest filmmakers ever. Uh it was insane sitting in the same room with him multiple times and actually him seeing like exploring our models like as like one of the like core researchers behind it was like just an insane feeling. Right. And at the same time I'm also like a big fan. Um >> so you sat in a room with Marty Scarces and showed him your tools. >> Exactly. Yeah. >> And what was his reaction? What what did he key off of? What was the thing that he found most inspiring or interesting? Okay. Um I think it was really this idea of like he has clearly um a a vision in his head of like a scene or a scenery [00:48:37] where like maybe a new movie um will be shot and he's trying to explore that and kind of like um we we we basically looked at the scenery of like a village in Eastern Europe somewhere and he was describing it. we saw some outputs, we iterated on the outputs. Um, and ultimately I think and that's what he said in the end is like the like getting like the mental picture of something out of your head and communicating it in a visual way by making like these images um or the series of images um is something yeah that just makes it like easier to communicate and con convey like an idea of like what is actually in your head and I think that's like one of the like very interesting and powerful ways to use this technology and I think ultimately >> is to get the inspiration to get the vision out of his head onto an image. >> Yeah. I mean like language ultimately is like a little bit of like a lossy um communication medium, right? >> Yeah. >> Um it's also interpreted in different ways, but then visual information is so [00:49:38] rich. So rich uh like an image or video, there's so much signal in it and it's just like another way of communicating. And I think that's like one of the beautiful things that this technology ultimately enables. And I think like to your question of making like full movies with I don't know like a video uh generation model for example, I'm not sure if that is like the ultimate goal. Maybe it's like interesting to plug this into like some kind of aic workflow and make like a very long video. And I think that's really cool to explore, but I think ultimately like the real interesting use cases they come when you have like a human in the loop who iterates and uses it as a medium. And I think this is this is at least like a perspective that that I take um that makes it interesting and that this is most often when the most interesting outputs um arrive or are actually being made. >> The brainstorming production level is so obviously a huge win for >> you can paralyze your brainstorming basically. >> Yeah. And yeah, I like that. Paralyze your brainstorming and and they have an [00:50:39] analogy for this. They do storyboards and some of the great directors Ridley Scott of Aliens and Gladiator was known for making his own. I also believe Spielberg was also like to sketch Raiders of the Lost Arc and some of these. George Lucas was known for collaborating with many amazing artists. Um even making miniatures and making storyboards for the Star Wars franchise. He had those people on full-time helping him with that. So that's the obvious place to start. But if we look at startups, startups uh always want to try to figure out how to do something cheaply. And people used to make a launch video for their startup for, you know, $100,000, $250,000. So they take their $10 million venture raise and spend $250,000 on a launch video. I've seen with [snorts] a lot of the startups I'm investing in now, they'll just spend a week or two working [snorts] with um you know, a director to make a launch video. You've probably seen this trend. Yeah. And I'm sure people use Flux and [00:51:40] some of your models for this. Uh have you seen this? >> Yeah, of course. Yeah. >> Yeah. What's your take on that? Because that feels like the early stage of storytelling. You're trying to communicate a product or service in a fun, engaging, punchy 30-second, 90second way. Yeah. >> I mean, like again, like I think we support these like exploration based on these tools, right? And I think ultimately it's great to see like all different kind of like >> I don't know like launch videos products being built on top of like the same kind of like base model or the same technology. Um and I think that's what's making it so interesting and also so powerful. >> Yeah. And what else are people using the technology for? I understand there's a Bitcoin movie coming out. instead of using a green screen in this Bitcoin movie. Um, I was talking to Galado, you know, the woman who played the actress who played Wonder Woman. >> Oh, Galado, of course. She I was talking to her at an event and she was telling [00:52:40] me um it was the Breakthrough Prize uh Yuri Milner's event and she was telling me she just did a Bitcoin movie and they did it on a sound stage without green screens >> but all the actors just worked in like a sound stage >> and then all of the scenery behind them was being done by Generative AI. That's a real movie. That's a $30 million budget movie. She said it would have cost $150 million if they had to build sets and the film would have never been green lit. Are you starting to see people use that in production? Not just in the back end and the ideation phase, but actually in production yet with your tools. >> Yeah. >> Um, yeah, we see some use cases like that in production. I think like high-end film production is kind of like the one of the like most demanding use cases. And I I think I'm glad that it's being explored, but I also >> really want to um like it's I think it's important to see that this technology is like on a trajectory and it's improving. It's improving rapidly. I don't know if I look back at like where we started like [00:53:41] a few years ago when I was doing my PhD in this field like the only thing that you could do was like images of 64x 64 pixels. Now you can do like multi-inut videos, right, at like a high resolution, but it's like it's not going to stop there. It's going to it's continue to improve and I think like then it's going to unlock like even more of these like high-end use cases. But I think the main thing >> before we get to that, yeah, >> hard to predict. I think hard to predict and I think ultimately >> couple of years >> ultimately I think you still want to have like the tool that enables like this human in the loop kind of >> of course. Yeah. >> Um production workflow, right? But I think when I look at multimodal generative models um as a whole, I think what really excites me is you can use the same kind of um AI model to make a movie and deploy that as a brain on a robot. Ah, and I think this is like this is so interesting. Um and I [00:54:42] don't know like there's like some thoughts around trying that in the digital world, right? which would be for example computer use remains to be seen if that is actually something that works or not but I think like the technology is so powerful and so versatile and it's just just moving into that and all the all the talk around like world models world action models all of that it's basically all the same and I think that's what's making it so interesting and what I find like most exciting >> so do you believe that the technology will be used to analyze or primarily to analyze is real world like here's a video of somebody you know making a sandwich. Now we have the robot study it and make the sandwich or do you think there'll be a lot of synthetic data made that then the robots will just study the synthetic or they're going to just in some way innately know based on all this massive amounts of training data. Um I think it's a combination of prediction [00:55:42] right and p prediction and is a way of you can think about it as simulation as generation it's um predicting actions which is you have to understand the input the visual inputs in order to actually predict a reasonable next action um and it's about perception it's like you can only do that if you understand if you perceive the content you then you can only I don't know like transform it into a new piece of content or predict an action or describe what you actually see in that um scene. And the and the combination of all of that is I think um yeah is I think what's what's driving it. There's not a single one of them. It's a combination of these thoughts. >> And what's the best way to get that training data? Do you need to have people put on glasses, get a firsterson perspective, have them put on gloves so you have that, >> you know, fidelity of understanding, hey, this glass is moving. I'm pouring this glass. I'm putting ice into it, you know, and and here's how that's works [00:56:43] and the splashing and the condensation water so I can pick it up and not drop it because it's wet on the outside. Or is it going to be just, hey, take the corpus of YouTube videos and the robots know exactly what to do because they'll find a thousand videos of people pouring drinks. >> I mean, ultimately, I think you would want to go to a place where you could like prompt a robot in context, right? as you can do with like a language model basically just tell it hey go and I don't know pick up this glass with the I don't know orange juice or whatever it is yeah exactly >> we're not there yet um but I think this this is like one of the goals and I think like how these models are deployed currently is there's like a lot of like different hardware different robots that are running in factories um that all have like some different kind of action representation that you need to kind of tune the models towards right So in practice um what you do is you have like all this like visual understanding in the models um and then you need only a [00:57:45] very little bit of like a few hours of um fine-tuning data to adjust the model on that specific task and I think the goal would be to kind of move away from that uh towards like as much in context as possible but it is a little bit of a research problem. Uh, I I think that >> open source is kind of having a moment right now. We've been discussing it on the podcast a whole bunch recently and people are also talking about sovereignty. You have companies that own incredible IP libraries. I mentioned Star Wars before. Disney owns an incredible library. What should your advice, what would your advice be to a company like Disney? Should they take your open source software, train their own models or work with you to train their own models to control it and then hey this is our IP? They've already made a point of working with chat GBT and saying hey you you can and cannot use certain characters. In fact, OpenAI had a relationship with them that's for Sora that's no longer happening but they [00:58:46] officially licensed on the output some characters. So how do you think about those major IP holders? What's your advice to them? Are you in discussions with them? We know about the Martin Scorsesei or tour deal, but how do you think about content libraries? >> I think it is um look, I think like the most interesting use cases of this like if you think about like content creation um is in generating something making something that hasn't been there before, right? Like that that's a fundamental like interesting aspect of this technology. And then I think like yeah when it comes to IP what we implement for example on like our public facing tools is you cannot generate certain IP with these models right and I think that's something that is a a sensible approach and then yes we do work with certain IP uh holders to develop models together with them um some of them based on our open source models some of them based on like our more powerful proprietary models but I think that is like a very like attractive >> value there what do you think that will [00:59:47] look like for consumers in another couple of years? What would potentially happen when you open up Disney Plus? >> I mean, that's a good question. I'm not I'm I'm not in Disney, right? So, it's up up to them to decide that. But, I think we want to enable them to build all kinds of stuff that they that they that they envision. And I think we can support them. We can support like other companies in that space to >> I don't know integrate the technology in the best possible way. I think like one of the very interesting angles of it is that it is like it's becoming much faster. It's becoming more interactive. I can imagine like a whole bunch of like very interesting interactive content creation tools that you could host on Disney Plus Plus or elsewhere. >> I think the most interesting thing I've seen in this regard is uh fan films, >> right? So there's a category before generative AI fanfiction. People would write their own Star Wars story. Then there came fan films where people would dress up as Jedi Knights and record their own films. And George Lucas said, "As long as you're not doing it commercially, you're not selling it, I [01:00:47] give you permission to go make Jedi movies." And they even released how to, you know, how-tos on how to make a lightsaber or, you know, sound files of like how to make the lightsaber sound. Now, people are taking the stories that haven't been told from the Star Wars universe and they're recreating them using AI. And for the fans, they're becoming quite popular on YouTube. Uh Star Wars Stories Untold is, I think, the biggest one. It's getting millions of views per video already. And I think that's really the future is letting the customer base pay a licensing fee or pay a fee, uh maybe rent software or maybe based on the output and let them be creative with the characters, let them make their own stories, and you could be in a unique position to empower that. >> No, 100%. I think like if you find like a model that works for like the um IP owners uh but then also can enable like these super like creative customization [01:01:47] use cases I think that's great. Yeah. I mean like like I mean like for myself like I when I read a book or whatever like watched the movie I had like so many like ideas how it could be done differently or this could have happened right this is like so nice that you can actually enable people to visualize these ideas. >> Yeah it's uh going to be incredible continued success with it. You have an office in San Francisco. You're hiring people? Yeah, >> we do. Yeah, we just >> we raised a bunch of money. >> We raised a bunch of money. Uh we just crossed 100 people. We're hiring in Germany and in San Francisco. >> Fantastic. Who are you looking for? What's the right type of person, the right type of scale? >> Yeah. Um on the one hand, we're always looking for researchers who have experience in large scale model training. Um experience in diffusion model training, flow matching training. We're looking for engineers who want to be working with the customers to you know develop these like customized um physical AI solutions or for example with like a IP owner like develop these models jointly with them. [01:02:48] Um we are looking for engineers who have experience in just like large scale compute infra managing that um and making sure that the training runs runs smoothly that we maximize our MFU and all that um and we're looking for people who have interest in you know like uh getting the technology out there >> in the hands of people >> the the the for deployment of this there's just so many great ideas and so many great partners for you I think you're going to with the open source specifically. You know, it seems like the corporates really want to have some additional level of control, but they also need the frontier models or your proprietary ones for some of those refined features. So, I think you have a very bright >> future. All right. Continued success. Thank you so much for doing the show. Pleasure. I'm going all in. [01:03:51] [music] I'm going all in.