AI Scientists Are Here: Autonomous Labs & Synthesis Superintelligence — Periodic Labs

[00:00:00] But also we feel like even if like Fable 7 gets really good at um like even better than what it is today, you will still have to [music] run experiments to get results. Um and the reason for that is right like machine learning is really good at what's been trained [music] on. But scientific discovery is almost by definition what you haven't been trained on. And that's why we're building these labs so that um whether open models or closed models can use these labs to tinker with the university. You don't feel like you can [music] make a big discovery without trying things. >> Yeah, there's there's not going to be like no one's going to like zero shot in the room super extra. >> Before we get into today's episode, I just have a small message for listeners. Thank you. We would not be able to bring you the AI engineering, science, and entertainment content that you so clearly want if you didn't choose to also click in and tune into our content. We've been approached by sponsors on an almost daily basis, but fortunately enough of you actually subscribe to us to keep all this sustainable without ads and we want to keep it that way. But I just have one favor to ask all of you. [00:01:01] The single most powerful, completely free thing you can do is to click that subscribe button. It's the only thing I'll ever ask of you and it means absolutely everything to me and my team that works so hard to bring the Inspace to you each and every week. If you do it, I promise you we'll never stop working to make the show even better. Now, let's get into it. Okay, we're here at Periodic. We're here today with Liam and Doge from Periodic. Welcome and thanks for having us at your >> Yeah, great to be here and yeah, thanks for coming in. >> U, I want to start off with one of these like quotes that I love from your website. It says, "Intelligence is necessary but not sufficient. New knowledge is created when ideas are found to be consistent with reality. It seems so straightforward, but why is it non-obvious? >> I think that's sort of the the thesis and premise behind periodic that Doge and I thought when we were building this, which is you can't just think your way to a solution, um the universe is so [00:02:01] complicated that in order to actually push the frontier of knowledge and to make progress, you need to create these conjectures and then actually see whether or not it holds. um no amount of rereading that textbook or paper and just thinking and like disappearing into a room is going to allow you to think through all possible um experimental outcomes um expected and unexpected and you really need this like iterative process and that's our core belief for putting this together and that's why from the very beginning we're like we need to have the AI systems the simulations of the physical world but then also to build up the the physical high throughput experiment and we think a different type of intelligence emerg emerges from that. >> I think it's also notable like with OpenAI and and Google, you don't have that resources in those big labs like you have to had you had to come out and do your own thing. Uh because you you are kind of inventing your own playbook as you go along. >> Yeah. I mean a team like this has never existed. I think the we try to bring together solid state chemists, solid state physicists, experimentalists, [00:03:01] theorists, hardware engineers, LLM experts, computer scientists. Um and some of these technologies are very recent like the high throughput experimentation, the robotic arms have only been tried in the last three years or so. The force field expertise, some of these force fields are also very recent. Um but yeah, we felt like having a lab is very important. Having this focus and bringing all these people together to work together is very important. Um, and I guess like the closest examples from history are places like Bell Labs where incredible theorists and experimentalists and chemists work together and achieved incredible things. So, we're trying to do the same here. Yeah. >> I like want to talk about force fields in a bit, but before we get to that, yeah, like what are they? Um, but before we get to that, I think one of the just for context, one of the biggest differences between, you know, some of our audience are uh AI, you know, engineers and some of the audience is scientists. So let's go and talk to the engineers real quick and say like what is the difference if you've been working at OpenAI as a big lab what's the difference between what you do there and [00:04:02] what is you know what is here I think you've already hinted at this but maybe explicitly what's what's the >> how do you have to reframe your thought process >> well I mean I think like one interesting thing is now our reinforcement learning environments literally derive from the environment from our physical labs uh our data comes from our physical labs And this is sort of our ultimate truth. It's not enough just to do optimization against um some answers that were known in some papers or textbooks because we're going beyond that. And I think that's one of the biggest differences. But along with that, there's issues of say decision-m under uncertainty. So when you're doing optimization against math, there's a high precision to it. You're not really dealing with, you know, kind of like variance or uncertainties or, you know, aberrant measurements. Whereas that's very key to to our process like um for example when we are doing materials discovery loop things don't come off the out of the furnace labeled right even the labeling process can be stochastic and noisy and [00:05:04] um sometimes you'll have aberrations between different machines. Uh maybe we'll have insufficient telemetry. So we thought we ran it at um this temperature but really the temperature was you know some delta away. and uh an intelligence that can take in this noisy data and make intelligent decisions like a scientist would um is a sort of a different set of reasoning strategies. So I think there's a huge amount of commonalities. So the standard mid-training reinforcement learning the construction of um tool using agents uh variance reduction um you know making sure infra doesn't have mismatches between like training and inference. But we have to go beyond that and really think about how do you do accurate um work when there's a high amount of uncertainty? How do you make really incredibly efficient use of a limited set of data? So we're scaling things up significantly, but still it's it's very different than a digital environment where you can arbitrarily add more [00:06:04] environments or more rollouts. We don't have that same capacity. So sample efficiency is another key thing. This seems a little bit abstract to me and it is a theme which has been on the science pods for a bit. But uh I think it might be helpful when you talk about experimental uncertainty. Um to to explain like what would a specific process that you were doing experimentally look like and like what would a human how would a human approach this problem before we jump into how do you go into the AI side. Do you have like a specific example of a type of material you make and like what sort of uncertainty in the measurements you would experience in the process? I think there are many dimensions to this. So one of them is I think every scientific experiment has to do some dimensional reduction and this is very different than coding or math. So um when you're doing math or coding all the context could be available to the human or the LLM. It's a bunch of axioms some corlaries people have derived from them. It's a bunch of like functions and API calls. So everything you need to reason through is available to you and then you just have to be very smart and like [00:07:04] figure it out. In physics, as you know, we start with more atoms than we could ever store on a computer. So clearly, we'll have to go from the original number of dimensions and bits that represent a system to the number of bits we can fit in the computer. And you know, this was I guess the original premise of thermodynamics. They were originally really confused about how steam engines worked. But then turns out you can find like five or six thermodynamic variables that explain what's going on pretty well, which is incredible. That's the beauty of physics. Like you have a room full of atoms and there are you know 10 to the 23 or 10 the 27th or something atoms in this room and yet you need temperature and pressure and you know simple variables and that tells you everything like tells you mostly what you need to know for most cases right >> so like let's say you make a material and you could do something like look at an X-ray structure and this thing is a very lossy um uh projection right it's like not actually a structure or doesn't tell you the true structure. It tells [00:08:04] you some of the principal components of that. So, uh how would a human take these simple principal components and then um kind of reason about them and then how do you extend that to an AI? >> So on the theoretical physics side, right, uh we've decided that energy the average value of energy and the fluctuations on energy is very important. So a lot of thermodynamics has been ded from this and then we realized reaction barriers are also very important. the kinetics is very important. Um so a human kind of tries to reason through this extremely complex system by coming up with some you know reduced dimensionality descriptors. Um people also really like automistic picture like do you think about what the local neighborhoods look like whether they from octahedra tetrahedra and that especially chemists really benefit from that. Um okay so that's on the theoretical side and on the experimental side you would try to collect as much data as possible. So you'd have a lab notebook and you'd write everything you see etc. But with the understanding of [00:09:04] course that you'll miss a lot. Uh so like as Liam mentioned there are all these issues that come up. For example your furnace will degrade over time because as you're using the furnace some of the things you're baking will evaporate and cover the heating element. So every time you use a furnace it gets worse and worse. Uh another issue is like you have a furnace but the temperature isn't perfectly uniform. So where you put in the furnace will affect the result. We we don't have this issue yet but we will maybe someday. The optical instruments uh suffer a lot from vibrations. So like every time somebody walks like I remember when I was doing my PhD one lab had this big issue because sometimes their results will be very different than other times and they brought it down to at a certain time in the night upstairs somebody would be walking and that vibration affected the laser setup. Um, so yeah, science is very hard, but that's what makes it special, right? Like the other thing Liam and I have been talking about is a lot of the current improvements focus on math and coding and theoretical computer science because it's easier for LLMs. But in real life, most things that require intelligence are actually more [00:10:05] like science. Like there's other uncertainty, there's a lot of noise, um, there's a lot of missing context, but you have to be the intelligent being and figure out what to do next. I think like a really interesting point to build on that too is the optimal reasoning strategies to do math, to do theoretical computer science, to do these types of things may not be the optimal reasoning strategies to do science. >> Yeah. Mhm. >> How do you define a RL reward function when uh your output is a noisy crystal structure or um some number of I don't know like collective variables or something like >> maybe um I'll give uh one part of the loop and I'll just kind of dig into that. So uh as part of the materials discovery loop, we have to first figure out uh what to make. So that's the stability of the atoms together. Um as well as do we expect it to have the properties we we want. So we don't just want a a novel configuration of the atoms. We want them when put together to have the property of interest. Next is how do you synthesize it? How do you actually make the thing? Um that's also [00:11:05] non-trivial. So even if you know something will hold together the process of actually you know what are the processing conditions to actually bring that forth is highly non-trivial but again like the thing once you've done those two steps and you've made something it doesn't come off labeled so you actually have to characterize it and figure out what you made. So in that case, rather than thinking about the oral environment of we're just going to kick off an experiment, wait a couple days and try to do like a update on like did you find a room temperature superconductor or not? That's completely infeasible. >> I was going to ask that. Yeah, like you can't leave your GPU sitting for a week or something. >> Yeah, it's completely infeasible because you don't have enough agents to reduce the variance sufficiently. The rollout time is going between the different instruments. You're waiting. there's just some physics limits of the experiments um for like the actual like um synthesis or furnace time. Uh so it's just too too slow, too noisy. So the way we think about the AI program is basically constructing agents around [00:12:05] this set of data that we've produced. And so in the case of characterization, you're looking for RL environments that are rewarding the identification of the phases actually present. are you able to um given the raw experimental data fit that effectively? So the way we do this is we'll shoot X-rays um at the material. Uh X-rays have a frequency or a wavelength roughly measurable or commensurate to the spacing between atoms. So you get nice diffraction patterns that kind of give a fingerprint of the crystal structure and doing this um identification of what actually is present is highly non-trivial. And so some of our early AI work has been on basically producing systems to do this. And this allows us to move much more quickly too because if you're running so many experiments, you really quickly become bottlenecked on your ability to understand what was produced. But again, now the reinforcement learning environment is fairly straightforward, which is you're going to reward the [00:13:05] identification of the phases present. um you're going to penalize spurious phases or things that don't have any like real chemical plausibility. Uh so that that's like a you know a little succinct thing where you can kind of do this for the different pieces and when you stitch this together that's the end to end discovery loop. >> Okay. >> Maybe like another another piece would be >> as we build up uh a large basket of experimental data you can think of another thing of um basically like timestamping the state of the world. So you could say at this date this was our experimental evidence up until that point and you can construct reinforcement learning environments where you say okay given that body of experimental evidence and given you know the choices what what was like the the next choice that the scientists made or what was an outcome of that experiment. And this is really interesting too because this allows us to do types of programs that would be harder externally because if the pre-trained model has [00:14:05] already memorized some of this data and then you try to do reinforcement learning on that if it already knows the answer and you construct reinforcement learning tasks that depend on that answer it can fake work. So and then I mean because it's like oh well it already knows the answer so it doesn't actually have to do the hard physics reasoning. it doesn't have to actually do the the calculation simulations and it gets to the right answer and then you reinforce that policy and you upweight those reasoning strategies those reasoning strategies will not generalize to novel system so it's not useful to us but as we build up baskets of experimental data and like stitching things together having the lineage the whole process that gives us the ability to construct new types of environments that are just not plausible elsewhere >> I want to ask my question but I'm worried that I'm taking it off the rails I'm just going to go for it cuz I again I I I come from the non-scientist but engineering sort of I'm familiar with the ML side but not the not the physics side. Okay. There's there's there's there's a few versions of this but I think the basic question is um don't we [00:15:06] have like most of the laws of physics worked out? Uh and like how come we don't have perfect simulators already? >> A great question like this very basic like and he was like haha that's cute but like dude like like like I have all these physics books what do you mean what do you mean we're not done? >> [laughter] >> Yeah. So there there are a couple of things going on here, right? So >> I think first of all we're definitely not done with any of the law of physics like even the >> like at quantum scale okay but and like maybe like you know very very large scale but like at human scale at like you know material scale we're we like what's what's left? >> We're not so [laughter] it's a really good question and it's very interesting why we're not done yet right. Um so there's a very um common story people talk about like when Dra was figuring out quantum mechanics like late 1920s he wrote a textbook about quantum mechanics and he kind of framed it as it's all done and then there's a phrase people like to quote I'm not sure actually how accurate it is but apparently Dak said [00:16:07] the rest is chemistry the idea being like he could solve the hydrogen atom compose everything yeah he could maybe solve like a chain of 1D hydrogen atoms but then he cannot currently solve say nitrogen interacting with oxygen. But that's okay. That's just chemistry. >> Physics physicists really love single like atom like really simple systems. That's all you can solve. >> Spherical cow. >> Yeah. Spherical cow. >> But it turns out I think what we learned in the last 100 years is first of all that's not true. Um it's not like a simple extension of what it was to do just hydrogen and it's not just the rest is not just chemistry. There's actually a lot of physics there. I mean I guess one of the things we still haven't figured out is high temperature super conductivity but there are a lot of other >> and high for you is like 175 200 >> when people say high temperature super conductivity what they mean there is unconventional super conductivity so there's the conventional superc conductivity that's just mainly driven by electro phone coupling >> and in that case you can kind of see the isotope effect like if you take the same system but just different weight for one of the elements you can see the superconducting temperature drops at the [00:17:07] rate you expect. So that one is conventional. But then higher temperature superconductors like couprates like the ones that are about 77 Kelvin, like 93 Kelvin, turns out don't obey that physics, but we don't know what physics they obey. Um they're just incredible superconductors. Um but it's not just that. So that definitely is one of these like very uh popular topics. We don't know the theory for yet, but there's so many other things we don't understand. Like for example, even strong correlation in simple quantum mechanical systems, we cannot simulate yet. Um, density functional theory is a very powerful tool. It's incredibly accurate on some things, but as long as soon as there's like some strong electron correlation, it actually fails to capture some of the effects. I mean, there's so much to figure out. That's why I'm really excited for AI to be applied to this field because theoretically, computationally, and experimentally, there's so much to figure out and every improvement we make here should improve human life because, you know, the better we can understand materials, solid state physics, the better devices we can make for them. Yeah, there's this uh famous quote by Phil Anderson which is more is different which is basically you may understand [00:18:09] all the basic laws of something but when you add many things they behave qualitatively distin differently from how the simple like physics should tell you. So understanding how lots of things behave together it seems like it should be simple like beginning laws are simple but the collective behavior is so complicated that it's really hard to model. It's like emergent and universal which is crazy and we're also seeing this with deep learning models right like we see power laws everywhere in deep learning we don't understand but it like is very reminiscent of physics where there's emergence and universality um >> there's been many theory papers in the physics world about emergence in neural networks >> absolutely yes >> and then my my other follow-up question was also just on the process of um let's call it the the process and the end result like let's let's call it like you want to have these properties that you're targeting for materials and then you have to figure out the process to get there. Is it worth se separating these things so that you can have like you just train a model that perfectly reverse engineers any process whatsoever given uh whatever [00:19:10] theoretical end state you want. Uh is that uh meaningful? >> Maybe going back to the conversation we had about how since there are more than a number of atoms, we'll never know the exact context of what happened. I think that might be a reason why we'll never be able to reverse engineer everything perfectly. But we're just trying to reverse engineer sufficiently to be able to improve the performance of the materials basically. >> Yeah. What you're asking is is there a forward model where you put in an input and you can predict the output and you want to you want to create a like a surrogate model which can >> like the input is like the the final state of that this configuration of atoms and then the prediction is >> you can split up the work almost like you can have one team do the process side and then the other team do the properties side and then you know just [laughter] >> raise them. >> Yeah. I I think that there's uh a lot of progress we can make in terms of these like you know splitting up these agentic workflows to these different areas. Uh I think there's a lot of like empirical work we can make uh on on synthesis prediction as well. [00:20:10] >> Yeah. Um where it's like you know taking in that that configuration of atoms what tools can you use how gi given this experimental evidence given prior literature prior papers how do you actually get to these things and you ultimately are testing you know are you able to actually replicate it or not >> the the reason I say this maybe also is just thinking about it in terms of a company um you know the way that TSMC is kind of like a fab for semis but they don't design the semis um you know you could be the TSMC of materials where like people come to you with like well this here's what I want and then like you can figure out the process I said. >> Yeah. It's like a matter compiler. >> Yeah. >> It's like it's like given these these requirements. >> Very Star Trek. >> Yeah. Yeah. Yeah. But it's like given these requirements, is this actually like a valid set of configurations? Can this can this exist? >> Is that a good objective function? I don't know. It's too too big. >> Yeah. [laughter] Well, I mean, we kind of talk about like one of our missions as like synthesis super intelligent. So, I think it's in line with this. >> Yeah. I want to double click on something that you said a little uh a little while back which was you talked [00:21:10] about phases and I think this is you know going to um Doge's comment about uh what is uh like emergence like the the concept of a phase transition I think might be foreign to a lot of the audience. So can you explain what is a phase transition and then why is this like a helpful signal for something like an RL environment? >> Phase transitions are fascinating. Uh I mean I think the maybe the one that people can most easily relate to in their life is uh ice melting probably or water boiling like you know you increase the temperature of ice and it still looks like ice and it behaves like ice but there's a certain temperature at which just stops raising its temperature and then turns into liquid. So it suddenly goes from this solid phase to the liquid phase. Um another one that's very relevant to us of course is the icing model. uh and I think computer science and mathematicians also study this maybe under different names but basically uh you can have spins up and down and they have different interaction terms if they're both up versus one is up one is down and at high enough [00:22:11] temperature they're usually just randomly up or down entropy wins you lower the temperature it's still looking the same but then at some point uh suddenly they perfectly start aligning yeah the phase transition pace are really nice because it makes physicist life easier for studying certain things they tend to have certain um spatial correlations that really help us um as well. Okay. So in our lab uh phase transitions come about usually because we mix the precursors that are different um crystals basically but then we raise the temperature or do something else to encourage them to react and then atoms start reacting and they form into this new crystal. Um and the this shows up in the XRD pattern because usually if the geometry of the atoms change drastically the XRD pattern changes a lot. But what really happens in like a typical practical material discovery campaign is when you first try something it doesn't work and the outcome isn't just like a clear yes or no but it's usually like a very mixed phase. Uh it will usually have some of the precursors. It might have some amorphous phase and then it [00:23:12] will have like a bunch of phases that you maybe didn't try to make. So that becomes a challenge and that's why we found it very difficult to bring together simulations and AI to do the characterization. So the simulations can always say, "Oh, this phase doesn't look like what we predicted, but it's expected var variation of it. Let me now do force field calculation on it to see if it's still stable." Or like the AI can say based on the previous experiment on this one, this is probably not the phase we want to make. So let's change the conditions. >> For listeners, XRD is X-ray defraction, which you already described. >> Yeah. In some cases, there's like a phase that just hasn't been recorded before. It doesn't exist in any papers or database. And so the AI has to actually go and sort of put use these tools to say well what configuration atoms would actually explain these these phases. But this becomes incredibly important for directing the scientific process because uh let's say we have a phase target in mind we really need to sort of hill climb that to get better measurements on that. So if it's uh 1% and we're like actually we want to increase this phase period. We want to [00:24:13] have a very good measurement on this experimental data to say you know what is present what's not. So one of the disadvantages is you have just a distinct like variable that you can just observe. Like going back to the statement about how do you deal with uncertainty? One of your it seems like part of your answer is you make your observable signature somewhat unamiguous. There's no interpretation there. Is that right? Is that like the logic behind this or is this like literally you just need to define a new phase and so this is the thing you care about? >> I mean I think there's still ambiguity in a lot of these cases. I mean like you know a very dumb thing is like replicates. replic two different uh phases can't actually be consistent with the same pattern. And so I think that's why it's really important for the system to use some chemical intuition to say like okay well this would be highly unlikely given the synthesis conditions um given this the [00:25:15] prior knowledge um and so that can be used to help disambiguate >> so you're so you're somewhat injecting priors there um based on what you expect. Yeah, I mean I think thermodynamics is the biggest prior, right? And then physics is a big prior. Another one that we really benefit from here is multimodality or materials characterization. So like we can do XRD and yes some phases might look similar on the XRD pattern but then we can also measure their electrical properties, magnetic properties. we can measure their morphology like some kind of electron microscopy and then some of the phases that look similar on XRD will look different in some of these dimensions and that that kind of multimodality really helps and again AI is really helpful here because like humans have limited context and computation power as well so if you give humans like 10 different modalities at the same time and say analyze these all consistently it's a bit difficult but it's actually not super intelligent for an AI to do it it's great yeah it's like the super intelligence is it can just do so much calculations it can look at thing. And this is kind of gets to that um phrase about like better decision-m [00:26:15] under uncertainty where now it's stitching together um the signal across many different instruments longitudinally across different experiments across the replicates and kind of getting to this like underlying better model >> rather than just overindexing on one measurement from one instrument. And let me just like um so you you know you started I think a while ago also saying that there's just more data than you can fit in any any reasonable computer or process or anything. So at some point you have to throw out data even though you have all these replicates even though you have like presumably like 10 different ways to measure the same thing because what if one of your temperature things is wrong? Um so like when do you throw that out? How do you decide what to do as a data hungry ML guy? I just want everything right and then I just you know waddle learn everything. I I don't know what I don't know, you know, just just throw the machine at it. >> Yeah. Yeah. So, we don't throw away in data. Um I I think maybe what I meant was potentially like if if God was observing the experiment, there is more data in the experiment than humans can fit in a computer. Unfortunately, us [00:27:15] mortals cannot observe all the atoms. So even though there's more data than we could store, we actually can't access that data. For example, I cannot trace all the atoms positions. Um so unfortunately we're still in a regime where any data we collect is very precious and we don't delete any of it. >> Maybe a way of thinking of it to connect to the AI side is like um think about probing like linear probing of a of a foundation model right this model has all this data. So God has this entire picture of what the material is doing and then the experiment is this little tiny linear probe which has like eight eight bits or something of output. And so you're taking this humongous thing and course graining it down to a little bit of information and then it's now uh either traditionally human's job or now your agent's job to reconstruct from those eight bits of information what is actually happening inside >> like it's say analog where you can't have full visibility over every parameter weight over every activation for that input. Um yeah you're getting like some sort of subset on it or some [00:28:15] like uh you know coarse grain features across the whole thing. >> Yeah. But you probably find that you probably do find Maxwell's demon argument very interesting because that kind of >> Can you can you recite it because I I don't uh he's mentioned it before I I keep [laughter] forgetting >> like so you know in the 1800s uh people realized that entropy of a system has to increase or like a for a closed system or stay the same but it can't go down. It's the second law of thermodynamics. And there was a a thought experiment that was asking what would happen if there was a very tiny all knowing demon that could kind of look at atoms and then whenever an atom had low energy kind of open a door and let it into this other room and keep doing this to reduce entropy of the system which would break second law thermodynamics and for a long time I think there wasn't like a very satisfying answer for why that was the case but then there was a more recent paper in the 1900s um more recently to us uh Landau came up with the argument that you'd have to spend energy to [00:29:16] delete information and Maxwell Zimmerman would basically have access to so much information by doing this thing he's doing that he'd have to delete some and for when he's deleting information he'd have to spend energy which would then make sure entropy increases so even though we're not at the level where we can be Maxwell's demon I think your uh question is like very visionary towards like a future where we store so much data we have to delete some >> yeah no I wasn't I wasn't obviously going to that to that extent uh it does remind maybe with air conditioning. That's kind of how air conditioning works. But also, I think it's just like uh a question of like why don't you just buy every sensor in the world and just like, you know, over ridiculously over instrument over replicate everything. >> Oh, we do that, >> right? Okay. Yeah. Okay. >> Right. All right. >> Yeah. [laughter] I think it's a matter of Yeah. Exactly. Are you getting visibility to individual atoms which is infeasible? But no, we add a huge amount of telemetry and that basically reduces the number of like hidden variables if you will. >> Yeah. Just just out of curiosity, it does like do you worry that you're just in like California? Like don't you want to do this in like Nepal and like [00:30:18] Australia? Yeah, we we could I mean we we we have plans to you know open more labs >> because obviously gravity and like you know where you face and where like where you are matters >> also like expertise can change right like here we have certain kinds of expertise for example Stanford physics department engineering department has certain kinds of expertise that really helps us because we can hire from there our resources can collaborate like as you said like geography can matter but then different cities can have some different sets of expertise in physics chemistry >> yeah I mean we have zoom for that. [clears throat] [laughter] >> It's just like your experiments are going to be affected by like where you are. >> Yeah. >> Yeah. I mean the future of periodic is is not a single lab and you got to hit up like your buddy Jeff and like go to space and like >> Yeah. I mean [laughter] I think it's it's necessary to have this sort of network of labs and and like yeah I think as D just kind of pointing out like there's an optimal place for each of these based on expertise or you know materials constraint. There's many different constraints permitting. Um okay so one of them is uh so one thing I [00:31:19] I've been curious about is how do you deal with computational tools or like how do you integrate computational tools into a decision-m workflow um especially when let's say computation and uh experiment don't necessarily you know meet um also like the theory in terms of like highle like understanding and uh yeah how does that sort of work together >> yeah so I mean one thing that's really helpful here is there's certain things that are easier to do in simulation and more accurate in simulation. There are certain things that are easier to do in experiment and more accurate in experiment. I mean in general of course experiments are more accurate but certain measurements are very hard to make with experiment. So you make a bad version of it. So we get inaccurate data. Um like let me just give you some examples. So one of them is um we use density functional theory to estimate formation enthalpy of uh materials and >> sorry quick pause. Uh can you explain densal fun density functional theory and form information entropy? Sorry. Enthropy. >> Yeah. Yeah. So uh density functional [00:32:19] theory is probably the most commonly used simulation method for materials. Um it kind of comes from con honing theorem and con got a Nobel prize for this. He when he realized that sure quantum mechanics is very expensive because you basically have this exponential Hbert space for every wave function. So you have to solve this like very difficult problem especially as the system size grows. Con and Honenburgg realized that turns out you don't need to deal with that exponential space. All you have to do is think about the charge density at least for the ground state of quantum mechanical systems. So this is it's actually the the theorem is pretty simple. So even I think an undergraduate in physics can understand um but the idea is all the properties of a quantum mechanical system ground state is a functional of the charge density which is amazing because charge density is just this three-dimensional object whereas the Hilbert space and wave functions are exponential um so that was a really interesting observation and then con published another paper I think [00:33:20] with his postto sham cone sham wave functions kind of had a practical way to solve involving some approximation of quantum mechanical properties of materials and today we use it very much both in our lab and externally and information enthalpy is one of the measurements we can make to understand the energy of the material. Like basically we we want to ask um what is the energy that this material has because if that energy is higher than other materials these atoms can come into then it's unlikely that this material will be able to be made. So stable just means like it's on the convex hole of other materials uh and their energies. Um was that clear enough or should we >> I mean maybe a simple like highle point uh so so like density functional theory is a way of taking something which is exponentially hard uh in number of electrons or number you know in your system size and then reducing it approximately with some introduced error to like maybe usually an incubed [00:34:21] approximation. Um so now you can compute quantities which are you know you can now compute things which are just uncomputable but at some cost right if you could do it perfectly you would we wouldn't need the lab but you know we can't this might be a good call back to our actually our second episode ever Heather Kulic had said um uh she famously said there is no alpha fold for materials not just computationally but the ground truth just doesn't exist like we don't really know what material crystal structures look like and DFT is generally our best route towards >> yeah I think so I mean maybe two things I would add is DFT by definition doesn't have to be an approximation like the con honing the term shows that it can be exact but I think as you're pointing out the exchange correlation functional we don't yet have access to and then for the charge density based models we don't know what the kinetic energy functional is and then even if we had perfect DFT I still think we'd need a lab because we even if we have perfect DFT we cannot fit 10 to the 23 atoms in the computer. [00:35:24] >> Um, >> you're very obsessed with that. [laughter] >> No. Okay. Okay. Yeah. But but with exponentially growing compute, you can argue that an incub scaling size eventually you could in principle calculate things. You just wait till long enough. Yeah. Yeah. Yeah. But as long as incubed is a good scaling, you can you can >> uh as the non-scientist I I will make an observation which I I don't know if you want to throw around or anything like that. Um so I I came from finance. Uh we had the Gaussian copula which was a way of pricing like uh credit default swaps uh by doing correlations and like reducing everything into a single kernel. Uh and it's very it feels also spiritually similar to the VAE um in terms of like like condensing all these things into like a single sort of parameter. Uh I wonder if just like there's like the same trick everywhere. Yours sounds you know a bit more highdimensional than mine. uh which [laughter] you know like VA is just like you know one sigma one sigma thing but like uh sounds the same [laughter] >> the charge density file is still quite a [00:36:24] big file. >> Yeah right >> you hand wave a bit but like it's good enough like this is this is the highest order bit like yeah >> and I I personally don't know of a better method to predict the stability of a new material. It's definitely not perfect but it's definitely better than other methods I can think of. Um and yeah so going back to your question the reason we use them is for certain things it's easier to simulate than experiment and they kind of complement each other well uh and then we can like do this in a loop like we feel like simulations will never be enough by themselves but in the loop of simulations AI and experiments I think we can make progress much faster than before >> one thing that simulations let you do is scale up much more quickly I mean because now like compute is much easier to bring online than bringing in bringing on new labs, how do you avoid or let's say you scale up a bunch of DFT, how do you avoid in having your models overindex on that and and and still be grounded like that the the real world is ultimately the truth you care about if you you know how do you [00:37:26] >> scale up the lab as well? Yeah, you sc the lab but I mean you could I could imagine scaling up DFT to millions hundreds of millions given modern compute whereas your lab I would imagine you're hundreds thousands a day I don't know like something still like fairly even in a really high throughput case like you know you're still fairly li limited by comparison right >> yeah I mean one thing is the the humans but also the LMS are kind of pretty aware of the limitations of DFD for example Even if we can do as you said 100 million trials with DFT, we could never actually try their microructure. And microructure is this idea that like DFT usually simulates the perfect crystal but crystals usually actually aren't perfect and their micro structure which is the structure at like a medium order can really affect the properties. Other issues of course like some properties like superconducting temperature cannot be easily simulated by DFT. Again like even if you do 100 million trials with DFT it won't be enough. So we have to rely on huristics, [00:38:27] experiments etc. >> And so there's a huge filter from all those calculations to actually what gets executed in the lab. >> Is that human mediated or LM or a mix? >> Can be a mix. Like for example, for a long time materials project which was the best open-source DFT database use experimental calibration for DFT. So they would actually use experimental data to calibrate DFT. Uh we also do this internally here like whenever there's a certain chemical system we're interested in you get experiments you get simulations and you can calibrate them. >> This is another thing um I've talked to a lot of biologist friends and they are always very confused when you say materials you can't just do an XRD structure and actually know what the structure is because if you're in the biology world you can look at a crystal structure of a protein and typically more or less with like sub with order instrom accuracy reconstructed. What is the difference between the materials XRD um you know X-ray uh X-ray diffus no >> defraction [00:39:27] >> defraction sorry experimental XRD for materials versus let's say the biology world like what information do you lose and why is this sort of like a lossy projection >> it's hard not to sound like crazy when answering this question but do you not feel like um organic chemistry and biology almost stem from a lower VC dimension like lower commal complex lexity model like you can almost represent as a 1D sequence and it can almost be compiled. So like there's a sense in which it's a simpler in the commod complexity sense >> 1D >> I mean like just DNA is like just or RNA >> I think most people would argue it's more 2D but >> okay I'm not a biologist so I'm sure they're right but uh with inorganic chemistry virtual is crazy it's clearly not coming from such a low complexity model because that 3D inorganic crystals can host metals insulated ers superconductors diamond it's like really [00:40:27] crazy different like people tried but we could never find a 1D representation right it's very hard to find a smile string for 3D or inorganic crystals and the way the atoms interact with each other is quite strange like coalent bonds ionic bonds metallic bonds um but yeah this it's very interesting um I I I wonder sometimes like you know have you ever looked into issues with solid electrolytes like why people cannot replace liquid electrolytes in batteries with solids uh there all these dendrites that form like basically the lithium will form these structures that go into solid electrolyte and crack it and you look at that and you think this is so primitive compared to biology like in biology these systems can do such complex nano technological progress whereas we can't even like make two interfaces work with each other because lithium will break it um yeah but then I guess inorganic stuff can be very resistant to high temperature you can make space shuttles you can make silicon can compute Mo's law. So, yeah, it's a trade-off. >> Yeah, we've we've talked about this with [00:41:28] several other guests. I think one of the the big differences in my mind is that biology we have a toolkit that you know that you can basically borrow like from millions of years of evolution. Yeah. >> Uh that has already given us all the tools and we can just reproduce those. But also evolution also constrains the yeah the VC dimension to be fairly low for like proteins and so on. The like emergent beyond behavior of large scale systems like cells can be much more complicated. >> Yeah. And consciousness. >> Yeah. Yeah. Yeah. But maybe even just like more specifically for um like if you just look at an X-ray, you had a perfect crystal and you shoot an XRD at it and you get the, you know, you get the spectrum that still doesn't actually tell you uniquely what the crystal structure is. Right. No. >> So, and that's something that >> what >> it's just average. It's just average behavior. But like for example disproportionation is a very complicated thing right where it's possible you have the perfect crystal and on average it looks like this perfect XRD but in [00:42:28] reality there is some other pattern going on where like atoms slightly shift to the right or left but on average they're in the middle. Um yeah it's a difficult life. Yeah. >> Whereas with proteins you basically know there's some sort of underlying protein with follow some rule. So you can you can combine a forward model with it, >> right? And then you can kind of move and move towards like you know single crystal XRD and that that can help. >> When we started the science pod there were all these like elements of of SC yeah in science and math yeah physics and all these things and we there was this like kind of pissing contest like which is harder and like I think like you can make the scientific argument that material science is the hardest. [laughter] >> Well, we didn't pick it necessarily for hardness. >> I mean there's actually like a lot of areas where it is easier. I mean we we have these like great simulators for large classes of materials. Uh whereas in biology it's incredibly difficult to model a cell or an organ or a full organism. Uh so I think that's that's a huge advantage. >> Yeah. That isn't that interesting that at the smallest level it is one-dimensional or two dimensional but [00:43:28] and then like it just has many orders more scale and structure and like and that actually introduces the complexity. It's that's weird like [laughter] whereas like maybe maybe complexity in materials is more the microsructure. >> Yeah. I mean I still you still get like different complexity at the different uh length scales for materials as well. So yeah micro structure and and beyond. >> Yeah. >> Uh just sort of like a brief like sort of pallet cleanser in terms of science fiction. Let's let's say you can invent what whatever materials you want. Um what is more valuable? Um you know I have I have a list here. Uh there's obviously room temperature super superconductors. Uh but also you know you mention batteries. I've always thought like batteries just like do better than lithium ion, you're good. Like you know that's that's stood around for like 100 years. Um and then the last one is carbon nanot tubes uh mostly for the uh space elevators. I I don't know if like those are those are three that come to mind. Are there any that like people talk about your world that like are the dream? definitely superc conductors um magnets especially if you [00:44:28] can lower the dependence on rare earth or like transition metals that are hard to source like cobalt reducing cobalt in batteries. One of the things I think is very fundamentally important is um if we can get close to land hour limit for uh compute energy efficiency. >> Can you uh explain land hour limit? >> Yeah. So if you look at um how much energy we spend per uh like flap of compute, it has followed the Moors law type behavior where it's gotten exponentially more efficient. So we've been spending exponentially less energy per flop of compute. I mean I haven't checked recently. This was the case 10 years ago. >> Yeah, that's great. I remember when I first learned about it, looked at that and we were what like 10 20 orders of magnitude away from it or something. The last time I looked it's we're weirdly close like we're within a factor of I don't know like three or four orders of magnitude away. It's actually like >> I guess exponential rolled out over >> Yeah. Exponentials over many years are Yeah. pretty impressive. >> But like in your lifetime I mean yeah [00:45:28] since just since I discovered this like what 10 or 15 years ago. Yeah I know. >> And the one of the things we have to do is dissipate heat because like as as these uh computations are happening is producing heat which has to happen but then we have to dissipate it. So like if we can get close to the land hour limit for how efficiently we do computation that's amazing for humanity right because that means now we're doing computation is probably one of the most fundamental things we do as humanity but as efficiently as possible from energy perspective which is a real currency in the universe >> that get affected by quantum computing I'm just going to throw it out there >> um theoretically massively embarrassing to be parallel compute >> uh so it definitely gets affected by reversible computing but honestly I'm not an expert I don't understand so if you can do reversible computing now you're not you don't actually have to spend energy to do computation because you're not actually deleting any information it's reversible uh but I don't know if that works I I don't know uh quantum computing I assume has to obey these laws somehow because uh they they are still like they have to obey >> it's so like it's reversible so [00:46:30] >> one of my most memorable conversations with Ilad Gil was like he actually was a very skeptical person about quantum computing he's like even if you [clears throat] had it today there's no applications >> I'm like whoa >> I I like it breaks it breaks it breaks security. I always say like that's it. >> I agree with that like [laughter] um like for example people talk about how if you had a quantum computer today you could simulate things so much better and I asked them okay let's accept that what would you simulate like you're still simulating perfect crystal like you still have the issues that DFT has if DFT was perfect in its prediction ability. So yeah, I I do think people are kind of glossing over some things because quantum computing is so exciting. Like if we can compute in a quantum logic space instead of classical logic. It's just so exciting that people are glossing over what it would actually do when it's made. Maybe that's fine because maybe once it's made, it will do amazing things we can't even imagine. >> Good. Okay. Um I was going to move to the uh automating the lab side uh where you've closed the loop on uh your experimentation. You talked talked about robotic arms. uh talk about basically [00:47:30] just everything you've done here. My favorite quote from you was that every piece of equipment in your lab is going to have 140 IQ. Okay. What does that mean? >> In the early days as we were scaling things up, we realized that we had huge bottlenecks imposed just by operating machinery. So um for one set of machinery, we would have technicians and scientists looking for um particular morphology and trying to see okay, what actually were we making? uh see if this is consistent with our intentions and we really quickly as we scale up the lab came into these bottlenecks where it just wasn't keeping up. So we started doing some programmatic approaches to capturing the data on SEM and the data that was being surfaced from this like very simple program where it sort of takes a field of view captures zooms in captures was just not all that useful. um it was a little too dumb for what we really wanted to to get. And so at that point it was then really pertinent to build AI systems directly onto the [00:48:31] machines to start controlling these things. And now they have the full context as to what we're trying to achieve. So what was the intent of the experiment? What were we trying to synthesize? What were some other experimental evidence? And it's actually looking um in the machine and capturing that data. And this is really valuable now because you know like we've been talking about these hidden variables not being able to capture everything but if you can do more intelligent data capture at the time of that experiment your data for future AI systems and future computational predictions is that much better you'll you'll have just a richer set of data and so that's sort of what we mean by just you know every everything on the uh the lab has to be incredibly intelligent to just make the data as useful as possible. So that was that was sort of the motivation inspiration behind that. >> What's the state of the art there in terms of like uh putting intelligence on every device, right? >> Well, I think uh one interesting aspect um is what's the latency >> exactly >> of controlling that instrument [00:49:31] >> because there's cloud latency but then there's also device compute which >> that's right and then there's sort of a a time scale associated with different like physical processes and if calling out to an API or something is simply too slow. So the amount of reasoning or tokens or tool calls uh is just not matched to the latency of that actual process then that that's infeasible. >> Okay. >> It's almost like you know self-driving cars, right? So >> I I real quick I just like I'm actually surprised to hear that because the latency I'd imagine for reasoning seems small compared to a lot of these things take like hours to run, right? Or maybe you know maybe maybe I maybe your experiments are much faster or high throughput or something. I'm just surprised to hear that actually. >> So ultimately the experiments do take uh you know many hours days but um there could be particular steps where latency would matter. >> Yeah like you might want to have finer control. Uh another thing is um how expensive it is because like the the same reason some of these models can be [00:50:31] really slow in analyzing data also makes them very expensive. And then finally the human patience like if a human wants the analysis of a certain XRD pattern and they have to wait two hours before they get a good result that's very different I think than if they can get it in like 2 minutes. >> Oh okay. >> Yeah. And then the reasoning um it's you know you're not doing like a single call to that model right like you're you're doing uh potentially many many tokens uh to kind of get to these patterns >> like run simulations in the loop. Exactly. Uh deep research in the >> it's not a single tool call. It's not a single inference thread. So then the latency can blow up quite a bit. >> I I see what you mean. So how much of your latency is tool calls? And I assume tool calls is like largely like DFT or computational techn. Yeah. So how much of your latency is derived from those versus like actual reasoning? Um >> it's really process dependent. >> Okay. Yeah. >> And then the other thing I think about is also I guess building up from small things to bigger things. um where I assume that the the the general temptation or the typical development is [00:51:32] incremental where like everything is human uh operated and then you you find pro ways in which to automate it and then you sort of build out from there. >> Um I worry that sometimes that that is the way that people evolve things but that's a local minima optimum >> short. Exactly. When like actually you should get a humanoid and just put them in there. [laughter] We we I think we're of the opinion that solving humanoids would actually be slower to kind of getting to some of our goals. >> Just checking [laughter] again like a lot of this is just like you do this every day. We we like see this but we I don't know the reality of the situation. A lot of people hyping humanoids. >> Yeah. I mean I think like maybe one one process is um by having like a mix of uh humans and automation you can identify really quickly like what are some of the bottlenecks in the experimental process and you start alleviating those bottlenecks one at a time. Uh and so for example if there's some really tricky dexterity task that humans are excellent at but you'd have to spend months automating machine maybe don't spend a [00:52:32] ton of time there. and maybe like a more promising thing is something very routine that takes up a huge amount of scientists and technicians time and it's easy to automate. So I think it's there's like this pragmatism to it. But I mean ultimately what we want from the lab is a huge quantity of data, high quality data, diverse data. Um and that those are our goals and full autonomy is a non- goal. It's sort of in the service that we use automation in the service of achieving the goals on the data. >> Yeah. Um, but also I mean I think another aspect to automation is uh robots will just be uh make fewer mistakes potentially in the lab. And it's been really helpful having AI systems with a full view over all of our data because sometimes if there's like a permutation in data uh then it can identify it. So for example, one of our steps at one point had um a cyclic error because one of the machines was loaded incorrectly and the patterns were inconsistent. And so the AI is reading [00:53:32] through these things and it says well given what was run this is not expected and it's looking at this basket of data longitudinally and then it realized if I do this like cyclic permutation and reverse it everything is consistent and then we're able to go back to the physical infrastructure and understand that a mistake had been made in the loading and I think like managing data quality is so foundational to doing AI in the physical world and you know these are the types of things that we're building in and so you can improve the operating process but at that point you're like okay this is another great opportunity for automation how do we make this just so reliable so durable that we never have those types of mistakes again yeah I mean the reducing the noise floor of experimental data I think is very important most experimental data in literature have such high noise floor that it's actually usually worse than DFD accuracy because you know when different people do experiments different labs different parts of the world different times it really introduces so much fluctuation to the result. So we are hoping that by [00:54:32] standardizing these workflows, one of the biggest benefits will be that the noise floor will be lowered. >> You've been pretty public about how you use open source models and fine tune them for for stuff like this. Is it better? Is it basically I can imagine a situation where uh you have like maybe the dumber models do one task and then you optimize for that task and then like sort of like a generalist frontier model that like supervises everything. Is that a good mental model to have? Are there more stages to this that I can't think about? Um I mean we basically use a mix of uh open source models and closed source models. You know we're huge beneficiaries of this from our simulation side building up the ML stack the simulation stack. We don't need to push on on that axis. But there's a lot of areas where um the latency is too high um the cost is too high. But in some cases we can actually you know by having access to this data we can actually push beyond the frontier of what these systems can do under even some like the highest reasoning efforts. And so it's not necessarily just about cost or um prao efficiency but in some [00:55:33] cases you can push beyond it when you have access to data that no one else has. One um way to characterize this is in terms of compute efficiencies. So by having access to this data you can be that much more compute efficient compared to um you know some of the frontier models and that's been able been like a really instrumental thing in our program. Yeah I'm I'm I'm so data quality or data access as a trade-off of compute like you know I think there's some amount of exchange rate of dollars for compute for dollars for data that people I feel like the pendulum might be swinging now towards data. I mean obviously you you would agree with that but like Yeah, especially [laughter] like you know talking about like the noise floor of experimental data like we need to have high quality data. If you throw a huge amount of compute against a bunch of noise, you're not going to have a good thing emerge >> and it's harder for you because you actively try to be sparse like you actually try to get like null results and like learn from that and that's right. >> Yeah. >> Yeah. You've tal Yeah. You've talked about null results and several other venues. Like what does a null result look like for materials and how do you [00:56:34] use that effectively especially when I think your overall signal is probably quite sparse usually in terms of success? M m >> um I mean a null result could be we intended to produce some structure and then all of the evidence points to us not producing that structure. >> You ever intend to not produce a structure? >> Of course. Okay. Okay. You do the negative controls regularly. Okay. >> Like there are impur phases that will kill your property. Uh yeah can be toxic you know. >> Exactly. >> 100%. >> Okay. Um and a negative result, you know, in some cases is actually, you know, we intended to make this the following thing and then we're actually able to uh identify a new structure that hadn't previously been identified. Um so it's negative in some respect like we intended to do something else but something else emerged from from the data. >> Yeah. And also in general like when you're training a machine learning algorithm especially like at a very basic level if you just a classification algorithm >> uh if you don't have negative samples you can't really train if everything is [00:57:34] positive and this is a particularly bad problem in material science because people usually publish crystals they could synthesize but they usually don't publish if they fail to synthesize a crystal sometimes they might um and also like we never know really if a crystal is not synthesizable ever right >> it could just be a skill issue a skill issue like census method issue technology. Yeah. So um I think it really helps us when we do our own experiments and get negative results in the context of what we tried. So then we can even train a classification uh algorithm. >> I think there's also like another valid thing too of when you have this sort of string of uh negative results and then finally through process iteration you're able to get to that positive result. It's a really interesting set of I'll call it like process engineering type data. Um so it's like through iteration um how did you actually get to that that correct result because so much of uh material science has this ambiguity in actually how something was made. Um and [00:58:36] so yeah and and so there's actually even if there's a known material it can be highly non-trivial to replicate that. Some things are, you know, in like, you know, a high school textbook, but other things are really at the frontier. And, you know, we're building up that knowhow as well. And I'm building up a system that, you know, given the string of negative results, how do you actually get to that um positive case? >> Yeah. Going to the classifier result, I would almost assume that if you are doing everything in house, most of your results will actually be negative rather than positive, which is kind of ironic because the literature only gives you positive results. So if you were sort of cold starting this, it seems like the problem is actually the reverse that you have an abundance of negative results and not enough positive. >> And as you said, like if you were doing per experiment labels, it would be mostly negative. But if you were doing per campaign >> and assuming campaigns end when we succeed, then it could be a bit more balanced. >> I see. So your reasoning traces could almost be over entire campaigns. Okay, >> this is this is the key thing. This type [00:59:36] of data basically doesn't exist anywhere else. And we spend so much of our time getting the full lineage of the scientific process into the model. And so like tracking all this data like the conversations, the intuitions, what was executed in the lab, where were the computations run, where was the code written, stitching all this together is is so valuable. Um, and I I think like you know the overall goal is rather than training on the final output of science, you're training on the process of doing science. This feels very much like you what you would try to do with RSI where you are having a model train to train better models. Uh you're now having a model train to make better experiments but the difference is uh it's not going back into the core model for so it's >> unless the physics models you're developing are improving chips which then improve AI. So this [laughter] is the wider >> the bigger yeah the higher level RS >> that's the big wheel. But does that mean that is it plausible that in uh you know [01:00:36] Fable 6 or something or you know GVT8 could just have a which has been tuned on these sort of higher order um reasoning traces could actually just do this without any >> logic because it's sort of the same thinking process right >> yeah I mean I think there's um decision-m under uncertainty obviously in machine learning there's there's noise uh in running like the AI loops. Um, but we do think there's like a different set of challenges when you're actually like interfacing with the physical world. But also I think there's another piece too which is getting the compression of everything into weights is still really valuable. If inference time reasoning was sufficient, all of the frontier labs would have stopped training at like GP4 and like okay from now on out we're going to get really good at inference time improvements. Um and so we think that by you know because of the differences between you know physical sciences and m machine learning getting that compressed into our own [01:01:36] weights will lead to like different types of systems and different types of capabilities. But also we feel like even if like fable 7 gets really good at um like even better than what it is today it will still have to run experiments to get results. Um, and the reason for that is right like machine learning is really good at what's been trained on. But scientific discovery is almost by definition what you haven't been trained on. And that's why we're building these labs so that um whether open models or closed models can use these labs to tinker with the universe because we don't feel like you can make a big discovery without trying things. >> Yeah. There's there's not going to be like no one's going to like zero shot the room temperature super [laughter] think that would be pretty cool. Yeah. Yeah. I mean as as someone who has worked closely with you know wet labs before certainly I I am more skeptical about zero shotting like scientific results than some but it is something that I think some people might ask so >> yeah I mean I think it's uh it's really important to distinguish uh the results we see in math and theoretical physics from the physical world [01:02:38] >> right [laughter] these are two different things >> two very different things theoretical physics like so far it's been empirical computer science coding and math >> right yeah >> maybe theoretical physics is they don't come. Yeah. Yeah. >> Can I get a sort of mental model of the layers levels of extraction that you can go uh so for example uh in terms of automation and I'm just on this theme again where we talked about the campaign talked about individual uh essays and running tests and how there uh humans are bad at it so we should stop humans from doing it. Uh are there others? Uh so for example, one level higher than campaign could be uh a a physics theory that you're testing >> or one level lower than campaign is what like uh you [laughter] know and and like um I I just I like to think about it from that point of view and then think about it from like okay well this is an API call now like you never have to touch this again and this one is no this is still like 90% human >> you know and maybe we we can sort of draw the map of the territory that way. is is there are there other levels? [01:03:38] >> I can tell you some levels. I feel like you asked a good question. I don't have a very systematic answer, but like let's talk about some levels. So one level is the automistic structure. That's an abstraction, right? Like there's no perfect automistic structure anything we do, but it's one approximation. Another one is the continuum model. So this one is like not automistic anymore, but it's a like continuous mesh representing the material. >> Okay. Um and then in a different dimension there's the thermodynamics like assuming uh you do this experiment forever what would be the final state and then there's another layer of abstraction which is kinetics acknowledging that we're not doing this experiment forever so time matters so how quickly will a reaction happen or not even if it's lower energy or not um so thermodynamics kinetics automistic continuum what else are there other levels of traction >> well I mean I think like your your point about like okay there could be an overarching new theoretical advance that informs multiple campaigns >> maybe difference kind of like as an investor I I want to go like here's the [01:04:38] bottleneck guys and like when we get this we get everything else I don't know [laughter] >> oh I mean I mean from the beginning our hypothesis has been that one of the big bottlenecks is automated characterization because it's not that hard to mix powders to get to try stuff but if you can't characterize and analyze it and then decide what the next step should be intelligently you don't really benefit much from mixing powders randomly. Uh so we focused a lot on automated characterization, closing the loop with simulations. Uh because that's how you like try a lot of things, make an informed decision and then decide what to do the next day. >> Maybe as a follow-up question, uh what is the how do humans live in this loop? Like where at what points do you pull like drop the human in? I mean, I'm guessing you probably >> you could you could randomly drop into any part of course and at value, but like where do you decide to spend your time? >> I mean, it seems like it might be all levels like you you have art you have people in the lab who are physically moving materials and then you also have scientists who are guiding decisions about what campaign and then you have [01:05:39] humans who are sorry I think for 100% I think it's always shifting too. Yeah. So I mean I think even just using like the guidance of campaigns in early days that was fully human driven. Now increasingly it's it's a mix of AIdriven and human driven. Um and then Dar was saying early on we identified that characterization was a big bottleneck. In the early days it was human driven and the scientists were very much overwhelmed and the balance has shifted significantly towards AIdriven and this has allowed the scientists who were previously spending all of their time doing these refinements to now elevate their work to to something else. Um but I mean I think we're it's it's really sort of a function of time. There's it's it's always changing at each of these levels. >> Yeah. In terms of just general search, are there things that perform well in uh let's say the physical world that don't perform well in the other worlds or or vice versa, right? Like like so like evolutionary search is pretty popular in LMS. I don't imagine it works well here. [01:06:42] >> So especially in simulations, people have been using evolutionary search uh you know for a long time. Um like it's pretty good at structure prediction for example, finding low energy structures. Okay. >> And then uh like process engineers in semiconductor industry will do DOE design of experiments and they'll often use some kind of zero to order uh like whether it's basian optimization or evolutionary search or I I don't think they use RL but similar idea. Yeah, >> you have to have some kind of algorithm to to figure it out >> and all zero to optimization behaves the same anyway, right? >> [laughter] >> Yeah. >> Okay. I I want to go back to a a question I had earlier, which was one of scaling. And so I'm curious, what is your vision for scaling the lab? So there's different ways. I mean, coming from the biology world, um >> there's a lot of cool tricks you can do in biology. Uh you can tag things, you know, you can use DNA sequencing for all sorts of interesting readouts. And if you can map your complicated assay onto sequencing, you can now scale to millions or hundreds of millions or [01:07:43] something. Can you do tricks like this? Like ultimately every assay has its own runtime. Well, I'll I have a favorite blog post I would want to I'll quote in the show notes, but like what is the run times look like for these labs? Is it is your scaling just for material materials just fundamentally linear in terms of you know you do more you just need more resources to do more things or can you parallelize things in clever ways and combine things and are are there tricks there? Well, I mean I think like one way to think about it is um the RL environment is not quite trivially run the experiment, wait for that agent to fully roll out um you know doing the calculations going through all the instruments to the final outcome. We basically start producing data across all the different instruments and all these different campaigns. And one way you can begin to expand that for machine learning is now reinforcement learning environments can be constructed based on again like dates for you know the [01:08:44] experimental campaigns or computational campaigns at that point. Or you can also um start taking um subsets of instruments and you're like okay we're going to create reinforcement learning environments on the basis of this instrument alone or maybe it's um across these different instruments to kind of get to some reward state. Um so that's a way where this finite basket of data when viewed in different ways um can then be expanded um for training purposes. >> So it doesn't get at the actual experiment run but from an ML perspective >> maybe I can give you some analoges for the bio examples you gave. So you know one thing people have tried is combinatorial sputtering. I don't know if you ever heard of this like you have these sputtering targets and you create this compositional gradient. So when you look at the final product because there is gradient from each precursor it creates different crystals at different special locations. So in one go you maybe try hundreds thousands of crystals. Um this anam similar right to [01:09:44] your example and as far as I know this hasn't worked very well so far because turns out [laughter] like there is diffusivity so things move around. Another one that we thought about but we haven't done yet but a lot of people talk about this is super conductivity measurements are a bottleneck because unlike XRD there's no high throughput connectivity measurement and it can take like 1 hour per measurement. So people thought about taking the different candidates mixing them all into one sample and putting it in and if uh it has super conductivity you know that one of the sources did and then you can do a bit like how people were doing COVID testing to um so there are ideas like this exist uh some of them work well some of them don't. >> Yeah. So you you I think are about to announce a big fund raise. How how are you thinking about um scaling >> like we've been doing our design of our labs for a while and so I think the resources allow us to continue to scale up the labs significantly but also the compute both from like the AI side and the computational side. Uh but I think really importantly too it's what are the new types of labs we're able to build. So I think we've been proving out this [01:10:46] loop um in our initial labs here in Menllo Park, but we'll be continuing to expand to new types of labs and repeat the same process. Yeah, like the kind of lab we're trying to build I think haven't been built before at this scale or at this approach. So we have learned a lot from our own buildout so far and those lessons guide us for the next version and the next version and then we can as you said like increase the scale, the ambition, the quality of instruments. and submissions are very expensive as we get a really good understand for what we get a lot of return from we can invest into those more >> and it makes hardware engineering has become so core to it. So for example in the early days just for speed we buy some off-the-shelf instruments and as we kind of push the scale of the lab >> you make your own >> we have to make our own so we realize okay for this instrument um it's pretty fast these components are actually pretty quick but this weighing machine actually becomes the bottleneck or there can be things pertinent to data quality where well the resting position of this robotic arm actually is above the plate [01:11:47] of previously mixed things so there's a contamination risk so in our hardware design. We're going to redesign it so that the resting position lies away from th those things. Like these are the kind of subtle details that allow us to kind of push the noise floor down for our experimental campaign. So um basically take these learnings and and scale it up. >> Uh you know you ship your org chart uh you have many teams. I think the we were surprised when we looked at your jobs page and we were like okay we don't think we fully map what periodic does. maybe some organizing principles. So to kind of make sense of the job page uh so we hire for AI research and infrastructure, computational uh experimental roles and hardware engineering roles and then product roles. We want to achieve synthesis super intelligence, right? And we feel like for that we need the right chemistry expertise, the right physics expertise, simulation theory and thin films, powder. So hardware engineering like these seem like very diverse roles which they are but they're all actually [01:12:48] um coherent in what they're trying to do which is a sense of super intelligence. And this also applies to the LLM researchers, the infra even the product engineers because the product engineers kind of make sure all the research gets made into a product that the experimentals in the lab can use. Yeah. So in order to do the endto-end loop with the physical world, it really requires your ability to have agency over the physical world to build these labs to run the campaigns effectively to create AI systems against that to make them good users of the computational tools. Uh that's like the difficulty but also the opportunity of periodic. It's like this group of people has just never been brought together before. It's irreducibly a multi-disiplinary problem. >> Yeah. And one other like guiding principle for us that is really important for us is we want the people who are very good at what they do and experience to do it hands-on. You know like the modern life has gotten us to this place where especially in academia when someone is really good at research in some area we give them so much responsibility for grant writing teaching that they stop having the time [01:13:49] to do research in that area hands-on but if you look back at like Bell Labs IBM institutions that made really good progress it was really experienced people doing hands-on work like Bardin was in the lab every day even though he's a theorist Alex Mueller was in the lab doing experiments even though he was the lab So we try to do that here too. So we have some of the world's leaders in different fields but they're doing hands-on work. >> Do you want to brag or just call them out like you know >> Oh I would love to. Yeah. So um you know when I was doing my PhD my favorite computational material scientists who's kind of from my age group is Muratan Aikol. He did his PhD with one of our adviserss Chris Wolverton. And he is kind of like our computational material science expert. He's incredible. uh he's sometimes you know a real good experimentalist even though he does simulations like he understands systems really well. Uh our lab lead is Joe Chaoski who is a professor at MIT but he's on leave to work with us full-time and Joe is this incredible physicist understands superc conductivity really [01:14:49] well but he also was a professor in Japan for a bit so learned synthesis and chemistry really well especially for a physicist. We have Daniel Chica who was a graduate student in mercury canis group which is maybe the best solstate chemistry group in the world and Daniel was one of his like synthesis experts. So he's like a magician with his fingers. Um I mean there's so many examples like this. >> No I know I mean I think like um Dimma Bahenda so he's been leading a lot of our AI and LLM efforts. He was the inventor of neural network attention. >> Um >> Oh that bad attention. >> Exactly. Yeah. >> Yeah. Yeah. very hands-on in sort of getting into like you know the traces, the training distribution, how to connect it to the lab, >> the chemistry, the I mean he'll go deep. Yeah, >> he was very deep. Um he actually has like solid state synthesis books like at his lab in Montreal Ray Nano. So he used to tech lead some of the um operator work at OpenAI. Uh he's in the lab. So when we were doing some lab tours, we were initially were like, "Oh, this new [01:15:50] uh scientist, this new technician really looks like Ry." And no, it was literally this was Rey like actually in the lab um with the scientists automating the machines and it's like this is sort of the the DNA and the type of people we need to kind of pull this off. >> Uh so uh we also saw that you have for deployed engineering uh roles. Um we happen to be spinning up a forward deployed engineering podcast because there's so many engineers that would do that role. They may not necessarily know that there even is a role for them at periodic. What is it? Um and how do they work with customers? So we've been building our own products, our own tools, our own AI and computational for ourselves. So we've been customer zero. Now what we're doing is we're taking these same tools to different industries and we're we have a huge focus right now in the semiconductor industry. >> Uh however, in order to do this like these are very uh um private companies. We need to be able to operate within some of the most secure environments. And so our periodic for deployed engineers and researchers will actually [01:16:51] be onsite working and actually integrating our AI systems or computational systems to help our partners get to their goal states more quickly. Uh so basically the tools the knowhow that we've built in doing our own materials discovery materials engineering we're now bringing to industry and we think this is the the most effective way to get partners to their end states and to their goals rather than just like throwing some technology over the wall and telling them to to figure it out. Then these four deployed engineers will also do the inference locally but then also can uh train on the data. So again once we take our system that understands these different areas and we deploy we can make it expert on the customers or the partners' data so that they can own their own intelligence and have systems that understand this much more effectively than uh you know than just like hitting some API or some untrained model. And so this is a much expanded um for deployment engineering role than typical because this really requires [01:17:52] some very deep uh machine learning expertise. uh you have to be incredibly precise from like a data training perspective, infrastructure perspective, but also the physics perspective. So we're working with you know highly technical industries. They have to understand you know different areas of like chemistry, material science, um some of the devices and so that that those are the roles we're building out right now. >> Part of it is willingness to spend extended periods on site in Taiwan. [laughter] Like okay well I know that kind of customer. Uh it's interesting that like you're monetizing that that that is the primary way that you're monetizing now, but it might change in the future. But like I mean you know that's that's like a a model that I think people don't really get about AP research lab like because then when you compare yourself to like a bell labs that's what people are thinking right like which is I think like a really great analog would be software engineering >> right >> so in the early days of software engineering like we had like you know GitHub copilot then we had early versions of chat GPT people were using these things as co-pilots to help them get to their solutions and as the [01:18:54] automation improved now we have things like codeex and very few of our engineers are writing code the way they used And we think a very similar thing could play out for periodic as well where you have systems to accelerate the researchers, the material scientists, materials engineers, the process engineers. But as the autonomy, intelligence, and capability grows, you can begin to um price outcomes. So help me get to this kind of goal state. Um and I think that's a a really interesting area for us. >> Yeah. I mean we kind of feel like our best contribution to solid state physics and science could be if we made these tools and topics profitable. Uh similar to how Chachi PT made CS and LLM majors way more popular in colleges before and after. We'd love to show that solid state physics material science research can make a big impact commercially and then that will attract more attention. Young people will want to study physics uh which will be uh a dream. Um, and you know, Bell Labs did make a huge [01:19:54] commercial impact, right? So, they failed to commercialize some of their incredible advances. They of course did some research that just couldn't be commercialized like the cosmic uh background, but but they did benefit a lot from like the vacuum tube connecting uh east coast to west coast by phone lines. >> Yeah. And I think it's it's really also interesting to think that technology and capital are incredibly intertwined. So if you look at what was the progress on chat bots for many years like imagine recounting okay what was the progress from chat bots from I'll say 2010 to 2015 we'd be kind of at a loss to say over that 5year period how how much did they improve whereas if you look at the period from uh 2021 to 2026 you know it's it's night and day and what happened was chatbt and these other systems were able to achieve a product market fit and it changed the capital landscape Cape entirely. This changes the landscape for hiring um compute data etc. >> The virtual cycle success breed success. [01:20:54] >> Exactly. So technology uh it's incredibly coupled to the capital and those resources and we want to achieve the same thing in the physical world. >> Yeah. So one thing I've been seeing is there there's this move from science being funded from you know government grants and so on to you know VCs and uh private funding. Do you all plan on making anything you're doing uh open source like either releasing data sets or actual models? Is this something like in the future even you know there are many models which may be not your your state-of-the-art but which could still be you know useful releases. >> Absolutely. I mean so uh we have a couple of things. So we have been contributing the open source to kind of like pimean the materials project uh codebase uh custodian their DFT runner uh torch sim was actually created by one of the researchers here Abijit Gangan and he maintains it still Jack MD uh Abijit maintains it so we do a lot of open source contributions >> Megatron yeah so we've been very active [01:21:54] contributors back into open source >> and and we also have an academic grant program where We give academic gift grants to uh academic groups in universities that we feel like are really advancing this direction towards synthesis super intelligence. Uh so that's also been great to see. I think the first paper from this funding is about to come out. So it will be exciting to just get more papers come out like this. Yeah. >> Okay. Um let's steal man this say that you've successfully have 100% lab automation which does everything characterizes everything correctly. You have all of the fun agents which can do everything. I I still am somewhat unclear about how you actually get to super connectivity. There's a lot of hard problems to solve. What is the path there when like there is no sort of like theory for model for most of the like strongly correlated systems or high temperature systems? >> Yeah. So I mean our view is that it's not hard to think of chemical spaces that would host superc conductivity. What's really hard is to synthesize them. That's why we're really like [01:22:55] emphasizing synthesis super intelligence. So maybe one example to give historically is Alex Mueller when he thought that transition metal oxides might be a good path to super conductivity. He was actually trying nicolates the nickel oxygen and some cations and that didn't work and then he tried coup which is c copper oxygen and some cations and that worked and then he got a Nobel prize but then we know that later on turns out Nicholas was a good idea I mean nickel and copper next to each other >> two two layers there only so many 3D transition metals and then Harold Huang one of our um Stanford professors here who's incredible he realized he can make nicin film form and show the superc condolivity um So you know if we have a really good synthesis super intelligence in the lab and we can really scale up these things I think we won't run out of ideas or the LLM won't run out of ideas about what directions to try uh like 3D transition metals oxygen like there are very related ideas here like many people have been trying like cobaltates which is like instead of copper or nickel you [01:23:55] try cobalt um so I think that's super exciting like maybe one historical example to also study is the Japanese group that discovered magnesium ium diporide as you know that's the highest temperature ambient pressure conventional superconductor and they discovered it by just trying a bunch of materials I think they tried 30,000 different things 30 of them seemed to host interesting superc conductivity and MGB2 was one of them but magnesium doride sat on people's shelves as a precursor for decades before then and BCS theory had been invented back in 1957 so the way people found these materials wasn't from directly from theory it wasn't because they couldn't synthesize until then. It was just like they tried a bunch of things. So now imagine if we have this perfect automation that you're describing which sounds like a dream. Um and we tried the 30,000 the Takito group tried in over his career in a month. We just really increase the surface area for luck. >> Uh no but yeah it's very inspiring what you've done. Congrats on all your success. I I feel like you're creating [01:24:56] Yeah. the modern like playground. I imagine recruiting must be like super easy for you. So I'm just like a little bit jealous. uh you've worked hard for it to get here. So, yeah. >> Yeah. Well, thank you so much. Yeah, it was great chatting with you both. >> Super fun. [music] >> [music]