Mini Brains, Beating Heart Organoids, and the AI That Predicts Them — Fabian Theis, Helmholtz Munich

[00:00:00] We can really now start guiding the experimentation. So, where before you we built these human cell atlases, we had sort of this mapping aspect. And we talked about the world model prediction problem. From mapping we went to prediction. But now, >> [music] >> we can go to actually steering the next experiment. But quite frankly, very often the papers ended with some model, right? Maybe some sort of concept figure, maybe had a little bit validation. But then typically you start the next project, right? And of course you learn a bit, but there's not like not like a continual learning type of thing. And new thing which we call lab compass, which is an inverse generative model that actually allows you once you have a forward world model whatever to tell [music] you what you need to do in order to reach a certain state. Obviously there the ultimate vision being that we want to RL everything, we want to do RL also in in this loop. It's a bit out there, but you know, I think we're we're slowly getting there. >> Welcome to the Latent Space AI for Science podcast. I'm RJ Honicky, CTO of Miro Omics. I'm joined by my co-host Brandon Anderson, who develops RNA therapeutics at Atomic AI. I can't emphasize enough how much Fabian Theis and his labs and companies have contributed to the development of [00:01:00] systems biology and various iterations of the virtual cell concept. Papers and models come out of his lab faster than I can read them, and the lab is consistently moving the ball forward on our understanding of how to observe and model the behaviors and the interactions of cells. Just as importantly, his work often focuses on creating the tools that researchers and scientists actually use in their day-to-day work. Most computational biologists have probably used Scanpy and AnnData at some point. And this work has led to the creation of SCverse, which is an amazing collection of open tools that's attracted many many contributors. Of course, Fabian is one of those academics who has way more honors than you can possibly list, but suffice it to say that he leads a bunch of the AI and computational biology efforts at Helmholtz Munich and related institutions, and participates also in many other industry efforts. Fabian, it's really great to meet you. Thanks a lot for joining us. [00:02:00] >> Thanks for having me. I'm really looking forward to the discussion, man. >> Yeah, I've been tracking your work for a long time, and so it's a great honor to finally get you on the show and talk to you. Let's kick things off with some of the hype happening right now. There's a ton of interest right now in structural biology, Anthropic and OpenAI biopharma initiatives focus heavily on designing molecules right now. On the other hand, virtual cell and systems biology approaches really haven't gotten as much attention recently. Maybe Zera and Noetik are good counterexamples there, but Biohub just announced a $1.8 billion collaboration with the DOE, NIH, Meta, and Google to collect and open-source multiscale biological data that can be used to develop a new generation of virtual cells. So, my question, do you think that we're about to see renewed interest from the pharma industry in virtual cells based on the activity that we're seeing in academia right now? >> I I I should disclose before I sort of go to to the answer that obviously my my [00:03:00] opinion is quite academic in the sense that whenever I talk to to my my friends and colleagues that really sort of day-by-day sit in in the startups, they have like like what I sort of have as an overview and sort of a bit of intuition what the different research labs do, they have the same thing for the startups. And they know this is sort of the the hot big thing that, you know, this and this investments are happening. I don't really have that on on that scale. But having said that, for example, you you you you may you mentioned a Biohub's initiative, which by the way is also heavily coupled to human cell atlas, which is sort of this international really worldwide consortium that sort of pushes this next uh level of data set generation. I do think that there's a whole bunch of reasons why we going to see that sort of scale up from all these models working well in terms of being generative, but also sort of lap in the loop on the molecule level now being exported and sort of really working one level up on on on the cell level. And I I think there's a bunch of reasons for that. And the first one I think it maybe the most important one, which you really hit here, is the data generation. I think we had that concept of a virtual cell [00:04:02] for a while, but my my lab we started this perturbation prediction modeling and and using machine learning in 2019 with a paper we called SEGAN at the time, which was like it was a non-linear model, but essentially sort of in the latent space did some linear transformation. And it was actually quite okay, and even nowadays many of these sort of pseudo-linear type of things are still doing a good job, but the reason for that being that I think the data sets and the questions that we asked on top were just sort of still on the simplistic side. That's why also sort of it didn't yet take up. So whenever people talk about virtual cells, maybe also your question, I think what you actually ask is how does Can you predict how a cell would behave under all kind of perturbations? And that's sort of I think a quite deep question. If you really think about it, it's maybe even a bit of a holy grail in computational biology systems biology. Sort of this this input-output system that is a cell with memory and like with all these specifications and with all the different sort of levels of molecules and so on, not just RNA. I I think being able to map this out, what does even [00:05:02] understanding mean? And I've been this sort of a bit of a meta problem, but I I think you understand or at least you model a system very well when you sort of predict its output behavior given different inputs. So I think it's a very good question, and this is why often people nowadays equate a virtual cell with um perturbation prediction. I think it's a bit too simplistic, but I think these perturbation prediction type of models and that's sort of being a maybe what you mean by virtual cell is taking off a lot in the startup landscape. I think it's a bit exploding there because I think a lot about US Europe, but also in in the Chinese ecosystem there's a lot of of these these additional efforts now happening. And I I hope that these are not just sort of now a kindling of of a flame because everyone sort of adds AI to their things, but because pharma is really desperately uh needing to be cell type context-specific in their screens, which at the moment they can't do yet. on that screen. >> Maybe you could expound a little bit on what does this So perturbations right now that like we talked to Zara recently, they're doing these perturbation screens. What does that unlock? And then what [00:06:04] What do you think the next generation of data is going to unlock in addition to that? >> I mean for for me, if I may, you stop me when I'm sort of getting to too much into these these these old stories. Like, you know, I'm sort of I've been trained machine learning scientist for longest time doing all kinds of things, but you know, I started with single cell early on qPCR. Maybe initially, if I'm really honest, not so much because it was like the biology that I was interested in, but because we had high sample numbers, purely for that, right? Because even earlier on it was sort of clear that the number of qPCRs at scale, but then of course transcriptomes was just getting really big. And you know, when I started my career earlier on, we had like a few bug RNA, now we sort of had thousands of them or whatever, right? So, this is like early times. But now, you know, we we're going to to regime, and that was sort of one of these eye-opening moments when when Pac Bio had like I think they called it the giga lab or something like that. And you know, there's sort of similar ones from from competing companies now, but when they had like in a single experiment 10 million transcriptomes measured, right? I felt that was like like next-level [00:07:04] stuff, and you know, this step from just getting more PBMCs as a control tool, like let's actually do something at scale to it sort of having here the cell space, but then having here the perturbation space, which typically was very scarce. It was like control versus disease or so. So, this sort of number of points what you had here in terms of sampling on the perturbation space was typically much much more scarce than the number of observations in terms of cells that you had. But now this is exploding. Now you sort of have, I don't know, 100 different cell lines where you do sort of full perturbsic like full CRISPR, or you have like batteries of 1,000 drugs or so added, and suddenly this space gets really rich, right? And then you can sort of start doing all these fancy perturbation prediction type of things. And this is currently happening. It's crazy what scale it's happening. We are ourselves involved in a whole bunch of things where we actually go beyond cell lines and do organoid perturbations, and maybe if you have time we can can talk a bit more about that because I'm quite convinced that we need to go to multi-cellular systems to have actually really interesting readouts, but I think everyone is scaling these these these levels up, and at some point, you know, [00:08:04] we're going to go maybe even to human genetics as a perturbation. >> Can you just like explain what is a perturbation? What do you mean by that? Just for the people who are not familiar. >> Yeah, yeah. You actually Would you mind maybe backing up even one step more than that? Like what do we even mean by like measuring transcriptomics? And you were talking about like PCR-based and then also perturb-seq. >> When I give sort of public talks I typically say, you know, human genome project, we sort of have the genome for for for the body, but it's pretty much the same everywhere, right? So like why do we have all these different cell types, right? What's sort of happening specifically? And you know, so first level what one of the first levels is that, you know, per cell different stuff is being read off of the genome, right? And these messenger RNAs, which then sort of encode the various downstream proteins, well, these are sort of differently expressed, as one says, in in in different cells. And to say in a human maybe there's something like 20,000 different types of these, and with splicing many many more, but that's sort of the the rough level. So what you de facto have is per cell a 20,000 dimensional vector [00:09:04] that has a whole bunch of zeros and stuff is not there, but then a whole bunch of sort of numbers, how many of these counts of these messenger RNAs you have. Because these things are called transcripts, right? This is called then the transcriptome. Yeah, and and and de facto then you have this sort of 20,000 dimensional space that you very often encode in a sort of nicer latent space that sort of has like an a sub-manifold, like sort of this this subspace where this actual cell cloud lives, right? And if the cell cloud then sort of like this sort of 20,000 dimensional vectors, each point then being one cell, right? If they sort of cluster up, then we call these things cell types or something like that. And if they sort of go into a sort of sausage or whatever, right? We call them into trajectories, like a cell differentiation, like the fertilized egg cell then making all the cell types in the body or something like that. >> So like DNA is sort of like a recipe book, or maybe it's a bunch of genes are a bunch of ingredients. And then as you each cell is its own sort of meal or dish or something, which is a combination of different ingredients. And now the trans- transcriptomics is sort of figuring out [00:10:04] uh taking different meals and then sort of decomposing them like, "Oh, what ingredients went into make this meal?" So as you is that a dumb analogy? >> I have a bunch bunch of of foodies as kids. I will use this metaphor a bit more often. I like it. >> Thanks, Brandon. >> I love it. And And then And then perturbations is when you somehow perturb the the genome or the transcriptome so that it has different transcripts in it. And then typically that's done with a CRISPR knockout where you remove a gene from from the genome and that causes it to not express that genome. You see and then from there then you see what are the phenotypic or the actual changes in the physical biology that occur because of a perturbation. Is that more or less >> Yeah. >> less right? >> Maybe I can take take a step back there because I think I'll tell you you're fully right but but this is sort of one of the hardcore type of perturbations, right? Like fully knocking out a gene. Soon enough in evolution happens a bit but mostly these things are sort of a tad more subtle. But of course in order to understand the system, it's a good idea to take out between pieces. I think [00:11:04] what happens sort of on on a normal development or normal sort of us being in homeostasis is of course that these these cells sort of talk to each other all the time, you know, they're sort of living these complex multicellular states, right? So, there's all kind of communications happening between cells. So, they always have stuff coming from the outside and then sort of doing something to them. So, for me a perturbation space in fact is just an input space to stuff that happens to a cell just maybe because it just sits to another one and there's like some signaling happening, right? Or some gradient, some hormones coming in, whatever, right? And all this stuff of course you can also sort of tune up or tune down. Maybe even something as basic as when you go these in vitro systems where pharma does drug screens, when you sort of put them in a dish, you know, you can do different what's called media conditions and they also have so much impact, right? And it's like a bit of for me still like a a bio-driven, maybe even like an alchemy, what type of cocktails people come up with to then sort of maybe take, you know, skin cells and turn them into into more or less embryo-like state cells, right? Where you can then sort of make new cells to [00:12:05] heal different stuff in the body or these what they call the Yamanaka factors for example, you know, these were um Nobel laureate type of stuff, right? So, I think having a space of perturbations essentially of things that you can do to a cell. This can be a very rich thing. It could can go on as much as actually changes in in in the gene, which would be like a full knockout or something more subtle, right? Which is sort of human genetics. And I think all of that we need to put in a big space of stuff that we can do that has an influence of cell state. And And like a virtual cell model should take all of it, right? and also combinations thereof, of course. And maybe you know disease happening, time, whatever, right? That then impacts that. >> That's a great connection to the the organoids idea. So, can you talk a little bit about Okay, so an organoid is is what? And And how does that connect to this concept of perturbation space? And what do you do to it in addition to it just this context? >> Dude, when I when I first heard about learned about this concept, I should say, you know, I'm I'm an overtrained biologist, so I typically learn about things when I read a bit or when I talk [00:13:05] to colleagues and and friends. And when I first saw that people are able to take a cell actually cells out of a body of a cancer cell or whatever, right? And let them then grow into small multicellular systems that actually cells start connecting to things. I was super super impressed. So, you know, people can now, for example, take some cells, put them a dish, and then grow little mini brains, essentially set of neurons that essentially really start connecting, talking to each other. And people have shown that they can get those sort of mini brains in a dish to compute. Similarly, people can take heart cells, grow them into little heart organoids. We have a bunch of local collaborations with fantastic partners that actually start beating. Like you have beating little heart cell clumps in a dish, right? And then of course, once you have these and you're the same thing is I spoke about brain, there's even simpler systems like the colon where this works very well. There's a whole bunch of cancer systems. But, you know, also for let's say if you want to study diseases with hard to acquire material. Let's say you want to look into the genetics of some type of of neurodegenerative [00:14:05] diseases or so. Like typically it's very hard to get access to tissue, right? So, you take stem cells from the skin You take cells from the skin, differentiate and make turn them into these mini little brains, and then you can potentially study some some things there. So, I think this concept of replicating little bits of organs, it's what they call organoids, in a dish is very fascinating. There's all kinds of limitations, obviously. Often immune cell is immune system is missing. There's like no uh maybe feeding them or something like that. So, there's all kind of problems, right? But it's much much more than just a cell line which is like one cell type. >> Moving to the modeling side, how does how do you model that? >> So, that's I think at the research frontier. So, I don't think people have agreed on what the optimum way to model is, particularly also because there's not as much data as one wants. Jay, right? You're you're involved, if I understand correctly, in a whole bunch of spatial omic slash um also uh um sort of H&E sort of pathology type of um effort. So, so like pathology, there's like so much samples available, right? And now increasingly we sort of do a bit more spatial omics. So, I think how we even [00:15:05] represent spatially connected data sets. And then you also have to break it down on an organoid level and so on. People are not as clear as as it is with dissociated cells. So, again, taking taking this a bit simpler, I think if we look at the space of dissociated cells and we look at these groups, we have this beautiful metaphor of what we humans call a cell type, which you know, nature doesn't care about. But this is just us putting categories on top of stuff. But it helps us understand this complexity. I don't think we have a similar metaphor for local niches of cells. Maybe a little bit, but also in H&E, right? Whenever I I I talk and see what pathologist do, it's often very qualitative and it's maybe not at that level yet. So, I think we need to develop a bit of a categorization language for us. Maybe computationally, maybe we don't need this so much, but computationally we also need to to sort of think a bit how we deal with this. At the moment we essentially deal with this as distributions of of of cells. So, this is sort of the solution at the moment, but I think it can be more much more subtle, like graphs or something like that. >> So, on the one hand you have the bitter [00:16:06] lesson and you know, sort of you you remove inductive bias and you end up being able to have a larger data set that's applicable to your problem and then that helps you to get more accurate models because your the distribution is wider and whatever. Um so that's like the one and then but then there's this like countervailing tendency which is like maybe you could call it the anti-bitter lesson or like the other the flip side of the bitter lesson where when you're in data poor regimes for one thing then you you have to induce introduce a lot of inductive bias to get enough statistical power to make good inferences. But there's also a second side of that which I think John Platt in our discussion with him not too long ago brought up was that you have these these like predictive and then descriptive models. And what you really want as a scientist is to get to a simplification a descriptive model where you can say, "Oh well, I've tested all the apples and the gravity works in such a way on [00:17:06] apples." And then but I'm modeling this as a universal concept with simply so that we know that it works on plants as well. Right? And And so that I want to understand from you, what are the simplifications or you know, sort of maybe simplifications, maybe insights, maybe actual mathematical or otherwise scientific models that are becoming apparent that are going to help to reduce all the complexity so that we're able to actually make meaningful gain meaningful understanding from these really large data sets. >> Yeah, that's that's a that that's a very important very I think kind of deep point which I think sort of breaks the field into maybe sort of a bit two different groups of people with answers, right? So how much do we need to put inductive biases, maybe priors into a model to maybe make it better explainable, right? [00:18:06] But potentially also more robust in going out of distribution. Or how much are we going to just rely on bigger data to sort of have the model be more expressive and so on. And I think And there I I I I need to be very clear before we may go into the full philosophy. I think literature has really clearly shown over um I think now decades at least for of us all very many data sets that if we build an a priori inductive biases too strongly, that we might lose predictive power, right? So, it comes at a cost. So, and if you know, let's say you're a physician and you just want to or maybe even if you're a patient and you you you maybe you might have cancer or not, maybe you want to have the best model as a predictor. Maybe she getting treated, you want to have a model that is maybe a bit less good, but you understand it a bit more. But for the prediction, at least I think you you want to have the best. And this is I think I think not not entirely clear. And I think in our field with this perturbation space scale, the cell scale I think is big, but the perturbation space scale, like the stuff that we do to it not being as big, I don't think we're yet in the super data rich regime. So, I think in our [00:19:08] case these hybrid models might be still the thing to go for, but more and more of course, and this is I think this I don't know if you can call it sad, but the honest learning from from big scale machine learning is that at some point data beats human-made additional concepts in these models. And I know at least one simple example I often put is AlphaFold, where you know, the first model sort of made this extraordinary effort to to make, you know, the molecules rotation invariant, they sort of built it into the learning process, and only in later versions, you know, sort of rotation symmetry was automatically found by the model just with more data and was a bit more flexible because whatever physical constraint you put in, it's not always as rigid, and then you sort of you gain something if you let the model learn from this. So, I think in the future the bigger part at least of the sort of more used models will be those more flexible, very much database ones, where you then a posteriori put the interpretation on top. >> This also brings up the the question for me. So, you actually I think recently wrote a or co-authored a paper on the statistical power that the current [00:20:09] generation of virtual cells have in terms of the data that the the lift that you get from additional data in those current generation of virtual cell models. And I think you had said basically that the objective function or the training methodology is potentially the problem there and not necessarily that the information is not in the data. So, like what can you explain that a little bit and why you think that maybe there's plenty of untapped information in the information and then like what where does that get us and what what do we need to do to get further? >> Yeah, I I I think it's fair to say that you know, early on we had some sort of very small sort of more limited prediction models of learning let's say latent spaces first and then also sort of perturbations on top. And then, you know, when like this really large-scale foundation model representations came out everyone was sort of buying into the hype and maybe potentially sort of bit bit over buying then, you know, obviously after this you know, comes what's it called the the sobering up value or something like that. And then [00:21:09] there were a whole bunch of papers coming up where they were basically saying, you know, hey, maybe even more or less linear model were doing good job. And actually now I see gen paper early on we had already some of these benchmarks showing that it's not so not so super easy and simple task is just to improve on that. Um I think I think there's a truth to it that the particular in the current data state for some things let's say you want to predict the effect of a crisper in a cell and you might not need the most fancy model. Whereas if you ask a bit or if you want to predict a cell type or something like that, you know, for that mostly logistic regression is fine. But if you want to predict something a bit more difficult, for example, in our niche former paper which is a foundation model for spatial omics, we showed we had a new task which we introduced where you say you have a you have the gene expression you have sort of the state in latent space of a single cell, can you predict the gene expression state of its neighboring cells? Because you know, they're always in communication or maybe the other way around. So, like they're always in communication, so because you know, they talk to each other you sort of see a remnant of that sort of cell state in this one. And then actually this is harder to do and if you sort of [00:22:09] have these harder tasks you need the stronger more linear things and you particularly need to look for more subtle data. So, your question, can you sort of find out more sort of information which type of things I think very much depends on what you sort of ask at the end of it. Of course, like in the self-supervised learning phase, I think we're definitely not there yet with the best way how we represent how how we learn these things. There's a lot of copying of what happens in NLP type of LLMs, right? And like masked autoencoders are always the typical way to go, but the type of losses, the type of representations, I think people are still debating. Maybe because you know, this is sort of often kind of tabular data, so you can just like tabular foundation models and all sorts of being tried out. There's a lot of JEPA type of approaches now because it's like we have one thing where we use contrastive learning. So it's just people playing around with the losses, but also with the representations. And I think nowadays reviewers, and at least I've been advocating this for a long time, are really much more closer looking that you really properly show scaling laws. This was what has missed from the early papers of the foundation models, and so [00:23:09] people haven't really shown that you need this complexity to actually sort of really scale up. But if you go really if you can really demonstrate at scale like across examples and and samples that you have this sort of scaling behavior, then I think you have this buying into these models. And for these, of course, there's not so much at least inductive biases used. And so I do think we're going to going to hit this regime actually quite soon, and we do see this also in a few papers. So for example, in our contrastive learning paper, we call it SE concept, we do show already quite quite quite strong scaling behavior. In the paper, we had something for going because we compared to Gene Former going to maybe 30 million cells, but on on Hugging Face now, we have a 350 million cell model with I think going up to maybe 100 million parameters, so really going to the big scale for us. And we do see nice scaling behavior already starting to emerge. >> Just to be clear, then at 100 million parameters and three you said I think 300 million cells or something like that, that you you get a lift in what kinds of applications? In these these sort of the niche former kinds of questions where you're saying, "What is my environment?" based on the [00:24:09] my transcriptome? >> Yeah, I mean you know, the the ultimate thing of course you want to do is you want to be doing this perturbation prediction better cuz that's sort of what people now at least understand by virtual cell, right? And for this, I would not claim that pure representation learning is the right way to go. Sorry for making the subtlety again, but I think typically you learn the state space with these foundation models, but then sort of the way how they behave under an action under perturbation, this is where you need this more world model like type of approaches. World model being the very hip term, I'm sure you had this already at some point in your podcast. So, it's been on all of them machine learning conferences in last year, but de facto a perturbation prediction model like a virtual cell is kind of a world model in terms of transcriptome or other type of omic space, right? So, it's it's not just a model that is sort of f of x, but f of x {comma} a, right? So, the state x plus the action, right? And then sort of predicting the next state x prime or something like that. And I think for this you might need other combinations. So, what we typically do we learn the representations with for example as a concept, but then we put a generative model for the push forward along the action. So, [00:25:10] for this we use something like flow matching or something like that. And in our self flow paper, this is not just uh got accepted for nature methods. We actually really show that these approaches are much better generating the effect of of of a perturbation, but they're very much dependent on a good latent space. And you're on this SE concept latent space. In this case, for these 350 million cells, we actually went into multi-species. So, you have now like 16 different species where you can potentially look for similarities. Suddenly, here you can maybe ask some evolutionary questions that you couldn't before. Not that we can do this so well, it's not our core expertise, but you know, sort of you can start doing these fancy um sort of cross species cross tissue type of latent spaces. >> I'm kind of curious at the history of some of these different models. I think you just gave us a sort of a rapid version of them. The one of the things that like I think was like somewhat famous in the community was this thing about linear models being kind of a a big point. Can you explain specifically what was the like key point here? Cuz I think this is like one of those things in classic machine learning, make sure you always understand your baselines [00:26:10] well, you know, otherwise you you spend a lot of time spinning your wheels and fool yourself. I think there was a specifically a famous open competition on with the through the Arkin's the two that was a sort of a class of like cold water on everyone's uh you know, faces. Can you maybe give a little bit of history about that and like why we thought this was exciting and then where why this had us pivoting a bit. Yeah. >> It sounds such an like such an obvious learning, right? Do your baselines properly. But in a field that is rushing towards the gold pot of the virtual cell, people sometimes maybe were not as careful with the baselines and there's a whole bunch of subtleties with respect to holdout and so on that, you know, wasn't always fully observed and why some of these things happen. So, one of the first observations there was in a thing as basic as cell type prediction, okay? So, nothing super fancy, okay? Basically, you you give me a new transcriptome, you tell me from which cell type it is, right? Maybe even from which organ or something like that. And it turns out that even nowadays, I think some close to state-of-the-art models are [00:27:12] really just logistic regression, right? So, for such a simple categorization, you know, this is actually not not so bad. There's one thing called cell type is for example, which is doing a good job and I think we had a paper maybe 5 or 6 years ago where we had the title very plain, "Deep learning is not needed for cell type prediction." This is true if it's within an organ. If you sort of go cross organ, suddenly gets a bit more subtle because, you know, sort of cell types reoccur in different organs and immune cells are a little subtle change depending which tissue they are. Suddenly, you know, there you need to do a bit more and we like for us, we published on a paper called SC tab, like a tabular foundation model, predictive model type of thing. We actually showed that this nonlinearity was really adding beyond the the baseline. So, that was sort of for that class. For the perturbation models, which then you were asking Brennan similarly like early on, many analyses particular perturbation could run with classical linear models in Suda pack. We had then our our SCGen that was like something around 2019 where we started introducing linear shifts but in the latent space, so sort of non-linear encoding and we always compared with subtle sort of simpler linear shifts. But then, the field getting complex and [00:28:14] you know, sort of data sets popping up, people started asking more and more perturbation type of questions. There have been a whole bunch of these early on competition for like a regular one run for example by MIT. We've been running a regular NeurIPS competition on uh prediction problems in single cell data, sometimes for integration data, also for perturbation prediction. Um that was very well attended. The cherry in the on on on on the cake was most definitely that that that arc competition which called the thing not just perturbation prediction challenge, but virtual cell challenge. To sound a bit cooler, right? We picked up very well. And but of with metrics that are robust were maybe but maybe not not that easy. So far actually our solution in in this this this consortium where a lot of labs contributed called open problems was actually that people need to submit their own metrics and additions so in order to really maybe subtle things that you might not have thought before. Dark one is also fixed a bunch of metrics that weren't even so so single cell specific and they could be just just exploited. So actually I think the winning model that was then in the end actually sort of crowned as the actual winning model, but essentially is a flow [00:29:15] matching best thing. So based on on this idea that you know people fought with this flow with this cell flow type of idea. I think shows you that some non-linear things are in this case needed. So that the actual real winners or if you sort of remove the the exploitation was actually something that is this is taking some some non-linearities and I think nowadays we do see that just linear or additive type of models are not cutting the the the cake for more complex things, but for I think even nowadays if you go in a simple if you cell lines and you have sort of gene knockouts then particularly when the effects are a bit subtle all these sort of more complex models are sometimes a bit overfitting and that's why a linear baseline is always very good. So I always tell my students, guys I know we we have guys I mean guys and girls. We always need to do I know you want to do the most fancy deep learning thing, but please please you know have the baseline and like say for drugs you know a baseline as simple as just predicting getting better at the mean. You know, like that's the like the hottest base. That's not even a linear model like better than the mean is sometimes not as hard if there are so much noise in the data and this stuff is super noisy. [00:30:15] >> I just wanted to add one point that Bo Wang and some of his collaborators actually just had a nature paper on on how the metric was actually the problem in some of the those linear baselines beating some of the models as well. So it's clearly it's not only metric hacking, but also just that asking the wrong question leads to the wrong answer. >> Our choice of the metric and I I think that they could you can discuss it. At some point we're going to have agents sort of trying out all kind of combinations of whatever algorithmic thing you can come up with. In fact, we actually do have some collaborations with some of of those sort of auto ML type of labs try this out. So, maybe what is left sounds a bit too harsh, right? But like what maybe our biggest ideas go into is actually this metric formulation, RJ, right? >> So, one thing I've been really interested in is the different models of funding in research academia. I think Germany has a different structure fundamentally from the US. How is your institution set up and like how does this give like distinct opportunities versus like a US-based research model? [00:31:15] >> I I very much agree that these funding models very much matter for the flexibility, but also the set of teams that you can set up and like the long-term vision that you can have. So, um Helmholtz, this is not just Germany, but I would say maybe Europe's largest what's called research organization in terms of funding. It has different set of areas that it focuses on. Health being one of the big ones, but then it has sort of material, information sciences, physics, and so on. And it's essentially getting fed federal money to put into a deep research question. I've been building up what's called the computational health center at Helmholtz Munich, which I think we has now something like maybe 45 group leaders and professors and maybe 500 to 600 people now. So, we are I think the largest academic place for health AI type of research in Europe where we have people really working on all scales. Like, you know, I'm sort of focusing on the cellular scale, but you know, there's people doing DNA foundation models. There's people sort of doing interesting things on protein design. There's people doing like, you know, patient cohort type of [00:32:16] representation, genetics, and so on. And now you sort of can start putting them together, and one of our big visions is sort of to to intersect and sort of go virtual human, and so on. But having said that, the key thing is in contrast to universities that even though we all have affiliation to universities, mostly to the Technical University or the University of Munich, the two top-ranked universities interestingly in the European Union. So, we have this sort of place where we have these affiliations at universities, right? But, we don't have undergraduate teaching. We essentially just have PhD students and postdocs, but we interact very much with the universities and in particular also the clinics because we want to have sort of this translational thing in there. And increasingly, that's maybe an interesting link to So, this is maybe a bit similar to NIH type of models like in US, but sort of distributed all over the country, not just sort of in one place. And maybe the one interesting um um um link is that increasingly the government and this sort of realization also in in in Germany is is sort of dawning that there needs to be much stronger transfer component, not just by going into translation to universities, but also in into sort of value creation [00:33:17] outside of the academic realm, which is obviously going so more more into startups and so on, which I think the Boston environment has always been one of the key examples, but now of course more so now also the the the Bay Area when, you know, recently I was was visiting uh I believe maybe Genentech or colleagues, and we went to a restaurant, and then one of the waiters was asking us, "Hey, are you guys talking about AI and bio? You know, I have been also interested." So, like everyone sort of thinking about this startup ideas and so on. And I I kind of felt that's a it's a good environment, and this is something I think our PhDs are now more and more doing, and and we we are sort of learning there, and I think also in terms of sort of value creation, Munich in particular is actually at least in continental or maybe the prime place now for VC capital, but also sort of deep tech and particularly the bio and and defense area and sort of really exploding. So, this stuff starts to work. >> One thing one thing we're talking about startup, I think is really fascinating. I've heard that Germany has had some issues with their startup ecosystem. It's it's try It's difficult to create a startup there compared to in other places in Europe. How like what is the [00:34:18] competitive landscape for startups right now in Germany, and specific and how does Germany become more competitive broadly? Um I I have [clears throat] a vested interest in this because I'm um probably going to get my German citizenship soon. >> Welcome. Let me know if you come come by Munich. So, um I think traditionally Europe but I would say definitely Germany is is quite bureaucratic and processes are far from as digital as you would expect them in different places. Not that I'm saying that the US is maybe the best example for fully digitized processes, but stuff is definitely faster. The thing is people have realized that now and I think the the particular startup creation and implementation is I think more than competitive in Germany versus Europe. It's actually sort of the the the the prime number and in particular like some hotspots burning, but in particular Munich sort of the I think is is the capital for for um um highest number of startups and I think even nowadays if you look into European Union, there's equally as many startups being created as in US. The thing is where Europe and particular also Germany drops off is at the scaling part. So, [00:35:18] you know, early VC and also sort of um non-diluted type of money, this is there. But then when you go into scaling, these big VCs that sort of take these bigger risks are I I think not not closely as present. So, I have I have I saw I saw this again and again that, you know, if you want to sort of really go into the um the sort of bigger tickets sort of series A and so on series B, that you people start going more into US also for that. But maybe I'm over simplifying here, but this is at least my current learning burner. >> Okay, you are on the board or I guess part of several local startups, right? What what does it take to go from, you know, idea to creating a you know, to creating a new startup in Germany? >> I I don't know if if if you heard that the European Union tries to regulate a whole bunch of stuff. And that can be sometimes an advantage. You know, it's kind of cool like, you know, all across Europe we can now go with a mobile phone. And you might have heard that everyone needs to have USB-C things now for their mobile phones. This is kind of kind of cool stuff. So, there's some some good stuff coming. But then of course there's something like things like the AI Act and so on that make things a tad more complicated. Luckily [00:36:18] not yet for research, but you know, for data access across hospitals and so on, there's some and you know, and maybe we're talking about about health deep tech type of stuff, right? So, there's some um systems that I think fantastic in Europe. And I'm thinking about Finland, you know, FinnGen um Denmark and so on with this sort of very detailed description of the population terms of sampling, getting medical records connected far better than I mean US is also similar to a mess like like Germany in terms of sort of getting all the systems sort of to talk to each other. Like when patients, you know, sort of go to to different states in US is similar as in Germany when they go to different places. It's like very hard to build patient directories, right? But there are some places that know how to do this and there's a thing like the European Union tells you it's called the European health data space where in like maybe next 2 years these things need to be sort of going across boundaries. You need to sort of start building these these health record type of combination. So I do think the visions are starting to happen, but the scale is is not as fast. So I think what you need for a tech startup [00:37:18] what I noticed is is always the biggest thing is you need to have the best team for that. And you know, we have professors and and PIs that really think about this. So for example, for for the department that I coordinated, I said, "Guys, every every PI every group leader, I would expect you to at least to have one one one spin-off in in in the next 3 years." And see how that works, right? But you sort of see this sort of get people a a bit a bit on that track. But I think this is happening very much from the bottom up nowadays anyway where the students are actually pushing for that. They do realize that there's a whole bunch of opportunities at the moment, particularly with with this sort of computational angle getting Yeah, I would say pushed even more in the forefront that we can now start actually guiding experiments. We haven't really spoken so much about, right? But once you have these perturbation world models, you can actually go back and drive the experiments, right? So I do think computational startups can be very good also data generation. And I think this is what will be also turned not just into nice papers, which is sort of mostly our business, but also I think it is next generation of startups. And you do see a few of these popping up where you actually get the lab in the loop really implemented. [00:38:18] >> What do you see as some of the the the most promising areas for commercialization of sort of these virtual cell adjacent ideas? >> It's interesting, right? Like if if you look at earlier and of course this is what people always reflect to what OpenAI and so forth and so on wanted to do. Like they didn't really know how to make money and some people might argue they don't even know nowadays, right? And there's a few startups that you know, got really quite strong investment that also said we build a foundation model of biology. We decode biology and so on, without actually saying what you want to do with it. And I've been seeing this again and again that these platform type of ideas are super attractive. You know, they sort of fit into what we know. I mean, you spoke about some of the tools and platforms that we built like Scanpy and scverse and so on, right? But turning this actually into a monetary thing is not as obvious. I mean, there might be some sort of consulting business, but if you say you build a virtual cell platform, typically what investors understand, and this is maybe even more so in the more conservative VC ecosystem in Europe US is that at some point you develop a drug and then you have this clear thing. [00:39:18] So, I saw so many of these platform companies pivoting at some point to we make a drug. And yeah, at the side, of course, we have our our sort of virtual cell, whatever business, but you know, we have two drugs that we found and now this So, we put a lot of money in. Um Maybe you have some other counter examples. And there's a few companies that are sort of still stick with the scaling up of of the AI business, but it's not as clear uh how you would sort of directly turn this into a viable sort of really scalable business model. And there's even room for many of these. Of course, then there's also this angst that at some point like Astra 2.0 might be doing parts of this already anyway, yeah. >> Yeah, I mean, I I think that it's actually this has been a thesis of mine that up until recently every single company that builds a model and builds a something ends up just building drugs because that's the path to monetization over time, right? And then like whereas you you have to like land sales with pharma that's very difficult or some other big big pocket spender. If you're [00:40:19] a tools company, if you say I'm going to build a port like a an asset and then I can sell that off, that's really attractive to an investor. But but we've seen actually the opposite happening this year, right? So, you have like you have in structural, you have a couple, you have bolts, you have Brandon will know them better, but there's a couple of those and then Ty is one that we talked to. And then we've interviewed a few that are doing this do doing reinforcement learning environment plays as well. I would love to see the activity level in structure that that we see in structural also doing things like what No it is doing. >> I mean I I I think No it is is is a good example in the sense that they also have I think a rather nice scale of data generation and a big part of that is also access to data. So you know we talked before about these organoid perturbation type of things and we've been setting up actually in collaboration with a whole bunch of farmer a consortium which we call the cancer organoid perturbation atlas with then farmer actually buys in because they want to get early access to some of these sort of perturbation type [00:41:19] of data sets right and I think some of these deals sort of go in these these these directions. So I think if you couple it with data generation not just with the modeling part, I think this is where sweet spot hits where then also actually farmer gets buying. I think more more of the companies I see they sort of build their mode not just around a virtual cell model but in combination with sort of unique sort of specific data. This is then of course also more value creation. >> I don't know if this is exactly correct but I read that you said somewhere that you have your students follow what what you I think called a Y model in which you have to have a methods and a wet lab advisor. That sounds super interesting, super relevant to the conversation we're having right now about data being so important and all the attention on lab work and that being mode for a lot of companies. Can you talk a little bit about that? >> Yeah I think thanks for raising this. So I've been setting up and since then this has been sort of become quite a big thing what's called the Munich School for Data Science right? So this is beyond just just bio so we had a whole bunch of other disciplines and there we had this supervisor model that is always [00:42:20] two PIs needed come together to make a proposal and sort of proposes to the students in a graduate student type of model which has this this Y so there's like a method e more domain one right and then there's supervisor student. And sometimes we had two students under the Y model so Y plus Y equals H. You get it? But but I I think this idea is it's somewhat even nowadays a bit trivial right? But I I thought it was good to bake this in so hard so that students really ideally go to two lab meetings. One lab meeting where they need to really learn and understand some deeper biological problem, understand the language maybe of a medical setup or something like that. Then they go to a computation typically all computation data science type of students, right? But then they go to the computation one they sort of get pushed the method aspect of it and thereby sucking up a whole bunch of of problems that are sort of not exactly precisely as formulated in in that bio lab. They sort of can't directly put that in a function, but then because they are sort of methody, right? And they're quantitative, they can actually put this into a metric or whatever that then turns into a model and then paper and I [00:43:20] I think this is the way how uh I mean we've been doing this now for maybe 8 years or so this this type of school. I really a whole bunch of papers came out where I I I learned about systems that I had no idea about this before, right? You know, people approaching me and could we do something like this and then you know, I learned something about the lung or the brain that you know, I had no idea about, but then uh I said, "Oh hey, cool data. I think our model types could fit. Let's build a project around it." And then those students became the experts in my lab for that sort of detailed biological thing together with a partner. And then maybe we drive this forward and this actually nowadays extends also across the data types. Earlier on a lot of things have been around just RNA, but you know, since then you know, we had a very nice paper in science last year on single cell proteomics. We've been extending things to spatial observations. You know, there's sort of all these additionalities and things that you can can can measure and I think then interesting questions come up in that particular domain where you know, these essays work very well. >> Yeah, I mean this is my natural inclination is to like gathering cross-domain stuff. It I think it's all often ripe for good good generation of [00:44:21] really good and interesting ideas. Have you just as a follow-up these you've been doing this eight years now, I guess. So what kinds of you know, sort of longer term outcome benefits have you seen from this? >> I think one thing that this is also I think how we got students to even join in this maybe initially it sounds like more work, right? You kind of need to learn two things and you go to two lab meetings and so on. But I think you once your students are done in in in in in Germany or I think in general but particularly I think in Europe in general people sort of typically do a master's first before they go into a PhD, right? So not just a bachelor's in the US, right? So, you do like one or two year more of master. And and then they sort of know science a little bit, but they don't exactly know what sort of interests them. So, in this combination type of thing, you know, they can sort of sample a bit because in this grad school they just get different proposals in, so they can sample a bit what they're interested in in terms of domain, but then they can also play around a bit with the method side. So, I think that was kind of attractive. Yeah, I I think with this sort of trying out uh different things, I had the impression that this sort of turned the students into very good data analysts, [00:45:21] but then also modeling people. And then a downstream, you know, the sort of areas where they they went into was also equally diverse, you know, so some people went like and now they're in the biomedical domain into pharma, but you know, some obviously stayed in research, some went more into some sort of startup direction, some went into the very pure AI type type of work. I think Nvidia just a few months ago were hired in 1 month two of my people going very much into sort of the computation aspect and also a bit of a bio part that Nvidia's now building up and there's similar other of the big companies. So, I think it it gives them an an an interesting profile, but I think when you're out there when you get interviewed, you know, it's it's a good idea to say, you know, I I didn't only do like this fancy model that improved maybe 2% in a NeurIPS paper type of thing, but you also sort of had the real sort of more complex, deeper application that they're like, you know, I had to help find as a use case. >> There's been a movement amongst Atomwise AI like several guests we've had or we've had a blog post about how the wet lab scientists have started basically becoming computational thanks to Atomwise AI. I don't know if this is I'm kind of curious like as this evolves, as [00:46:23] basically now Atomwise AI enables any scientist to be now become their own computational, what do you think the evolution of this sort of like co-advising strategy is and also what do you think the evolution of like integration between wet lab biologist and computational teams are? I mean, I also can think of like I've heard several you know, famous CEOs say things like our strength is integration between computational and wet lab teams. So, I'm I'm curious to see how you think that evolves and how you also like set up culture to exploit, you you these new technologies. >> Let let me maybe sort of keep the the culture part a bit out because I think it's a hard one to answer, but a very good one. So, let me first sort of talk about the I think very cool cool point that I think you asked about where where these things happen sort of goes with deep integration. I think one of the most exciting frontiers we currently face and I think maybe have as an opportunity is we can really now start guiding the experimentation. So, where before you we build these human cell atlases, we had sort of this mapping aspect. Then we talked about the [00:47:23] world model prediction problem, right? So, from mapping we went to prediction. But now we can go to actually steering the next experiment, right? And this has always been the scientific method, you know, sort of putting our hypothesis in the loop and then doing something with it, but quite frankly, very often the papers ended with some model, right? maybe some sort of concept figure, maybe had a little bit validation. But then to be you start the next project, right? And of course you learn a bit, but there's not like not like a continual learning type of thing. But now, you know, I think we can actually really loop back more and more. This is happening very much so in uh molecule design or material design and so on. You sort of the lab in the loop is being worked on out. And even though some big pharma have been putting this out as sort of a general general big theme, there's not many examples that where really people do multiple [clears throat] loops of this integration of you know, experiment, a computation model, then going back, whether this is now two person one, right? But like this integration is hard to set up. And it's hard to set up once because of modeling because actually sort of once you have a model, actually deciding what's an optimal next question is not trivial. And we actually brought out I think it [00:48:23] should be on on bio archive today or tomorrow and new thing we which we call lab compass, which is an inverse generative model that actually allows you once you have a forward world model or whatever to tell you what you need to do in order to reach a certain state. So, it's super exciting for us, right? And I think a big part of the work was not just the model, but actually the work with experimental people, in this case a good friend and colleague from University of Cambridge who was doing hematopoiesis. In addition, had a sort of a rapid cycle system where we actually set up three four lab in the loops where we iteratively improved on a model. And this is I think a really hard thing. And this sort of goes into your second question, Brandon, right? Like how do you bring a culture where once your computational person or model or whatever predicts something that you actually get to convince experimental people to put their life time in because these experiments often take some time right to actually buy into it and actually measure some of these follow ups. And sometimes you know this works a bit better if it's sort of molecules because then you know also the stakes are not as high but you know if you like say now you need to do this new cell and you need to add these very strange things. Do you really want to do this? Like shouldn't we maybe not take a bit [00:49:23] of our gut feeling and then you know these sort of disintermediate solutions come. So I think really setting this up is far from trivial. That needs culture and it might be the same person doing it. A lot of people then start throwing around robotics and so on as well but I I think this is a tad too early because the the cycle times in terms of our essays are still a bit on the long side but I do think this will come and you know obviously the the ultimate vision being that we want to RL everything. We want to do RL also in in in in this loop. It's a bit out there but you know I think we're we're slowly getting there. At the moment it's more maybe active learning. >> I think that very much like we talked to Lila there that is there they're kind of their core thesis is is is is that basically the lab is part of the RL environment. Kind of is what I mean. Yeah, no that's great. Fabian, it is late at night for you now. I'm I really appreciate you taking the time out of your evening to to talk to us. Look forward to to watching your research and when I say you I mean your lab and colleagues and you as well. I'm I really look forward to watching this progress and keeping careful track of the things that we talked about [00:50:23] today. >> Yeah, thank thanks so much RJ Brendan for having me. I I think these links are important. This sort of looking beyond what maybe you do like in your own sort of smaller disciplines and you know these discussions helped me a lot. I really enjoyed the discussion. Had a lot of fun. Thanks. >> Great. Thanks. Take care. Thank you.