Causal Models Need Causal Data - Xaira’s X-Cell model (Bo Wang & Ci Chu)

[00:00:00] [Xi Chu] And what really blew my mind away is when I saw the model make prediction, [Xi Chu] just printing out the heat map of the gene expression changes, [Xi Chu] look at actual raw data and line up the linear baseline prediction, [Xi Chu] the ground truth and the X-Cell prediction altogether. [Xi Chu] It's visually very clear to see that X-Cell prediction is much more similar to [Xi Chu] ground truth than the linear baseline. [Bo Wang] This is a wow moment I was talking about in the beginning. [Xi Chu] This is the first time that someone can put together not just one perturb-seq, [Xi Chu] but seven genome-wide perturbs campaigns together. [Xi Chu] Something that jumped out to us biologists right away is that some of the [Xi Chu] perturbations are context universal. [RJ] Hi, I'm RJ Honicky, CTO of MiraOmics. [RJ] This is Brandon Anderson who builds RNA therapeutics at Atomic AI. [RJ] And this is the Latent Space AI for Science podcast. [RJ] One of the themes that has run through the podcast is how the lab and the [RJ] experimentation and the real world have probably the biggest impact and have [00:01:05] [RJ] the most relevance to whether something is AI for Science or something like [RJ] B2B SaaS. [RJ] We're really happy to have in the studio with us today, Bo Wang and Xi Chu [RJ] from Xaira Therapeutics. [RJ] At Xaira, they're building with a bunch of other people, a AI drug discovery [RJ] platform. [RJ] They're using high throughput experimentation system to collect very large [RJ] data sets and then training AI models that can predict the way that your cells [RJ] in your body will respond to drugs and therapeutics. [RJ] Really happy to have you, big fan of your work. [RJ] Why don't you two introduce yourselves to the listeners? [Bo Wang] Hello everyone. My name is Bo Wang. [Bo Wang] I'm SVP and Head of Biomedical AI at Xaira Therapeutics. [Bo Wang] Joined Xaira about eight months ago. [Bo Wang] And before that, I was Associate Professor at the University of Toronto in [Bo Wang] Canada. [Xi Chu] And I'm Xi Chu. [Xi Chu] My first name is incredibly difficult to pronounce unless you speak Mandarin, [00:02:09] [Xi Chu] so I go by Chu as in Chewbacca or Pikachu. [Xi Chu] Favorite fictional character. I'm the SVP of AI enabled discovery at Xaira. [Xi Chu] I joined about more than two years ago when I was still in stealth mode. [Xi Chu] And here I lead the high throughput biology group generating the kind of data [Xi Chu] that will feed our AI models and also think about their applications. [Xi Chu] Before this, I spent about a decade at the intersection of AI and big data and [Xi Chu] biology. [Xi Chu] As I previously worked at Insitro, leading the in vitro discovery platform [Xi Chu] there. [Xi Chu] And before that, I was at Verily, which spun out of Google X. [Brandon] Okay. [Brandon] So you, you're at Xaira, the company, which is on the Pareto frontier of [Brandon] confusing names and mega rounds. [Brandon] So Xaira is, I think kind of came out of stealth like a few years ago and just [Brandon] really big or kind of out of nothing. [Brandon] So I'm curious if you can explain a little bit about what is Xaira's mission? [00:03:09] [Brandon] What is their thesis statement? [Brandon] Like what is, you know, special about Xaira and you know, kind of where you're [Brandon] going in the future. [Xi Chu] Yeah, Xaira is a AI enabled drug discovery company. [Xi Chu] And at the core of our mission, we're using AI platforms to generate better [Xi Chu] therapeutics to advance patient care. [Xi Chu] And so we will be making drugs using different AI capabilities. [Xi Chu] There are three main AI platforms that we're building here. [Xi Chu] The first one is protein design work that spun out of our co-founder, Dr. [Xi Chu] David Baker's group from UW. [Xi Chu] A lot of the current generation of protein designers are here in the company. [Xi Chu] So there the thinking is to use advanced AI technology to develop [Xi Chu] molecules against previously undrawable targets. [Xi Chu] The second AI platform, [Xi Chu] I guess we'll spend a lot of time talking about today is the one that Billy [Xi Chu] and I have been working on for quite some time and just released a pre-print [00:04:11] [Xi Chu] on. [Xi Chu] That's the virtual cell or foundation model of biology work. [Xi Chu] There are the hope is to build a model to predict biology exactly like you [Xi Chu] said and predict what genes and drug molecules will affect cell biology. [Xi Chu] And the third piece, which we're beginning to build now is patient [Xi Chu] representation models. [Xi Chu] And the goal there is to have AI models that can understand which patients [Xi Chu] will respond to which therapeutics. [Xi Chu] So hopefully together, [Xi Chu] these platform technology will help us make better drugs faster and with a [Xi Chu] higher success rate than previous technologies to transform what is used to be [Xi Chu] artisanal trial and error in the past into more and more into an engineering [Xi Chu] discipline. [Bo Wang] I think what sets Xaira different is not just the 1 billion around, [Bo Wang] but also I think Xaira is one of the very few AI native companies for drug [Bo Wang] discovery that works from end to end of all sections of drug discovery from as [Bo Wang] early as, [00:05:11] [Bo Wang] you know, target ID and the protein design, small molecules, and to face one, [Bo Wang] two, three clinical trials, [Bo Wang] we aim to use AI to accelerate every part of the drug discovery. [Bo Wang] So that not only we increase the success rate of developing drugs, [Bo Wang] but also greatly, you know, [Bo Wang] reduce the cycle time so that we can have new drugs instead of every 10, [Bo Wang] 20 years. [Bo Wang] So hopefully we can have the cycle time. [Bo Wang] So we have more useful drugs for patients. [RJ] That's really interesting. [RJ] I know there's a lot of interest right now in that third thing, maybe called [RJ] translation from the lab to the clinic. [RJ] Where are the bottlenecks? You have these three models. [RJ] What are the bottlenecks that you're addressing and sort of like, how are you [RJ] doing that? [RJ] Why are you doing it that way? [Bo Wang] There is a AI native company. [Bo Wang] Almost every parts of the sections of drug discovery, trying to use AI to [Bo Wang] revolutionize how we develop drugs. [Bo Wang] So the early part, we build causal foundation models, or sometimes we call it [00:06:14] [Bo Wang] virtual cell proteins. [Bo Wang] We have, you know, state of arts, protein engineering models, [Bo Wang] and we have also patient representation learning models. [Bo Wang] And I think what Xaira is trying to do is not only we develop AI models, [Bo Wang] but also we create the right data sets to empower these models. [Bo Wang] And I think what's really make me excited to work at Xaira is we always aim to [Bo Wang] connect three AI models together instead of letting them work individually by [Bo Wang] their own. [Bo Wang] So when we design virtual cell models, we look for connections to that, [Bo Wang] can we find targets that is easier to apply the protein engineering models? [Bo Wang] And then even when we design the cellular causal models, can we connect to [Bo Wang] patient representations? [Bo Wang] What are the right patient data to connect to the cellular models so that we [Bo Wang] have something to show clinical utilities? [Bo Wang] So I think what really makes me excited is before I joined Xaira, [00:07:18] [Bo Wang] I'm kind of a professor in computational biology department or computer [Bo Wang] science department, [Bo Wang] where we mostly working on computers. [Bo Wang] We look at the data, we look at arrays, et cetera. [Bo Wang] But once coming to Xaira, [Bo Wang] what really excites me is that I get to talk to people like Chu, [Bo Wang] lots of drug hunters, you know, [Bo Wang] extremely experienced drug hunters to really understand their pinpoint. [Bo Wang] So when we design AI models, we think about questions that really excites [Bo Wang] biologists. [Bo Wang] So later maybe we can talk about how one of the rewarding signals I receive [Bo Wang] after we develop X-Cell is that like the wow moments from biologists that this [Bo Wang] is the first time biologists actually find the model can predict exactly how [Bo Wang] these unseen cell lines kind of respond to different perturbations. [Bo Wang] So that's kind of the part really excites me is the integration of kind of dry [00:08:18] [Bo Wang] lab or AI models to wet lab or the biology or even eventually to the clinical [Bo Wang] side. [RJ] With this clinical model, I know you guys are aiming to take a drug all the [RJ] way to FDA approval and beyond. [RJ] Where do we stand now? [RJ] I don't know if you're able to talk about this, but like, [RJ] are you able to collect data from clinical trials and tie that back yet? [Xi Chu] As Bo said, I think if you think about drug discovery process, it's easy, [Xi Chu] right? [Xi Chu] You just need to find the right target, make the right molecule, and find the [Xi Chu] right patients to give them to. [Xi Chu] Of course, each of those steps are incredibly difficult to get right. [Xi Chu] And so far, like I said, just now, it realized a lot of untraddling and erring [Xi Chu] guesswork. [Xi Chu] And the main issue, I think, is that we don't have the right biological data, [Xi Chu] really the power, the training of a predictive model. [Xi Chu] And in protein design space, I think that's where we have seen the most rapid [Xi Chu] progress so far. [00:09:18] [Xi Chu] That's partially because we have a lot of data, high quality data, [Xi Chu] over 70 years curated by the entire community, [Xi Chu] people deposit protein structures into a database, right, called PDB. [Xi Chu] We also have a lot of sequence data, right, [Xi Chu] collected over the years from different genomes that can help inform the model [Xi Chu] as well. [Xi Chu] And it's these high quality data that are collected and accumulated that usher [Xi Chu] in this revolution in a protein design and alpha fold and other folding [Xi Chu] models. [Xi Chu] In the other domains, such as clinical model prediction, [Xi Chu] such as virtual cell, [Xi Chu] we are nowhere near the same kind of massive data that are high quality. [Xi Chu] And I think it's mainly a data limitation issue. [Xi Chu] So to your question, that's why we're very invested in generating these data, [Xi Chu] particularly causal data in cell biology in a lab. [Xi Chu] And that's, I think, what made it possible to innovate on the algorithm side [Xi Chu] as well to usher in virtual cell models. [Xi Chu] On the patient side, it's a very interesting question. [00:10:21] [Xi Chu] Perhaps that's one of the hardest data to get because getting access to high [Xi Chu] quality patient samples is difficult in itself. [Xi Chu] Getting it matched to the right clinical annotation so that you can actually [Xi Chu] learn the difference, [Xi Chu] the bridge between molecular data and clinical response, that's even harder. [Xi Chu] And you might be able to do that across different disease severities, [Xi Chu] but it will be harder to collect the right data to predict which drug [Xi Chu] treatment will or would not respond in a particular patient or not. [Xi Chu] And so that takes a lot of thought and a lot of careful curation to generate [Xi Chu] data out of. [Xi Chu] And so we're beginning to go into the area, but hopefully we'll be able to [Xi Chu] share more soon. [RJ] Awesome. Maybe we should switch gears now. You just released Xcel. [RJ] Why don't you guys describe, I'll butcher it. [Bo Wang] Xcel is Xaira's first virtual cell models. [Bo Wang] It is an AI model that can predict the response to genetic perturbations. [Bo Wang] Certainly we can extend it to other type of interventions such as drug [00:11:23] [Bo Wang] perturbations, [Bo Wang] chemical perturbations, et cetera. [RJ] So can you just describe for the non biologists that [RJ] are listening, what is a perturbation? What do you mean by that? [Xi Chu] In ourselves, when Bo talked about genetic perturbations, our human cell [Xi Chu] typically have 20,000 genes. [Xi Chu] Not all cells express every gene equally. [Xi Chu] That's why your eye cell, your skin cell, your heart cell, [Xi Chu] even though they share the same genome, they function very differently. [Xi Chu] A lot of that is determined by selective gene expression that determine the [Xi Chu] type and the state of the cell. [Xi Chu] So what we do is to build a model that you can in silico [Xi Chu] ablate certain genes from the cell. That is a in silico perturbation. [Xi Chu] That's to say, if I reduce the expression of this gene in a cell, what is the [Xi Chu] implication for the rest of the cells? [Xi Chu] What's the biological consequence? [RJ] You basically turned the knob down on one gene. That's right. [00:12:25] [RJ] And then what happens to all the other genes in that cell? [Xi Chu] Correct. [Xi Chu] And the hope is of course to predict the effect on all the other genes, [Xi Chu] but maybe even more things than gene expression, [Xi Chu] such as the function of the cell. [Xi Chu] And that's important that that's therapeutically relevant because a lot of [Xi Chu] drugs are inhibitors and they function through exactly that, [Xi Chu] turning down the activity of a protein or gene. [Xi Chu] And so we can start with gene perturbation prediction. [Xi Chu] The hope is that we can also go to pathway inhibition prediction, so on and so [Xi Chu] forth. [RJ] So a pathway is just a set of genes that all kind of talk to each other by [RJ] this gene expresses a protein. [RJ] That protein has some impact on another gene. And so forth and so on. [RJ] There's this long chain reaction of genes and proteins. [RJ] And then so that that's called a pathway. [RJ] And so if you interrupt that or somehow change it, [RJ] then that has an impact on the larger phenotype of the cell, [RJ] what the cell looks like, does, et cetera. [RJ] That's exactly right. [Brandon] Yeah. [Brandon] So you have what you call the virtual cell or you're creating a virtual cell [00:13:28] [Brandon] and virtual cells are very popular. [Brandon] It's a lot of people are interested in this concept, [Brandon] but I think your approach is somewhat unique or separate from other people are [Brandon] doing. [Brandon] Can you explain what do broadly people mean when they say virtual cells? [Brandon] What are some of the distinct other strategies? [Brandon] And then like, what is your specific strategy that you're going for? [Bo Wang] Certainly, [Bo Wang] virtual cell is a very high level term to describe an AI model that is able to [Bo Wang] predict or describe what cell looks like or predict the cell functions after [Bo Wang] certain interventions. [Bo Wang] It's a very high level concept. It was first of all, it was not a novel idea. [Bo Wang] We had virtual cell project almost 20 years ago. [Bo Wang] But back then, [Bo Wang] sometimes we call it virtual cell 1.0 is that people are trying to derive [Bo Wang] differential equations to trying to use mathematics to describe what's the [Bo Wang] response for certain pathway intervention, [00:14:30] [Bo Wang] as you just mentioned, [Bo Wang] and by feeding these equations to different observations. [Bo Wang] And largely speaking, [Bo Wang] that was a failed attempt in the sense that the biology is just way too [Bo Wang] complicated to write in a few predefined set of differential equations. [Bo Wang] Moving forward, with the rise of language models, [Bo Wang] I think the idea of using AI models to mimic how cell responds to different [Bo Wang] interventions by data-dreaming approach start to get popular. [Bo Wang] And I think three years ago, [Bo Wang] almost just four months after Chagibee was released, [Bo Wang] our lab at University of Toronto published one of the early foundation model [Bo Wang] of single cell genomics, [Bo Wang] call it SCGPT. [Bo Wang] It can kind of interpret it as a GPT-like model for single cells. [Bo Wang] And it quickly become very popular in the sense that for the first time, [Bo Wang] we have a foundation model that is able to tackle different downstream tasks [00:15:33] [Bo Wang] using the same model, [Bo Wang] such as we can use the same model to integrate different batches of single [Bo Wang] cell RNA-seq. [Bo Wang] We can use the same model to predict multi-omic integrations. [RJ] Let's define those things. [RJ] So batches, integrate different batches of RNA-seq. [RJ] So you have different equipment, you're all collecting. [RJ] So maybe you're collecting the same labs. [RJ] Yeah, different labs, different time of day, different phase of the moon, [RJ] whatever. [RJ] And those actually have a big impact on the data that you collect. [RJ] And so there's a big problem of how do I even compare this dataset to that [RJ] dataset when there's all this other differences that have nothing to do with [RJ] the gene expression and just how I measured it. [Bo Wang] We call that batch effect. [Bo Wang] We certainly want to remove the batch effect while preserving the cell types, [Bo Wang] which are important biology we want to reserve. [Brandon] So this is sort of analogous to the tank problem in image classifiers, [Brandon] for example, as the models pick up on these crazy, spurious features, [Brandon] which have nothing to do with what you actually care about, [00:16:36] [Brandon] the underlying biology. [Bo Wang] Exactly. Certainly the core idea of integrating different batches is to [Bo Wang] keep the biological signals while removing the batch effect. [Bo Wang] And before these foundation models, [Bo Wang] what happens in single cell domain is that for every task, [Bo Wang] biologists have to choose the so-called specialist state of arts approaches. [Bo Wang] And with foundation models such as SGV or Geneformer, [Bo Wang] what we hope to bring is that one model that solve all the tasks in single [Bo Wang] cells. [Bo Wang] And with the popularity of foundation model, [Bo Wang] lots of researchers come together under CZI, Chanzaku Brain Institute, [Bo Wang] and we published a perspective paper as journal cell to coin, [Bo Wang] for the first time, coin the term virtual cell, almost virtual cell 2.0, [Bo Wang] in the sense that let's use data-driven approaches if we cannot describe, [Bo Wang] let's learn it. [Bo Wang] So that's the idea of virtual cell so that Genesee can build language model or [00:17:40] [Bo Wang] language type of model to predict what the cell types looks like, [Bo Wang] how the cell respond to different interventions. [Bo Wang] And eventually we can replace all the cellular experiments by simply running [Bo Wang] simulations on computer without even running the actual experiments. [Brandon] Maybe for a bit more context, you can think about this as, [Brandon] so a virtual cell is just a general concept, [Brandon] but you think cells have 20,000 genes in them. [Brandon] And in most human cells, [Brandon] I think what roughly four to 5,000 are usually active at any given time or [Brandon] expressed at reasonable levels. [Brandon] So you look at a normal cell, you might have four to 5,000 genes doing things. [Brandon] And so your question is, in many cases, the way medicine works is you [Brandon] target a protein, [Brandon] or you target some sort of something which makes proteins more common or less [Brandon] common, [Brandon] or they stop the protein from doing something. [Brandon] And your goal is, given [00:18:41] [Brandon] some number of genes which are in a cell, every cell has a different [Brandon] composition of genes, what is going to change? [Brandon] Will some pathway die off? Will some pathway grow? [Brandon] And from this, [Brandon] you could predict how medicine is going to work by just understanding how [Brandon] changing one specific gene or some cluster of genes could change everything. [Bo Wang] Is that a correct understanding? [Bo Wang] Yeah, that's a correct high-level understanding about virtual cell. [Bo Wang] What's happening for this field is that we are lacking a concrete definition [Bo Wang] of virtual cells, [Bo Wang] and people almost equate foundation model with virtual cell. [Bo Wang] But in my view, virtual cell is probably a much broader concept than just [Bo Wang] foundation models. [Bo Wang] Foundation models mostly provide a reliable, semantic, meaningful [Bo Wang] representation of cells. [Bo Wang] But I think virtual cell is more dynamic in the sense that, [Bo Wang] can we build AI models, [00:19:42] [Bo Wang] even predict the development of the cell states across different times? [Bo Wang] Or can we even describe the spatial changes at the cells, different cellular [Bo Wang] resolutions? [Bo Wang] In my understanding, [Bo Wang] we are really at the early stage to develop such comprehensive virtual cell [Bo Wang] models, [Bo Wang] and the foundation model is really just a starting point. [RJ] AI models always begin with the data. [RJ] You are building a high-throughput experiment or have built and are continuing [RJ] to develop a high-throughput experimentation system. [RJ] That sounds really cool and really complicated. [RJ] Can you tell us what that entails? Like, what are you doing? [RJ] What are the experiments that you're running? [RJ] How does that inform the building of an AI model? [RJ] And why do this rather than pick up the Cell X-Gene database, [RJ] which is a collection of gene expression data that has been aggregated over [RJ] the public data sets? [Xi Chu] Yeah, great question. I want to pick up where Bo left off. [00:20:43] [Xi Chu] I think Bo said something pretty profound, going from a representation model, [Xi Chu] the foundation model of biology, to a virtual cell. [Xi Chu] And the key difference there is perturbation prediction or dynamic processes [Xi Chu] in biology. [Xi Chu] That's a causal concept. For that, I think we need causal data. [Xi Chu] And if you look at Cell X-Gene, that's a fantastic dataset that curated, [Xi Chu] in the beginning, more than 33 million cells, now a lot more than that. [Xi Chu] And at the time when SGP-T was trained on that dataset coming out of Bo's lab [Xi Chu] in Toronto, [Xi Chu] that was mostly an observational profiling dataset. [Xi Chu] It's a descriptive data, not causal, and mostly profiling healthy human [Xi Chu] donors. [Xi Chu] And so the model that was trained on this dataset is very, [Xi Chu] very good at doing descriptive tasks, [Xi Chu] such as harmonizing across batch effects, [Xi Chu] removing effects from different labs, different technologies. [00:21:45] [Xi Chu] But I think Bo, us, [Xi Chu] and many others in the field have found that these models that are trained on [Xi Chu] descriptive data do not yet outperform linear models on causal tasks, [Xi Chu] perturbational tasks, or what we call counterfactual tasks. [Xi Chu] If I did this to the cell, then what would happen? [Xi Chu] That makes intuitive sense to a biologist because the correlation data in the [Xi Chu] descriptive dataset can be fit with many, [Xi Chu] many possible causal structures. [Xi Chu] In a very simplistic case, let's say you observe gene A, B, C, [Xi Chu] all go up and down together in your descriptive dataset. [Xi Chu] You can infer that A regularly B and C, that's why when A goes up, B and C [Xi Chu] also go up. [Xi Chu] You might also say that B regulates A and C, and that would be perfectly [Xi Chu] reasonable as well. [Xi Chu] You could also say that A regulates B and C is completely regulated by [Xi Chu] something different. [Xi Chu] You see the problem there. [Xi Chu] There's N number of ways to fit a causal regulatory network into descriptive [Xi Chu] data. [Xi Chu] Fundamentally, we believe that the original data are underpowered to learn [00:22:48] [Xi Chu] causality truly. [Xi Chu] This is why we realized pretty early on that we need to really start building [Xi Chu] causal dataset to train a causal model. [Xi Chu] What are the ways to do that? [Xi Chu] I think the field has come of age to do these at scale technique that we call [Xi Chu] high-throughput biology. [Xi Chu] There are many ways to generate these causal data at scale. [Xi Chu] The technique that we have focused on is something called perturb-seq. [Xi Chu] For the listeners who are not familiar with that technology, [Xi Chu] it combines high-throughput pulled crystal perturbation together with [Xi Chu] single-cell RNA-seq technology to build 2D datasets. [Xi Chu] Let me break that down. [Xi Chu] We [Xi Chu] just talked about in a cell, there are at least 20,000 analytes to measure. [Xi Chu] These are the genes. These are both the features to measure. [Xi Chu] These are also the liver to perturb the cells with. [Xi Chu] For clarity, let's call them perturbations and gene expressions. [00:23:48] [Xi Chu] Perturbation on one axis and the features that you measure that describe the [Xi Chu] cell on the other axis. [Xi Chu] Perturb-seq is a technique that leverages the latest breakthrough in live [Xi Chu] biology, CRISPR-Cas9. [Xi Chu] These are bacterially derived enzymes that allow you to disrupt gene [Xi Chu] expression in mammalian cells, [Xi Chu] in human cells, for example. [Xi Chu] We can do so in one at a time fashion. I can take out one gene at a time. [Xi Chu] Of course, [Xi Chu] that would be incredibly difficult to scale if I want to do all 20,000 gene [Xi Chu] expression back out in one single experiment. [Xi Chu] Probably need a huge factory, a lot of robots to do that. [Xi Chu] Or you can do them in a pooled fashion. I love pooled experiments. [Xi Chu] These are hyperscalable. [Xi Chu] We have lab tricks that allow us to disrupt one gene per cell, [Xi Chu] but do all 20,000 genes across many, [Xi Chu] many cells in one single pooled experiment. [00:24:48] [Xi Chu] Perfectly scrambled so there's no batch effects, there's no play-to-play [Xi Chu] differences. [Xi Chu] That's off the perturbation throughput. [RJ] Basically, [RJ] use some sort of commutatorial trick to first perturb all the different genes [RJ] in different combinations, [RJ] and then you can read them out and do some math on it. [RJ] And you basically pull out a whole bunch of different experiments in one [RJ] experiment. [Xi Chu] Correct. It requires barcoding technology. [Xi Chu] And that barcode is actually achieved by directly reading out what kind of [Xi Chu] CRISPR guide RNA is present in which cell. [Xi Chu] So for CRISPR-Cas9, [Xi Chu] this bacterially derived machinery to work in mammalian cells, [Xi Chu] you just have to deliver two things to each cell. [Xi Chu] You have to deliver the protein, the Cas9 protein, that does the job. [Xi Chu] And you have to deliver an address barcode encoded by a short piece of RNA [Xi Chu] called a guide RNA. [Xi Chu] The guide RNA tells the protein where to go in the cell, purely via [Xi Chu] Watson-Quake base pairing, ATCG. [00:25:48] [RJ] So it matches a part of the gene. [RJ] It's sufficiently long to say this will match the correct gene, [RJ] and then that guides it to connect to the right and reduce the expression of [RJ] that particular gene in the cell. [Xi Chu] Correct. We designed this guide to go to the promoter part of the gene. [Xi Chu] That's the beginning stretch of every gene before the transcription starts. [Xi Chu] And if we bring the Cas9 protein to there, armed with the right effector, [Xi Chu] the silencer, that promoter will get shut off, [Xi Chu] and that gene will never be transcribed out of again. [Xi Chu] So effectively, we tune down the expression level of that gene. [Xi Chu] And so all you have to know is figure out which guide RNA is in which cell, [Xi Chu] and that can be done using genomic readouts. [Xi Chu] That's the barcode. [Xi Chu] And you can then infer which gene is being silenced in which cell. [Xi Chu] So that's the way you scale throughput on the perturbation side. [Xi Chu] On the readout side, it's a 2D dataset, right? [Xi Chu] So we just talked about one of the dimensions. [00:26:48] [Xi Chu] On the readout side, we leverage single-cell RNA-seq technologies. [Xi Chu] So these are also recent technologies in the last decade that have been scaled [Xi Chu] that can let you read out the expression level of all 20,000 genes [Xi Chu] simultaneously from each cell. [Xi Chu] So armed with both high-throughput CRISPR perturbation and high-throughput [Xi Chu] single-cell RNA-seq technologies, [Xi Chu] all of a sudden, [Xi Chu] we can generate these 2D datasets where we systematically perturb or knock [Xi Chu] out, [Xi Chu] knock down every single gene in the human genome in the cell type, [Xi Chu] and we read out this impact on every other genes in the same cells. [Xi Chu] So we generate these 2D-rich datasets, [Xi Chu] not that different than the type of PDB data that trained alphaphone models, [Xi Chu] right? [Xi Chu] If you think about that, that's hundreds of thousands of protein entries. [Xi Chu] If those are the rows, columns are the XYZ coordinated of every single amino [Xi Chu] acid. [Xi Chu] That's also a 2D dataset. [Xi Chu] And I think it's these type of rich 2D datasets that power the training of [Xi Chu] foundation models of biology. [Brandon] I find it really fun how you have turned a fairly straightforward assay in [00:27:52] [Brandon] using... [Brandon] This is NGS sequencing, right? [Brandon] Next generation sequencing, very high-throughput. [Brandon] You've used this to scale a simple perturbation response, [Brandon] which is inevitably maybe not all that interesting to this massive scale of [Brandon] over a... [Brandon] Basically, an arbitrary number of cells. [Brandon] I think you did 25 million or something. [Xi Chu] So it's actually a lot more than that. [Xi Chu] So 25 million is what came out of the most stringent quality filtering. [Xi Chu] It's actually as much of a scientific challenge to figure out how to do CRISPR [Xi Chu] and scRNA-seq as it is an engineering challenge. [Xi Chu] In the first part of the experiment, oftentimes we have to harvest tens, [Xi Chu] if not hundreds of millions of cells, [Xi Chu] and they go through various quality funnels to arrive to give the Bowen team [Xi Chu] the highest quality data at the end. [Xi Chu] That's incredibly difficult to do because, as you can imagine, [Xi Chu] all of these techniques have been published by academia before, [Xi Chu] and they work very well in small-scale experiments. [Xi Chu] But when you think about scaling them to a genome-wide perturbation, [00:28:55] [Xi Chu] we're talking about handling hundreds of millions of cells. [Xi Chu] Techniques that are published in academia used to be all about handling fresh [Xi Chu] cells. [Xi Chu] Cells are still alive. [Xi Chu] And that may be okay if your entire experiment takes only an hour or two. [Xi Chu] It's not quite easy to handle cells across a 14-hour day. [Xi Chu] That's hundreds of millions of cells. [Xi Chu] And so by the end of the day, is the Jokumet team, [Xi Chu] you can easily detect stress signals from the cells and from your scientists [Xi Chu] in the lab. [Xi Chu] And quickly we realized that's not the way to do these data generation. [Xi Chu] Machine learning is very quality dependent, and we want to make it the highest [Xi Chu] quality data to our AI teams. [Xi Chu] So we're putting a lot of engineering thought in, [Xi Chu] industrialized the whole workflow step by step, [Xi Chu] introduced chemical fixations so that we lock the state of the cells in at the [Xi Chu] beginning of this experiment, [Xi Chu] but figure out ways that it doesn't disrupt all of the molecular biology steps [Xi Chu] afterwards. [Xi Chu] It doesn't impact data quality so that we can do all of these data generation [00:29:57] [Xi Chu] in a time-shifted operational manner that's very [Xi Chu] not prone to batch effects. [RJ] One thing that you didn't mention is that you're using some sort of stem [RJ] cells. [RJ] And so obviously you don't have [RJ] brain cells or blood cells. [RJ] Or if you did, then you would have a big combinatorial effect on that. [RJ] So why are you [RJ] convinced that working on stem cells, [RJ] which my understanding is that there are actually blood cells that have been [RJ] sort of the stem cell behavior has been unlocked on them. [RJ] And that causes some sort of stress on the cell as well. [RJ] So you have these like sort of not quite blood cells that are stressed. [RJ] And then how, why are we convinced that that is a good proxy for a brain cell [RJ] or whatever your study? [Xi Chu] Yeah, not quite. So we [Xi Chu] didn't actually start with stem cells. That was more of a later development. [00:30:57] [Xi Chu] When we started data generation, [Xi Chu] so we put out the method that I talked about, [Xi Chu] as well as the first two datasets, [Xi Chu] which is the world's largest perturbsic data release at the time. [Xi Chu] Last June in the preprint, we call it dataset XLS Orion. [Xi Chu] That was actually generated from two cell lines. [Xi Chu] A lot of this field's early work started with cell lines. These are cells. [Xi Chu] Cancer cell lines. One of them is a cancer cell line. [Xi Chu] The other is just a cell line. These are immortalized cells. [Xi Chu] Some of them are derived from cancers. Hence cancer cell lines. [Xi Chu] Others are just derived from primary cells but have been immortalized. [Xi Chu] Many times grown for many, many years in various labs. [Xi Chu] People start with these cell lines in the beginning, as you can imagine, [Xi Chu] because those are easy to do. [Xi Chu] It's easy to scale, easy to grow a lot of cells out of. [Xi Chu] Turns out the ability to grow millions of cells is actually critical for doing [Xi Chu] these large experiments. [Xi Chu] So we started there first. [Xi Chu] And they actually still capture the characteristics of the cell types that [Xi Chu] you're derived from, [Xi Chu] colorectal cancer, as well as [00:31:59] [Xi Chu] hematopoietic cells. [Xi Chu] But later on, in the most recent preprint, we actually explained it to many [Xi Chu] more cell types. [Xi Chu] Now some of these are still cell lines or T cells. [Xi Chu] We chose to use cell lines, but some of these have now gone into primary [Xi Chu] cells. [Xi Chu] So we did one experiment in iPSC. These are induced pluripotent stem cells. [Xi Chu] And another experiment in, and we think this is the most ambitious and coolest [Xi Chu] screen that we've done to date. [Xi Chu] This is a pen differentiation multi-cell type stem cell project. [Xi Chu] So effectively, we differentiate iPSC into 10 different cell types in one [Xi Chu] single experiment without restriction. [Xi Chu] And we did a genome scale perturbation across them. [Xi Chu] So you can imagine, [Xi Chu] instead of just generating 10,000 different biological experiments, [Xi Chu] we did 10,000 by 10 cell types. [Xi Chu] So it's almost a library on library experiment. Why are we doing this? [Xi Chu] We think that in the beginning phase of data collection, as Bo said, [00:32:59] [Xi Chu] I think we're just in the early days of virtual cell building, [Xi Chu] context and diversity and richness of the data matters. [Xi Chu] It's not just the total number of cells or total number of sequencing reads. [Xi Chu] It's about bits per dollar in information content. [Xi Chu] So we want to scale not only in the genetic perturbation landscape, [Xi Chu] but we also want to scale across biological contexts so that we can give our [Xi Chu] AI teams the best rich dataset to build a generalizable model on. [RJ] Is there any thinking about, so you say context, [RJ] but obviously these cells and these experiments have been sort of D, [RJ] I forget the term, but they've been separated from their cohorts, right? [RJ] Is there thinking about using spatial transcriptomics or other sort of [RJ] technologies, [RJ] to build models with perturbations, [RJ] but in the context of the cells that it lives near? [Xi Chu] Great question. So we're thinking about that in a couple of ways. [Xi Chu] Number one, that's actually exactly why we want to build a virtual cell model [00:34:02] [Xi Chu] in the first place. [Xi Chu] You might think that, well, you can already do exhaustive screening in these [Xi Chu] cell lines. [Xi Chu] Why do you still need a model? [Xi Chu] You can just do the experiment and generate the data. [Xi Chu] Certainly, if your query is just about cell biology in cell lines, you're [Xi Chu] right. [Xi Chu] We don't need a model, right? [Xi Chu] At least if for genetic screening, we can just do this experiment. [Xi Chu] But you're also correct that oftentimes good targets, biological insights are [Xi Chu] not about cell lines. [Xi Chu] These are about primary cells, [Xi Chu] about cells in their native physiological context in organs or even [Xi Chu] multi-organ coming together and have some emergent properties. [Xi Chu] A lot of immunological disease are that way. [Xi Chu] You cannot do exhaustive, [Xi Chu] highly put experimentation in animal systems or in organs or in all of these [Xi Chu] complex translational models. [Xi Chu] You can do some experiments, and these are expensive and high-stake. [Xi Chu] The ability to build a model that can be trained on massive data where it is [Xi Chu] possible to scale and be trained in a way that can be fine-tuned and [Xi Chu] transferred to make high-quality causal predictions in these complex models so [00:35:06] [Xi Chu] that we can go into the lab and have the highest quality hypothesis possible [Xi Chu] to validate. [Xi Chu] I think that's the whole point about building a virtual cell model. [Bo Wang] But from AI side, [Bo Wang] I think you're absolutely right that I believe the future virtual cell model [Bo Wang] should be able to incorporate multiple modalities, [Bo Wang] not just RNA expressions. [Bo Wang] Spatial single cell RNA-seq is already a popular technology. [Bo Wang] Even for SEGBT, we actually have an extended version. [Bo Wang] We call it SEGBT spatial. [Bo Wang] That is specifically designed for spatial single cell omics. [Bo Wang] We also have papers on early attempts to try to take the HNE images, trying to [Bo Wang] predict the gene expressions. [Bo Wang] There's already some signals you can find. [Bo Wang] Eventually, [Bo Wang] what I predict is that a virtual cell model will be able to integrate not only [Bo Wang] RNA-seq, [Bo Wang] but can integrate more functionally [Bo Wang] related, for example, [Bo Wang] pruning omics or other regulatory side of omics such as ATAC-seq to overall [00:36:08] [Bo Wang] combine all your descriptive omics data sets to predict the future states of [Bo Wang] the cellular functions. [Bo Wang] I think that's probably the future for virtual cell model. [Brandon] We had Ron Alfa and Dan Baer from Noetic as guests recently. [Brandon] Viewers who want to hear a little bit more about that, I think we go quite in [Brandon] depth there. [Brandon] It's a background, you can go to that. [Brandon] Can you explain a little bit about what spatial transcript omics and spatial [Brandon] proteomics are? [Xi Chu] So maybe a bit of a history lesson here. Before, we had single-cell RNA-seq. [Xi Chu] We had RNA-seq. And before that, we have microarray technologies. [Xi Chu] What RNA-seq and microarray used to do is take a chunk of my tissue, [Xi Chu] grind it all up, put in a blender, imagine, make a smoothie out of it, [Xi Chu] and take all of the RNA from different cells in that piece of tissue and [Xi Chu] measure all of their expression levels. [Xi Chu] It is great. [Xi Chu] For the first time, you can measure gene expression all 20,000 at a time. [Xi Chu] And we used to do them, we used to have to do them one at a time. [00:37:11] [Xi Chu] But it is not great in that we don't know which RNA came from which cell. [Xi Chu] And this is particularly a problem if you're dealing with a multicellular [Xi Chu] piece of tissue. [Xi Chu] You want to attribute RNA to the immune cell, to the skin cell, [Xi Chu] to the fibroblasts, [Xi Chu] but you can't because you want everything up in a smoothie. [Xi Chu] What single-cell technology allow you to do is analyze them cell by cell. [Xi Chu] So now, I can attribute RNA gene expression to the cell that they originate [Xi Chu] from. [Xi Chu] But there's still a problem. [Xi Chu] I don't know spatially where these signals come from. [Xi Chu] And for many disease, it matters, right? [Xi Chu] In immuno-oncology, for example, [Xi Chu] you want to know when T cells are close to a tumor cells or when a T cell is [Xi Chu] not able to penetrate the solid tumor, [Xi Chu] what is the difference between them? [Xi Chu] Or when a T cell is attacking the tumor cell, when a T cell is not, what is [Xi Chu] the difference about that? [Xi Chu] For that, you need spatial information. [Xi Chu] You need to observe cell in situ in their context. [00:38:11] [Xi Chu] And so now there are different technologies that solve that problem. [Xi Chu] Essentially, take that chunk of tissue. I don't have to grind it up anymore. [Xi Chu] I just make a cross-section, lay it down in a piece of slide, [Xi Chu] and I can measure its morphology using standard techniques like H and E [Xi Chu] staining. [Xi Chu] I can then also measure many protein expression using multiplex IF assays, [Xi Chu] immuno-forsense assays. [Xi Chu] Ultimately, [Xi Chu] I can also look at the gene expression up to genome-wide in all of these cells [Xi Chu] in their native spatial coordinates by using some of the latest spatial omics [Xi Chu] assays. [Xi Chu] So you have the x-y coordinates of every cell, but also all of the molecular [Xi Chu] analytes that we talked about earlier. [Xi Chu] And that's an exciting new direction for genomics field. [Bo Wang] Personally, can you imagine that the spatial omics adds more difficulty to AI [Bo Wang] modeling? [Bo Wang] Because instead of looking at the individual cells, [Bo Wang] you have to look at the neighboring niche cells to better kind of learn the [00:39:11] [Bo Wang] representation that is spatially cohesive. [Bo Wang] That is the challenge the current spatial foundation model is facing. [Brandon] But that context is going to be crucial for [Brandon] understanding, let's say, [Brandon] cancer where the interaction of immune cells and cancer cells and non-immune [Brandon] or cancer cells. [Brandon] Yeah, that is absolutely right. [Bo Wang] It's absolutely crucial for getting a complete file of zero. [Bo Wang] You can extract spatial-aware biomarkers to predict some of the clinical [Bo Wang] response. [Bo Wang] I think that would be extremely important to build such models. [RJ] Getting back to Excel, this presumably can inform a spatial model as well, [RJ] right? [RJ] Because you have one cell in one place. [RJ] You can imagine, OK, I can just throw away the coordinates and just do [RJ] inferences on one cell at a time. [RJ] And now I can create a more complicated model that does that, but it also [RJ] knows who its neighbors are or something. [Bo Wang] You're absolutely right. [Bo Wang] But the current version we were releasing, we're not dealing with spatial [Bo Wang] omics. [00:40:12] [Bo Wang] However, [Bo Wang] definitely our ongoing work and the next version of Excel will be able to [Bo Wang] infer the spatial-aware representations for different cells. [RJ] We've talked about the data collection a bit. [RJ] Let's talk about the architecture, get some red meat for [RJ] the AI engineers listening in. [Bo Wang] Sure. [Bo Wang] Let's get to the history of virtual cell modeling, particularly virtual cell [Bo Wang] 2.0. [Bo Wang] I think our SDGB sets the foundation for most of foundation models of single [Bo Wang] cells, [Bo Wang] is that we adopted autoregressive training, [Bo Wang] extremely similar to how chargeability is trained on languages. [Bo Wang] It's next [Bo Wang] token predictions. [Bo Wang] So we mimics the way how chargeability is trained on languages to train the [Bo Wang] single-cell foundation model on cells. [Bo Wang] By doing that, you have to assume inherent order of genes. [Bo Wang] The way we assume the order of genes is by attention mechanism. [Bo Wang] There's many other methods that are using different orders of genes, [00:41:15] [Bo Wang] some as simple as just rank the genes based on the expression values. [Bo Wang] There's also more complicated methods to rank different genes. [Bo Wang] But in hand, you have to assume an order of genes. [RJ] And just to be clear, when you talk about genes, those are intrinsically [RJ] ordered. [RJ] They're a sentence spelled out in ATGC. [RJ] So genes themselves have the nucleotides and there's a long chain, [RJ] and that makes a lot of sense to have an order to them. [RJ] But what we're talking about is something different. [RJ] That's the expression data, the expression levels. [RJ] So the expression level means how it's just a count for each gene of how many [RJ] of these genes did I see when I was measuring. [Bo Wang] In sequences, the order of ATGC make total sense to us. [Bo Wang] But for expression data, they're literally just matrices. [Bo Wang] So it's really hard to assume [Bo Wang] an inherent order of genes. You might shuffle the order of genes. [00:42:15] [Bo Wang] I think the biology doesn't change much. [Bo Wang] However, because of the way the language model is trained, everybody has [Bo Wang] preset tricks to train such models. [Bo Wang] So it's easy to adopt. [Bo Wang] That's how all the foundation models are started for single-cell. [Bo Wang] And then I quickly realized that with diffusion language models, we actually [Bo Wang] don't need to assume the order of genes. [Bo Wang] Instead, we can have a bidirectional diffusion process to generate such long, [Bo Wang] high-dimensional gene expression datasets. [Bo Wang] So just to think about it, what's the difference between autoregressive [Bo Wang] training versus diffusion language models? [Bo Wang] Is that you can think of autoregressive training as typing. [Bo Wang] For example, I like coffee. [Bo Wang] You have to type I and then I like and then coffee. That's inherent orders. [Bo Wang] But diffusion language model, you can treat it as editing. [Bo Wang] You iteratively generate a sentence from a very vague, very rough sentence. [Bo Wang] And then you can iteratively refine it. So same thing with gene expressions. [00:43:19] [Bo Wang] You can generate a very rough representation of the gene expressions and then [Bo Wang] iteratively from noisy representation to more refined representations. [Bo Wang] So you iteratively edit the gene expression predictions until it minimizes the [Bo Wang] losses. [Bo Wang] So this is a very different philosophy to generatively predict the response [Bo Wang] after perturbation. [Bo Wang] And it turns out it actually fits more to a single-cell RNA-seq. [Bo Wang] So that's why we switched it from a scGPT-like model to the current X-Cell [Bo Wang] model, [Bo Wang] which is using diffusion language models. [Brandon] When I think of transformers, they're fundamentally objects that operate on [Brandon] sets. [Brandon] The community spends a lot of time trying to make them things which have some [Brandon] sort of causal ordering to them. [Brandon] But if you just naively take your transformer, it's a set operation. [Brandon] So given that, why think about this in terms of diffusion or autoregressive [Brandon] LLMs? [Brandon] Why not have your initial prediction strategy be something like take just a [00:44:22] [Brandon] set of genes, [Brandon] each of which has its own cotton-coated identity, [Brandon] and then use that as sort of a prediction? [Brandon] That seems like a much more natural architecture to me. [Brandon] And it's not just your work. A lot of people work on things like this. [Brandon] And I have been somewhat confused why there's this bias in the community about [Brandon] this. [Bo Wang] So I think what you were referring to is more related to representation [Bo Wang] learning, [Bo Wang] where you can take sets of genes and try to project to low-dimensional latent [Bo Wang] space. [Bo Wang] But what we care about for building generative modeling for virtual cell, [Bo Wang] because you want to predict the dynamics of cells. [Bo Wang] So we want to have a generative model. [Bo Wang] So that's why we're mostly using decoder-only architectures in order to [Bo Wang] generate the full transcriptomics instead of just predict a predefined small [Bo Wang] set of genes, [Bo Wang] because you want to model the whole gene regulatory networks, [Bo Wang] which are extremely kind of high-dimensional, right? [RJ] So just to be clear, input is genes plus a perturbation. [00:45:23] [RJ] Output is new gene expression levels. [RJ] Is that gene expression levels plus perturbation is input output? [RJ] Plus time cells, normal cells. [Bo Wang] Like for each cell? Correct. Yeah. That is correct. Right. [RJ] And so the way that I think about diffusion language models, [RJ] and you can correct me here because I don't know a lot about them, [RJ] but the way I think about them is they're like BERT, [RJ] but you do it over and over again. [RJ] Is that kind of a good thing to think about? [Bo Wang] Yeah, that is a rough understanding of how diffusion language model works. [Bo Wang] Yeah. [RJ] So you just apply the diffusion. [RJ] The diffusion process is like basically unmasking or editing once over and [RJ] over again. [RJ] The similar to how like an image diffusion model kind of refines the image [RJ] over and over again. [RJ] In this case, I'm using BERT. So it is a transformer, basically. [RJ] It is a transformer. [RJ] But it is like repeatedly updating the sort of sentence in this case, [00:46:24] [RJ] which is a bunch of expression levels over and over again. [Bo Wang] That is correct. [Bo Wang] Actually, in our paper, [Bo Wang] we show that as the number of diffusion steps goes on, [Bo Wang] the loss function keeps decreasing, [Bo Wang] the fitness of the partition to the ground should keep increasing. [Bo Wang] So this which means the model start to understand how iteratively refine the [Bo Wang] predictions. [Bo Wang] I see. [RJ] We're talking about diffusion versus autoregressive. [RJ] I noticed in the paper, there's a bunch of discussion of preconditioning using [RJ] a whole bunch of stuff. [RJ] Can you want to talk a little bit about that? [Bo Wang] Another major innovation we made in Excel is the way we incorporate prior [Bo Wang] knowledge into the model. [Bo Wang] So incorporating biological priors has always been a good idea in biology in [Bo Wang] general, [Bo Wang] because biologists spend decades to understand some of the biologists already. [Bo Wang] How do we tell the model some of the prior knowledge, some metadata about the [00:47:26] [Bo Wang] cells? [Bo Wang] Before Excel, [Bo Wang] what people do is they try to incorporate a single type of priors, [Bo Wang] for example, gear, [Bo Wang] using gene-regular network as a prior to predict the perturbations. [Bo Wang] So, as each video sometimes trying to incorporate PBI as prior as well. [Bo Wang] Excel, to my knowledge, [Bo Wang] is one of the first models in trying to incorporate extremely diverse sets of [Bo Wang] biological priors. [Bo Wang] So in our preprint, we incorporate five types of priors, [Bo Wang] including literatures, as simple as just ask chat video, [Bo Wang] tell me everything about this gene, [Bo Wang] and then we embed the output as the embedding. [Bo Wang] So it's like a gene PT. Exactly. That's a gene PT. [Bo Wang] And we also incorporate PBI, protein-protein interaction networks. [Bo Wang] We also incorporate DevMap, which is cancer-related essential gene [Bo Wang] information, morphology information. [Bo Wang] We even try to incorporate acid-gbt embeddings, which is basically cell types. [00:48:28] [Bo Wang] So with a set of prior knowledge as conditions to the model, [Bo Wang] the model start to have more accuracy in terms of context-specific [Bo Wang] predictions. [Bo Wang] And what's more interesting to us is that by looking at the weights of [Bo Wang] different priors, [Bo Wang] we can actually understand which prior knowledge are more important to these [Bo Wang] particular cell types. [Bo Wang] So it adds more interpretability to the models. [Bo Wang] So we find that combining diffusion language model plus a very diverse set of [Bo Wang] prior knowledges, [Bo Wang] Excel does much better in generalizing to unseen contexts. [Bo Wang] So this is some of the AI innovations we made for Excel. [RJ] Do you now need to provide all of that context in order for the model to work, [RJ] or those are like preconditioning that it can also do without if you want? [Bo Wang] We don't need to incorporate these prior knowledge anymore because these are [Bo Wang] already learnable parameters inside the models. [00:49:28] [Bo Wang] However, what you suggest is more prompt or in-context learning for virtual [Bo Wang] cells. [Bo Wang] We can do that as well, [Bo Wang] basically by adding more conditions into the prior knowledges so that to [Bo Wang] prompt the model to predict towards certain directions. [RJ] In other words, [RJ] the model now takes advantage of the learning using the priors that you [RJ] provided during training and doesn't need them, [RJ] but has some advantage because you provide them during training, [RJ] but you can even get more advantage if you are able to provide those priors [RJ] during inference. [Brandon] How [Brandon] much does that matter? [Brandon] Whenever I see big machine learning papers with tons of things thrown in, [Brandon] I'm always wondering where's the big alpha and where's the little alpha? [Brandon] Are these adding this little bit of incremental performance boost, [Brandon] or are all these actually crucial to generalization? [Bo Wang] There's multiple factors we have to consider. [Bo Wang] How much contribution the data contributed? [Bo Wang] How much of the AI architectures contributed? [00:50:31] [Bo Wang] Even for the architecture, what's delta from switching to autoregressive [Bo Wang] training to diffusion? [Bo Wang] What's delta from the prior knowledges? [Bo Wang] Certainly, all of these needs very specific ablation studies. [Bo Wang] From empirical experiments, the quality, the amount of the data sets matter [Bo Wang] the most. [Bo Wang] This is why we were extremely excited to publish the Pisces data sets, [Bo Wang] which has 16 different cell types across 25 million cells. [Bo Wang] It's genome-wide. [Bo Wang] It has a huge tensor, if you really think about it from a computer [Bo Wang] perspective. [Bo Wang] Genome-wide perturbation, genome-wide transcriptomics plus number of cells [Bo Wang] plus times the number of conditions. [Bo Wang] It's massive tensors. [Bo Wang] Because of the poor screening technology, we don't have batch effects. [Bo Wang] You don't need the model to climb the heel of batch effect. [00:51:31] [Bo Wang] So we find that trained on perturbation data sets, [Bo Wang] high-quality perturbation data sets, already gives a big boost to the models. [Bo Wang] We also did ablation that if we train all the virtual cell models out there, [Bo Wang] including state cell to send us original SDG video on the same data sets, [Bo Wang] what's the delta we are observing? [Bo Wang] We reported the results there as well. [Bo Wang] We find that switching from autoregressive training to diffusion-language [Bo Wang] models give a significant improvement over some of the harder tasks, [Bo Wang] particularly generalized to on-scene tasks. [Bo Wang] And the prior knowledge, more or less condition-specific, [Bo Wang] for certain cell types, some of the prior knowledge makes a huge difference. [Bo Wang] But for certain cell types, the delta seems to be marginal. [Bo Wang] We are thinking about how to better incorporate the prior knowledge. [00:52:31] [Bo Wang] We still believe that [Bo Wang] letting the model know a big chunk of existing biology should be helpful. [Bo Wang] But maybe it's the way we incorporate the prior knowledge through [Bo Wang] cross-attention, limited the scope of the metadata. [Bo Wang] But I think it's certainly a research topic. [Bo Wang] But overall, if we have to give an order, [Bo Wang] my order would be the quality among scale of the data sets and then the [Bo Wang] architecture and then the prior knowledge. [Bo Wang] Personally, this only applies to our architecture. [Bo Wang] I'm sure there's different choices of architecture and have different ranks of [Bo Wang] contributions. [RJ] First of all, this is a really fascinating, very cool model. [RJ] I hope everyone has a chance to look at the paper. [RJ] There's obviously a lot of resources that were put into doing this. [RJ] I don't know if you guys can disclose how much. [RJ] It's a lot of money, whatever it was. [RJ] Operating wet lab, probably very complicated training runs. [RJ] I think there's a couple four billion parameter model. Is that right? [00:53:33] [RJ] Four point nine billion. Yeah, four or five billion parameter model. [RJ] So much larger model probably took a lot of GPUs to train. [RJ] What's the lift that you get from this effort versus let's just put the money [RJ] into wet lab work and the sort of traditional pipeline that [RJ] basically has been the status quo up until now. [Xi Chu] Biology is a multi-scale [Xi Chu] discipline. [Xi Chu] There are cells, there are DNA sequences on the most fundamental level. [Xi Chu] There are cells, there are multi-cellular pieces of tissues, co-cultures. [Xi Chu] You have tissues, you have animal systems and finally you have human. [Xi Chu] I think we would like to be able to do causal protection towards the right of [Xi Chu] the spectrum. [Xi Chu] Ultimately, do call the prediction in human, [Xi Chu] know what drugs will work in which patients, [Xi Chu] but that's very difficult to collect high-throughput data on. [Xi Chu] The whole vision of virtual cell is to generate data where it is possible so [00:54:34] [Xi Chu] that we can transfer the causality protection towards the right, [Xi Chu] towards the more translational, the more complex systems. [Xi Chu] Certainly you can mine the data already. [Xi Chu] We generated a lot of data as Bo said, seven screens, 16 different biological [Xi Chu] context, genome scale perturbation. [Xi Chu] There's a lot of good ideas in that already. [Xi Chu] There is a figure that we put out in the preprint that just looking to [Xi Chu] inactivation of T cells. [Xi Chu] We already saw some, we saw known biology, TCR complex. [Xi Chu] We also saw some palliative new biology, which we're very excited to validate [Xi Chu] in the lab. [Xi Chu] Some of that were actually also called out in a very recent screen last [Xi Chu] December published from Alex Morrison lab, [Xi Chu] also in the Bay area. [Xi Chu] Very excited to see that. But the hope is to not just mine the existing data. [Xi Chu] The hope is that the model can generalize and we will be able to do in silical [Xi Chu] experiment into the future. [Xi Chu] Nobody knows before how much data and what kind of data are needed to do that [00:55:38] [Xi Chu] with the whole field is waiting for the demonstration that the model can be [Xi Chu] linear baseline in perturbation prediction and it can generalize out context, [Xi Chu] not just within the cell line you have training data on, [Xi Chu] but out of that context. [Xi Chu] That's where that's why you need a model. [Xi Chu] So what's very exciting for us is that in this preprint, we saw that [Xi Chu] generalization capability. [Xi Chu] A few demonstrations. We first did in T cells. [Xi Chu] We actually generated the data expressly for this purpose. [Xi Chu] We generated the resting T cell perturbation screen. [Xi Chu] So these are T cells in their baseline condition, not activated. [Xi Chu] And now we have an activated T cell perturbs. [RJ] So just T cell activation means I'm going, I'm trying to kill something. [Xi Chu] These are regulatory T cells, but yes, we activate their receptor so that [Xi Chu] they're starting to proliferate. [Xi Chu] They become more active. They can do their physical job. [Xi Chu] And we only train the model on the resting T cell. [00:56:39] [Xi Chu] And we told the model, Hey, [Xi Chu] this is how the active T cell look like now go and predict what all of the [Xi Chu] part of which are going to do in this active T cell. [Xi Chu] And the model have not seen how perturbation working active T cells. [Xi Chu] And we set up a couple of rigorous tests. [Xi Chu] One linear baseline took the perturbation delta in the resting case, [Xi Chu] just transpose that linearly onto the active T cell. [Xi Chu] And that's our linear baseline. [Xi Chu] Essentially, think about this as a combinatorial perturbation prediction [Xi Chu] problem. [Xi Chu] One of the perturbation is activation of the cell. [Xi Chu] The other is all of the genome wide perturbations. [Xi Chu] Can I just linearly add the two effects together? [Xi Chu] That would be a linear baseline. [Xi Chu] And second, we apply other models from the field. [Xi Chu] And last, but we applied X-Cell critically, X-Cell has not seen active cell T [Xi Chu] cells. [Xi Chu] And it's able to make accurate prediction, known biology, the TCR complex, [Xi Chu] predicting their effect accurately that these are going to inactivate the T [00:57:42] [Xi Chu] cells, [Xi Chu] which is exactly what we would expect to see. [Xi Chu] But also it predicted the putative T cell inactivators that we found in the [Xi Chu] screen correctly as well. [Xi Chu] So that's very exciting to us. [Xi Chu] And that suggests the possibility that one might be able to use these virtual [Xi Chu] cell models completely out of context in the unseen context and predict new [Xi Chu] biology. [Xi Chu] And so we're very excited to follow up on those heads and validate them in the [Xi Chu] lab. [Xi Chu] Just very briefly, a couple other cases that we saw exciting generalizing [Xi Chu] capability of this model. [Xi Chu] Remember, we did a multi-cell type differentiated iPSC experiment. [Xi Chu] There we specifically held out one cell type from training. [Xi Chu] So the model has not seen that cell type. [Xi Chu] Train on the other cell types, as well as the rest of the datasets. [Xi Chu] The model made very good prediction across thousands of genes, thousands of [Xi Chu] perturbations in that unseen cell type. [Xi Chu] So again, it suggests the model's ability to generalize out of cell type. [00:58:44] [Xi Chu] And the last experiment, I think we're very excited, [Xi Chu] is that we trained this on T cell cell line, [Xi Chu] but there was just very recently a primary T cell perturb-seq published from [Xi Chu] Alex Marson's lab. [Xi Chu] That's an impressive amount of work. [Xi Chu] It's not easy to do this scale screening in primary cells. [Xi Chu] Very few labs have that kind of capabilities. [Xi Chu] It's much easier to do that in T cell lines. [Xi Chu] Again, the model is able to generalize out of cell lines into primary cells [Xi Chu] and make accurate predictions there. [RJ] So they actually perturbed primary cells, not cell lines? [Xi Chu] Primary T cells harvested from donors. [Xi Chu] And we were able, from multiple donors, [Xi Chu] and the Excel trained on just one T cell cell line is able to make predictions [Xi Chu] across multiple donors from primary T cell experiments. [RJ] This is a validation of the whole theory, right? [RJ] That you can train on these slightly weird cells, [RJ] and that it will be good because you're covering the domain well enough or [RJ] whatever it is, [RJ] that you're able to actually predict in real cells that come directly from [00:59:47] [RJ] real people. [RJ] That's right. [Bo Wang] Yeah. [Bo Wang] I think building virtual cells is not to replace biological experiments, as [Bo Wang] you mentioned. [Bo Wang] What are we trying to do? [Bo Wang] Really, [Bo Wang] the holy grail of virtual cell is to have a model to generalize to on-scene [Bo Wang] context that is harder or even impossible to conduct biological experiments [Bo Wang] on, [Bo Wang] right? [Bo Wang] So far, Excel focusing on cell lines. [Bo Wang] And eventually, we want to extend to more complicated biological systems such [Bo Wang] as animals, organoids. [Bo Wang] And eventually, before, to patients, to human biology, right? [Bo Wang] And bearing a few numbers, 90% of disease has no cure. [Bo Wang] And most of the drug failed at the phase three clinical trials on patients. [Bo Wang] And the success rate of phase three trials is as low as five to 10%. [Bo Wang] And phase three means? The final stage on the patient trials. [RJ] So that's when you generalize from toxicity in phase two to efficacy in phase [01:00:50] [RJ] three. [RJ] From a small cohort. [RJ] Oh, sorry. Toxicity is one. [RJ] And so a lot of large cohort. [RJ] So the generalization problem, okay, this drug, I've very carefully selected [RJ] my patients, and it works pretty well. [RJ] And now I get a bunch more patients, and suddenly, it doesn't work very well. [RJ] And that's the big problem that you're... [Bo Wang] The promise of virtual cell is that can we build such a model that learn all [Bo Wang] the causality of biology so that can be grounded to predict the response [Bo Wang] eventually on patients so that we can, [Bo Wang] for certain drugs, [Bo Wang] we can select the right patients to conduct the clinical trials on. [Bo Wang] So this is a long-term but we already see some early hopes that Excel trained [Bo Wang] on diverse set of causal datasets can already generalize to some unseen cell [Bo Wang] types. [Bo Wang] So certainly there's a lot of experiments to be done to validate this model. [01:01:50] [Bo Wang] So even continuously finding this model, but I think that we certainly see [Bo Wang] some early hopes. [Brandon] So you were talking about linear models and this brings up this famous or [Brandon] infamous arc challenge about perturbation. [Brandon] And there's been this theme about complicated foundation models oftentimes not [Brandon] beating linear baselines. [Brandon] And I'd like to get your take about that as terms of, [Brandon] is this... First of all, is this different? [Brandon] I think some of maybe your own models might also have had trouble beating [Brandon] linear baselines in the past. [Brandon] Is there something different about your current data strategy or where you're [Brandon] going and where's the field going? [Brandon] And what is the role of foundation models versus, or in virtual cell models [Brandon] versus these simple baselines? [Bo Wang] Yeah, a few things. [Bo Wang] First of all, those benchmarks, as you mentioned, are conducted on replogor [Bo Wang] datasets, which are very small datasets. [Bo Wang] And the metrics people report are mostly MAE. [01:02:53] [Bo Wang] Certainly you can imagine if the... [Bo Wang] Because single cell data set are so sparse, the average profiles of the cells, [Bo Wang] certainly you can imagine is a great [Bo Wang] local optimum to minimize the MAEs. [Bo Wang] This is why sometimes the average profile of cells has lower MAEs even than [Bo Wang] technical replicates, [Bo Wang] which are considered off-ground truth for perturbation experiments. [Bo Wang] So that itself shows that that metric is not reliable. [Bo Wang] However, [Bo Wang] most of these benchmarks are still comparing foundation models that trained on [Bo Wang] static expression datasets, [Bo Wang] such as cell by genes. [Bo Wang] SGV is often benchmarked against internally. [Bo Wang] We also find that when it comes to MAE, [Bo Wang] sometimes SGV kind of fail to outperform linear models, [Bo Wang] just because of the reasons I just stated. [Bo Wang] And what sets Excel different from these static expression models, [Bo Wang] such as SGV or geneformers, is that we actually, [01:03:56] [Bo Wang] instead of training on gene expression datasets, we train on causal datasets. [Bo Wang] We train on massive amount of genome-wide perturbation datasets, [Bo Wang] so that it learns better about the dynamics of the interventions. [Bo Wang] And in our preprint, we extensively compare with linear model as well. [Bo Wang] And as you mentioned, linear model totally failed to extend to unseen cell [Bo Wang] types. [Bo Wang] You can quickly imagine why. [Bo Wang] And I believe that foundation model or other more complicated AI models that [Bo Wang] train on the right data will outperform these linear models in harder tasks, [Bo Wang] particularly in generalization tasks. [Bo Wang] And that's why I keep mentioning the right dataset with the right AI model [Bo Wang] will lead to huge improvements. [Bo Wang] But I think the field still needs to see more biological validations to be [Bo Wang] more convincing than the virtual cell direction is the right one. [Xi Chu] Yeah, I think the field suffer from a lack of consistent and uniformly [01:04:59] [Xi Chu] accepted benchmarks. [Xi Chu] What gets measured will get improved. [Xi Chu] And in our paper, we measured, [Xi Chu] I think one of the metrics that we put a lot of thought into and saw the model [Xi Chu] really shine is metrics around gene expression changes. [Xi Chu] So in Pearson Delta, the similarity between predicted changes and ground truth [Xi Chu] changes upon a perturbation. [Xi Chu] That's very hard to cheat on. You have to really get the changes right. [Xi Chu] And what really blew my mind away is when I saw the model make prediction, [Xi Chu] just print out the heat map of the gene expression changes, [Xi Chu] look at the actual raw data and line up the linear baseline prediction, [Xi Chu] the ground truth and the XR prediction altogether. [Xi Chu] It's visually very clear to see that XR prediction is very much more similar [Xi Chu] to ground truth than the linear baseline. [Bo Wang] This is a wow moment I was talking about in the beginning. [Xi Chu] And it's not hard to understand why when we put, [Xi Chu] so this is the first time that someone can put together not just one perturbs, [01:06:01] [Xi Chu] but seven genome-wide perturbs campaigns together. [Xi Chu] Something that jumped out to us, [Xi Chu] biologists right away is that some of the perturbations are context universal, [Xi Chu] meaning that the genes do the same thing in regards to the cell types you [Xi Chu] experiment in might not be surprising to you that these are your housekeeping [Xi Chu] genes, [Xi Chu] right? [Xi Chu] Of course, they do the same thing every cell. [Xi Chu] And then there are all of these other clusters of genes that have very context [Xi Chu] specific functions, [Xi Chu] they do different things in different cells. [Xi Chu] Again, not hard to imagine why in iPSC in our stem cells, [Xi Chu] we saw developmentally relevant genes, [Xi Chu] that are important for neural differentiation. [Xi Chu] They only light up in iPSC experiments, of course, right? That makes sense. [Xi Chu] So if you think about biology, [Xi Chu] it's so complicated that you have to capture these, you know, [Xi Chu] context universal perturbation effects. [Xi Chu] You also have to somehow learn the context dependent perturbation effects. [Xi Chu] It's not hard to then see why a very sophisticated nonlinear model is able to [01:07:02] [Xi Chu] capture and learn all of those biology much better. [Brandon] Are your perturbations always single gene perturbations? Or do you have more? [Brandon] Because my understanding of regulatory networks is oftentimes, [Brandon] sometimes it can be a single gene does a ton of things. [Brandon] For example, [Brandon] I think males are just differentiated due to one gene being enabled at like [Brandon] day seven of embryo development or something. [Brandon] And that differentiates everything is this one gene. [Brandon] But then sometimes you have large networks of genes, [Brandon] which all are very redundant, [Brandon] which allows for more subtle feedback mechanisms and so on. [Brandon] So I could imagine a lot of single gene perturbations as being kind of [Brandon] irrelevant. [Brandon] And that you might want to start having a more commentorial strategy here. [Xi Chu] Yeah, that's a great question. And Bo and I have thought about this a lot. [Xi Chu] Actually, it's interesting that you brought up reproductive biology in, [Xi Chu] I studied a lot of in female cells, the compensation, [Xi Chu] the dosage compensation mechanisms. [01:08:03] [Xi Chu] There are female cells have two X chromosomes, [Xi Chu] male cells have one X chromosome to match the dosage output from the X [Xi Chu] chromosomes strategy that the mammalian cells employ is one gene that produce [Xi Chu] a RNA that does not encode for any protein, [Xi Chu] just a non-coding RNA. [Xi Chu] That RNA wraps around one of the female X and turns down most of the gene [Xi Chu] expression from that chromosome, [Xi Chu] shove it away in the corner of the nucleus and it's called the bar body. [Xi Chu] It is never heard from again. [Xi Chu] And so I absolutely agree with you, one gene can do a lot. [Xi Chu] But in biology, you also have redundancy, you have compensation, [Xi Chu] you have all kinds of mechanisms where knocking down one gene is not [Xi Chu] sufficient to always see a phenotype. [Xi Chu] What if four genes redone only do the same thing, right? [Xi Chu] Taking out the RNA is not going to be sufficient. [Xi Chu] So where we started with one cell type at a time, loss of function, [Xi Chu] single gene perturbation, and look at only RNA expression as the output, [01:09:04] [Xi Chu] we're expanding the platform along all of those axes. [Xi Chu] So that's what we do today to build a scaffold of the data for training models [Xi Chu] like Excel. [Xi Chu] We are now beginning to grow in all three axes of the platform, [Xi Chu] going beyond transcriptomics alone to like a multimodal data, [Xi Chu] going beyond just one gene perturbation alone to look at also pathway [Xi Chu] activation and inactivation, [Xi Chu] turning on and off entire cascade of chain [Xi Chu] reactions. [Xi Chu] And also going beyond just cell lines, [Xi Chu] monocultures into more and more complex translationally relevant systems into [Xi Chu] primary cells, [Xi Chu] into organoids, doing even direct in vivo perturbation swings. [Xi Chu] So we believe with all of that expansion, that data will be all the more [Xi Chu] exciting to train models on. [Bo Wang] This is also why we incorporate PPI networks as the prior knowledge into our [Bo Wang] model. [Bo Wang] And although the model right now are trained on single gene perturbations, [Bo Wang] but once the model is trained, [Bo Wang] you can actually predict combinatorial perturbations just on the model in only [01:10:08] [Bo Wang] silico, [Bo Wang] right? [Bo Wang] So in the sense that we can just perturb the tokens of two genes at the same [Bo Wang] time and see what's the response. [Bo Wang] Certainly without training the actual combinatorial perturbation data sense, [Bo Wang] the accuracy may not be there, but at least with the existence of such models, [Bo Wang] we can start to generate hypotheses using in silico perturbations. [RJ] How do you see the role of the scientists changing in the age of AI? [RJ] And you may, because you're not using, [RJ] your focus is not language models themselves and agentic science and things [RJ] like that, [RJ] then you may have a different, slightly different take. [RJ] You're building these very specific models, but still one, [RJ] how are you and your students able to maintain such a high pace? [RJ] And I suspect it may have something to do with generated by AI partly, [RJ] but also, and how do you see the role of the scientists, [RJ] the academic changing? [Bo Wang] Yeah, that's a great question. [Bo Wang] So my official split of time is 80% on the era, [01:11:10] [Bo Wang] 20% on my university affiliations, [Bo Wang] but turns out the reality is 100% on the era, 100% on the university. [Brandon] You invented a time machine, that's the answer. [Bo Wang] So the way I try to keep up is something we, [Bo Wang] my lab use lots of agentic AI trying to monitor all the AI papers every day. [Bo Wang] And which every week we have lab meetings, [Bo Wang] we're trying to discuss different topics in AI for biology, AI for healthcare, [Bo Wang] et cetera. [Bo Wang] And it's, to be very frank, even as a professor, I find it's extremely hard to [Bo Wang] catch up. [Bo Wang] The pace of AI is just so incredibly fast. [Bo Wang] And to the point that sometimes I feel anxiety, waking up says, "Oh my God, [Bo Wang] so many people published and what happened to our existing on-purpose work." [Bo Wang] And suddenly, [Bo Wang] you can imagine students probably face 10X anxiety. [01:12:10] [Bo Wang] So sometimes I try to encourage students to really kind of using different [Bo Wang] tools, [Bo Wang] trying to stay focused, and find the niche areas that we become experts on. [Bo Wang] But with the era of generative AI, now agentic AI, I find the way people, [Bo Wang] at least academics, does science are extremely different now. [Bo Wang] Overall, [Bo Wang] most of professors or students in academics start to be very struggling in [Bo Wang] terms of fundings, [Bo Wang] in terms of the pace of publications. [Bo Wang] And that's why you can see lots of major breakthroughs come from industry, [Bo Wang] like Alpha-Fold, for example. [Bo Wang] So how academics survive or even thrive at such era of agentic AI is certainly [Bo Wang] something everybody is thinking about. [Bo Wang] We see lots of faculties left university and join industry [Bo Wang] simply for the reasons of resources. [01:13:12] [Bo Wang] If you are doing research on AI, do you have enough GPUs at your school? [Bo Wang] It's the first question you should ask when a student joined a professor's [Bo Wang] lab. [Bo Wang] The first question they often ask is, [Bo Wang] "How many GPUs do you have?" So suddenly, in that sense, [Bo Wang] industry has a major advantage over academic labs. [Bo Wang] But I think what academics labs have advantages on is really the pace of [Bo Wang] innovations and also the niche areas of this specific academic lab can be [Bo Wang] extremely expert on. [Bo Wang] So also having the freedom of thinking sometimes also makes you easier to [Bo Wang] innovate on ideas that maybe industry people didn't even think about. [Bo Wang] Overall, [Bo Wang] I think the whole field needs to be a lot more innovative to catch up and [Bo Wang] hopefully with the help of different tools and hopefully the government starts [01:14:12] [Bo Wang] to invest more into academia because I still deeply believe that academic is [Bo Wang] the main source of innovation for the whole field, [Bo Wang] and particularly when it comes to biotech. [Bo Wang] So hopefully we see more investment into academia so that we stay afloat. [RJ] I agree with you, but why? [RJ] Why should the money just go to industry? [RJ] Why should the government put any money into academia? [Bo Wang] I still believe the power of academic freedom. [Bo Wang] This is actually the original motivation we have, [Bo Wang] the existence of academic professors who are the only we teach, [Bo Wang] and there's also benefits of teaching and research at the same time in the [Bo Wang] sense that when you teach a subject, [Bo Wang] you actually have to become the experts and that force you to keep updating [Bo Wang] your knowledge base and then find easy ways to convey your knowledge to [Bo Wang] students. [Bo Wang] And by doing that, you actually start to innovate on different ideas. And [01:15:18] [Bo Wang] myself, for example, I teach a big class at the University of Toronto about [Bo Wang] deep learning and neural networks. [Bo Wang] It's a gigantic class with 600 students every year. [Bo Wang] And by teaching that, it forces me to update the slides, lectures every year, [Bo Wang] and myself reading lots of materials, [Bo Wang] trying to update myself so that I can find ways to convey some of the [Bo Wang] knowledge to students. [Bo Wang] So that's how I keep updating myself whenever there's transfer. [Bo Wang] I remember vividly we have GPT-2, we updated the lectures, [Bo Wang] and then different language models, [Bo Wang] how the multi-thread GPU communication is used in training, [Bo Wang] largest scale neural networks, et cetera. [Bo Wang] So I force myself just update the models. [Bo Wang] And by talking to different students, [Bo Wang] we really generate lots of novel ideas to apply the cutting-edge AI models to [Bo Wang] very specific niche areas in biology or in healthcare. [01:16:18] [Bo Wang] Maybe it's unique to Canadian academic system. [Bo Wang] By being a professor in academic, [Bo Wang] we also have access to lots of healthcare data sets, [Bo Wang] which are very hard for industry to access due to many illegal reasons or [Bo Wang] regulatory reasons. [Bo Wang] And that's why you see some of the papers we published through academic [Bo Wang] hospitals in Canada, [Bo Wang] where we developed some of the state of arts foundation model for ultrasound [Bo Wang] images. [Bo Wang] So that's what I mean that there's certain level of freedom of academic [Bo Wang] thinking that really drives lots of innovations. [Bo Wang] And I still believe, maybe it's biased, [Bo Wang] but myself still believe that having certain level of freedom of academic [Bo Wang] thinking will lead to lots of kind of innovations that is unthinkable in [Bo Wang] industries. [Xi Chu] I 100% agree with Bo there. So just thinking about the lab [01:17:18] [Xi Chu] workflow that we do, a lot of these are building upon innovations that were [Xi Chu] first pioneered in academia as well. [Xi Chu] CRISPR, of course, was discovered in academia. [Xi Chu] CRISPR applied to mammalian high-throughput screening, also demonstrating [Xi Chu] academia first. [Xi Chu] scRNA-seq, this kind of drop-up encapsulated scRNA-seq first demonstrated in [Xi Chu] academia. [Xi Chu] Then different companies tried to build it up into commercial offerings. [Xi Chu] Putting all of these together to do Perturb-seq at first, also pioneered in [Xi Chu] academia, right? [Xi Chu] CRISPR lab Jonathan Weissman's lab, Aviv Regev's lab, many of the pioneers in [Xi Chu] academia. [Xi Chu] And then I think these, especially on the lab side, [Xi Chu] these innovations take so long, [Xi Chu] and the discovery process can be so accidental, right? [Xi Chu] That it's perhaps not ideal for a pure industry to take on. [Xi Chu] But once they show early promise, scaling them and robustifying them, [Xi Chu] and generating data that's not only massive but high quality, [01:18:20] [Xi Chu] especially for AI scale, [Xi Chu] I think that's something that can be very well done in industry, [Xi Chu] both the mindset, as well as the kind of the resources that we can support. [Xi Chu] That sometimes can be hard for academia labs to match. [Brandon] Xaira has been very generous with releasing your data sets and your models. [Brandon] You've been, seem to be very committed to open science. [Brandon] Given some of the things you were just talking about, [Brandon] and this discussion about what is the best, [Brandon] most important data strategies for virtual cells, [Brandon] or understanding human biology, where do you think academia should go next, [Brandon] now and next? [Brandon] Since Xaira has probably a budget comparable to [Brandon] probably dozens or hundreds of bio labs right now, [Brandon] what do you think that if you are an academic, a professor, [Brandon] especially in a wet lab, what would you want to be focusing on? [Bo Wang] First of all, myself is a deep believer of open science. [Bo Wang] That's why all the models we talk about here, that are open source. [01:19:23] [Bo Wang] You can find all the data weights from my lab GitHub. It's very thorough. [Bo Wang] Yes, thank you. [Bo Wang] The reason I believe open science is that, as you mentioned, [Bo Wang] most of the time, academic lab, we started an idea and we prototype it. [Bo Wang] It's not scalable, it's not even a good product, and industry can take it to [Bo Wang] scale things up. [Bo Wang] This is also why SDGPD quickly becomes one of the most widely used single cell [Bo Wang] foundation model in pharma companies. [Bo Wang] That's very encouraging to us. [Bo Wang] This is why Xaira also started to open source some of the data sets, some of [Bo Wang] the models. [Bo Wang] Part of the reason is that we believe virtual cell is such an early field, [Bo Wang] and it doesn't help to withhold certain data sets or certain models because [Bo Wang] it's so early. [Bo Wang] A better win-win situation is everybody gets to in this field start to [Bo Wang] contribute [Bo Wang] models together to exchange ideas so that this field can move forward in a [01:20:23] [Bo Wang] much faster pace. [Bo Wang] We see successful examples in protein space because of the availability of [Bo Wang] open source data sets in PDBs. [Bo Wang] Therefore, we have models such as alpha fold, rosetta fold. [Bo Wang] Again, alpha fold is also open source. [Bo Wang] Therefore, we can quickly iterate different models. [Bo Wang] That's why you see a booming situation in protein space. [Bo Wang] We want to do the same thing for virtual cells. [Bo Wang] Let's put all the data sets together. [Bo Wang] Let's have the same standard protocols to generate high-quality data sets. [Bo Wang] Let's put all the resources together to generate next generation of virtual [Bo Wang] cell models. [Bo Wang] When it comes to academic labs, then you can comment on what lab, how what lab [Bo Wang] academics can survive. [Bo Wang] From Joy Lab's perspective, I do encourage all the [Bo Wang] Joy Lab AI researchers in universities start to collaborate with industries so [Bo Wang] that they can get more resources to develop their own ideas. [Bo Wang] With the era of HNT AI, now everybody can code. [01:21:26] [Bo Wang] It's more important to have a right taste about your project so that you don't [Bo Wang] just [Bo Wang] burn tokens without purpose. [Bo Wang] We want more academic students, [Bo Wang] academic professors to have a taste of research so that we make the right [Bo Wang] utility of HNT AIs. [Brandon] As a professor, your job is to have taste, obviously, but as a student, [Brandon] how do you develop taste as in a world where so much of the thinking [Brandon] scientific process could be essentially outsourced to an LOM or a, [Brandon] you know, [Brandon] some there's not a world where you're forced to bang your head against [Brandon] something and you're forced to take a taste by the hard way. [Bo Wang] This is why we need academic training where you get into a field you know [Bo Wang] nothing about and hopefully after you graduate become the expert about this [Bo Wang] particular topic in the world, [Bo Wang] right? [Bo Wang] This is why you have to go through different programs to talk to your peers, [01:22:29] [Bo Wang] talk to your professors to get an idea about what's a good taste to begin [Bo Wang] with. [Bo Wang] But more importantly, I always teach students that the best way to learn [Bo Wang] something is to just program it. [Bo Wang] By programming, [Bo Wang] you kind of know what's the details hidden in all the mathematical equations [Bo Wang] in the paper, [Bo Wang] which often you omit. [Bo Wang] But in the era of agentic AI, [Bo Wang] things are slightly different in the sense that we used to spend lots of time [Bo Wang] coding, [Bo Wang] a little bit of time just debugging. [Bo Wang] But now we let the agent do most of the coding, [Bo Wang] but we spend most of the time debugging, [Bo Wang] which seems to be definitely interesting to me. [Bo Wang] And we had lots of discussion with the students in the lab what's the best way [Bo Wang] to spot bugs by by AIs. [Bo Wang] So how do you find places where AI is particularly good at? [Bo Wang] Also, how do you find places AI are still limited at? [01:23:31] [Bo Wang] You need lots of China era as well. [Bo Wang] And in the end, you still need to kind of validate your model using real world [Bo Wang] evidence, right? [Bo Wang] So that's why collaboration with wet labs, [Bo Wang] collaboration with clinical teams to validate your model, [Bo Wang] provide feedback signals to your taste, as we discussed, [Bo Wang] is certainly a very important training program. [Xi Chu] On the wet lab side, academic has extremely important roles to play. [Xi Chu] I think we'll enter a field of [Xi Chu] symbiotic innovation and cross-pollination of ideas. [Xi Chu] Just like in AI field, I think we have great ideas coming out of academia [Xi Chu] still all the time. [Xi Chu] But industry now increasingly are contributing new ideas on architecture, on [Xi Chu] all of that as well. [Xi Chu] In the wet lab side, certainly industry seems to be able to scale these type [Xi Chu] of data generation quite well. [Xi Chu] But biology is so much more than just cell-based perturb-seq. [Xi Chu] Beyond RNA-seq, we would like to measure many other analytes, right? [01:24:35] [Xi Chu] Proteins, metabolites, lipids, [Xi Chu] protein-protein interactions. How do we do that at scale? [Xi Chu] Beyond individual cells, [Xi Chu] we would like to be able to measure cell-cell interaction, spatial, [Xi Chu] cell in their native context, [Xi Chu] or even whole animal level in vivo perturbations. [Xi Chu] Again, how do we do that at scale with lots of great innovation coming out of [Xi Chu] academia? [Xi Chu] Actually, just one great paper last week. [Xi Chu] And so how do we connect all of these together? [Xi Chu] I think we have many years of work ahead of us to fully crack data for all [Xi Chu] biology. [Xi Chu] And that I think we need to scale the industrialization, the innovation from [Xi Chu] industry. [Xi Chu] We also need that from academia. [Xi Chu] I think we'll together move this field to the next level. [RJ] One question that we've been trying to ask everyone is in your field, [RJ] which you could say maybe is AI and sort of high throughput experimentation, [RJ] or however you wanted to find out. [RJ] If you could wave your magic wand and have a bottleneck removed for you, or a [01:25:37] [RJ] key problem solved, what would that be? [RJ] Protein. [Xi Chu] If there is a way to do protein sequencing or high throughput protein [Xi Chu] measurement, [Xi Chu] the same scale that we can do genomics, that would be amazing. [Xi Chu] I'm training genomics field, but if I can do that, I would incorporate that [Xi Chu] technology in a heart beat. [Xi Chu] RNA is amazing. It foreshadows which proteins are going to get made. [Xi Chu] But protein by and large are the functional units in a cell. [Xi Chu] Not only does their abundance matter, [Xi Chu] their post-translational modification matter, [Xi Chu] their localization in a cell matter. [Xi Chu] If you can measure all of those things, their conformational states, [Xi Chu] their modifications, their abundances, their localizations at scale, [Xi Chu] single cell, or even spatially, [Xi Chu] I think such data sets would be incredibly useful to train the next generation [Xi Chu] of initial models. [Xi Chu] I know there's a lot of innovations in that direction. [Xi Chu] Can't we just see that become a come of age? [Bo Wang] My hope is I hope to see a breakthrough in sequencing technology, [01:26:37] [Bo Wang] not just the reduced cost, [Bo Wang] but sequencing technology that can sequence the same cells at different time [Bo Wang] points. [Bo Wang] I think this is much lacking right now because you have, in order to sequence [Bo Wang] the cell, you have to kill the cell. [Bo Wang] Can we have a technology that can measure the cell states at different time [Bo Wang] points for the same set of cells? [Bo Wang] I think that will bring a very different dimension to the data set so that we [Bo Wang] can start to measure the temporal dynamics of cells. [Bo Wang] So far, everything we measure, everything we model is extremely static. [Bo Wang] Can we have a technology that measures different cell response at different [Bo Wang] time points for the same set of cells will unlock massive opportunity to model [Bo Wang] the dynamics of cells? [Bo Wang] To me, that is a real virtual cell. [Brandon] That's a really interesting idea. I never would have thought about that. Wow. [Brandon] Would you be okay with even just partial small snippets of genes or maybe [01:27:37] [Brandon] three prime regions of a small number of transcripts? [Bo Wang] Yeah, we can start with a small set of gene panels to begin with, right? [Bo Wang] But eventually, since we are talking about the magic, [Bo Wang] eventually, [Bo Wang] if we can have a system that observes how cells evolve at different time [Bo Wang] points and we have enough data to actually model such development, [Bo Wang] I think that would be a real virtual cell model. [Xi Chu] There are small attempts, also earlier attempts, in just, for example, [Xi Chu] sucking out portions of the cells, [Xi Chu] taking almost biopsies from the cell to do a fraction of the cytoplasm [Xi Chu] measurements. [Xi Chu] So that might be similar to the idea you talked about. [Xi Chu] There's also work from a public in this lab to have the cells secrete little [Xi Chu] vesicles and they harvest that in the cell culture media to measure what the [Xi Chu] cells are producing longitudinally. [Xi Chu] But there hasn't been technology that can let you measure the entire cell's [01:28:39] [Xi Chu] transcriptome while still keeping the cell. [Xi Chu] You can't have the cake and eat it. [Brandon] Cool. Yeah, thank you for taking the time to chat with us. It's been great. [Brandon] We've learned a lot. It was a lot of really interesting discussions. [Brandon] I especially really appreciate your commitment to open science and all of the [Brandon] cool models and data, [Brandon] if you're at least. [Brandon] Is there any last thoughts you have or anything you'd like the audience to [Brandon] know? [Brandon] Follow up on that. [Bo Wang] Overall, I think virtual cell is such a new and fast moving field. [Bo Wang] We hope to have more and more people join us and our Excel paper is out and we [Bo Wang] look forward to receiving your comments and feedbacks. [Bo Wang] And also we're hiring. [Xi Chu] Yeah, we are always looking for talented engineers, technologists, [Xi Chu] biologists, drug hunters, AI scientists, computational scientists. [Xi Chu] So look on xaira.com. Look for the open roles. [Xi Chu] We'll be happy to chat with you. [RJ] Thank you. Thank you very much. Great. [Xi Chu] Thank you.