The architecture of GI-ImmunoWorld. (IMAGE)
Caption
(a) the use of a hierarchical, variational encoder to encode multimodal (multiple types of) data through modality-specific encoders (graph attention network [GAT], convolutional neural network [CNN]–transformer, and multilayer perceptron [MLP]) into a single 64-dimensional (64D) latent space using cross-modality attention, (b) the use of a causal transformer decoder to model temporal dynamics and predict the future evolution of the immune system given treatment actions, and (c) a treatment optimization layer using model-predictive control (MPC) and the cross-entropy method (CEM) to identify patient-specific treatment strategies. The latent space has biologically interpretable axes (e.g., the tumor-associated macrophage [TAM] M1/M2 polarization spectrum, CD8+ T cell infiltration levels, and the cytokine gradient signature) that can be used to understand the immune response. An example case demonstrates that the model-guided treatment strategy of anti-programmed cell death protein 1 (anti-PD-1) and rivoceranib resulted in favorable changes in the microenvironment of the immune system.
Credit
Lei Zhou, Central South University.
Usage Restrictions
News organizations may use or redistribute this image, with proper attribution, as part of news coverage of this paper only.
License
Licensed content