Overview of the proposed zero-shot colorectal polyp detection framework (IMAGE)
Caption
The framework leverages a frozen Contrastive Language-Image Pretraining (CLIP) backbone adapted via parameter-efficient modules. (a) Training Phase (Source Domain). Utilizing a single annotated dataset, Attribute-Enhanced Prompts guide the Text Encoder to generate discriminative normal and abnormal anchors. The Image Encoder is refined using Gated Visual Adapters and Generalized Mean (GeM) Pooling to capture multi-scale features. Training is guided by a segmentation loss (LSeg) and a pseudo-normal contrastive loss (LPNC), which uses background regions as pseudo-normal samples to distinguish lesions. Only the adapters are updated to preserve pre-trained knowledge. (b) Zero-Shot Inference Phase (Target Domains). The trained model is evaluated directly on unseen datasets without fine-tuning. Anomaly localization is achieved by computing pixel-wise similarity between the adapted visual features and the learned text anchors, generating precise anomaly heatmaps.
Credit
Sichao Tian, Jing Zhang, Weidong Zhang
Usage Restrictions
Credit must be given to the creator. Only noncommercial uses of the work are permitted.
License
CC BY-NC