Prompt-guided multimodal framework for interpretable inverse design of photonic structures

Designing complex nanophotonic systems requires inverse design frameworks that can efficiently explore vast parameter spaces while maintaining physical realism and interpretability. Traditional adjoint-based optimizations often rely on manual tuning and remain limited to specific problem settings. Building on our previous work that combined topology adjoint optimization with a physics-conditioned latent diffusion model for thermophotovoltaic metasurface design, we introduce a prompt-guided multimodal framework for generalized photonic structure design optimization. This approach employs text–image dual training to correlate descriptive prompts with topology metasurface designs. Structured textual inputs—capturing material coverage, geometric orientation, and clustering features—are encoded alongside topology data into a shared latent space, enabling the diffusion model to translate human-readable descriptions into physically consistent photonic patterns. The use of descriptive prompts provides a pathway to incorporate realistic nanofabrication constraints, allowing theoretical designs to be adjusted according to process limitations and material tolerances. The framework exhibits broad generality by combining transfer learning, enabling language-driven generation of diverse photonic designs through an intuitive and unified approach.