VITRON

• Paper: VITRON: A Unified Pixel-level Vision LLM for Understanding, Generating, Segmenting, Editing
• Project Link
• Publisher: NeurIPS 2024
• Author Affiliation: National University of Singapore
• Functional Division
  • Understanding
  • Generation
• Design Division
  • Tool-using
  • End-to-end
• Input Modalities $\rightarrow$ Output Modalities
  (I: Image, V: Video, A: Audio, 3D: Point Cloud, T: Text, I_D: Document understanding, I_B: Output bounding box, I_M: Output segmentation mask, I_R: Output retrieved images)
  • I+V+T $\rightarrow$ I+V+I_B+I_M+I_R+T