Post

AudioGPT

Posted Updated
By 1 min read
  • Paper: AudioGPT: Understanding and Generating Speech, Music, Sound, and Talking Head
  • GitHub Link
  • Publisher: Arxiv
  • Author Affiliation: Zhejiang University & Peking University & Carnegie Mellon University & Remin University of China
  • Functional Division
    • Understanding
    • Generation
  • Design Division
    • Tool-using
    • End-to-end
  • Input Modalities $\rightarrow$ Output Modalities
    (I: Image, V: Video, A: Audio, 3D: Point Cloud, T: Text, ID: Document understanding, IB: Output bounding box, IM: Output segmentation mask, IR: Output retrieved images)
    • I+A+T $\rightarrow$ A+V+T
Arxiv
MM-LLMs
This post is licensed under CC BY 4.0 by the author.