CABBAGE_BLAMEMIA_ZHANGWORKS     ABOUT
LINKEDIN INSTAGRAM EMAIL
CABBAGE_BLAMEMIA_ZHANGMESHY_UNITY_TOOL



Meshy2Cavern
Meshy2CAVERN is a Unity plugin to integrate AI generated 3D assets into the Unity immersive experience development pipeline. It is a demo tool for exploration of potential real time 3D models to the CAVERN(Unity) development pipeline. Even though this tool only has a basic working demo of the asset generating and importing pipeline, it showcases the potential of integrating AI generation and CAVERN immersive experiences. 


Feel free to try it out and improve it!



  
Goal
The objective of the independent study will look at the pipeline of importing AI generated 3D content into an active Unity Environment running on the Cavern Setup. 

While we already know that AI Generated models and meshes can be imported into Unity (and thus the Cavern Toolkit setup) this exploration looks at ways in which a user (or designer) can import AI generated content in close to real time as the unity environment is running. 

The promo video produced by NVIDIA demonstrates what this would look like in theory - 




Final Plugin
DOWNLOAD LINK

HOT TO USE
  • Download the Package and Import
  • On the top of the Navigation bar, Meshy to CAVERN
  • Type your API key
  • Type prompt and generate
    



CURRENT USER FLOW
  • Meshy Bridge do not support runtime editing and modification, it only works in editor mode.

  • Meshy Interface works as a search engine to ford models already generated in Meshy’s database. It does not support prompt generation so far.

  • Meshy Cavern Tool creates an Unity Editor interface to allow in-editor generation and modification of the meshy AI 3D model generation. It also supports real time editing.











     

DEVELOPMENT & ITERATION
Week 1: GUI - Model importer Manually download from Web app and automatically import into the scene
Basic functions: import, randomize and delete
Installed Meshy Plugin

Week 2: Meshy API Using the API key to download the model directly into Unity Folder and Scene
Analysed the differences for each assets pipeline
Adding Prompt input in GUI
Debug the threading control

Week 3: Realtime Generation Refine the GUI Importer
Adding download format and the task status
Refine the code structure
Implemented the in game UI system (WIP)

Week 4: Final Refinement Adding texture and material for the model
Finalize documentation