interesting
/
3d-to-photo
mirror of https://github.com/Dabble-Studio/3d-to-photo.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
3D to Photo is an open-source package by Dabble, that combines threeJS and Stable diffusion to build a virtual photo studio for product photography. Load a 3D model into the browser and virtual shoot it in any kind of scene you can imagine
11 Commits 1 Branch 0 Tags 23 MiB
JavaScript 66.6%
CSS 25.9%
Python 7.5%
Go to file
Erjan Kalybek 4fb0831124 Fix readme typo 2023-10-22 21:00:15 +02:00
components scaling object to fit canvas 2023-10-13 10:42:54 -04:00
image_proc_server scaling object to fit canvas 2023-10-13 10:42:54 -04:00
pages adding some sample models 2023-10-12 17:22:22 -04:00
public adding some sample models 2023-10-12 17:22:22 -04:00
sample_3d_models scaling object to fit canvas 2023-10-13 10:42:54 -04:00
styles Initial commit 2023-10-12 14:27:51 -04:00
.env.example Initial commit 2023-10-12 14:27:51 -04:00
.eslintrc.json Initial commit 2023-10-12 14:27:51 -04:00
.gitignore polishing up 2023-10-12 16:47:21 -04:00
LICENSE Create LICENSE 2023-10-19 14:15:13 -04:00
README.md Fix readme typo 2023-10-22 21:00:15 +02:00
jsconfig.json Initial commit 2023-10-12 14:27:51 -04:00
next.config.js scaling object to fit canvas 2023-10-13 10:42:54 -04:00
package.json Initial commit 2023-10-12 14:27:51 -04:00

README.md

3D to Photo

3D to Photo is an open-source package by Dabble, that combines threeJS and Stable diffusion to build a virtual photo studio for product photography. Load a 3D model into the browser and virtual shoot it in any kind of scene you can imagine. The app currently uses Stable Diffusion 1.5-inpainting, hosted on Replicate.

Demo Video

3D Photo to Studio Demo

How it Works

  • Upload a 3D model of any object in .glb format.
  • Adjust the position and orientation of the model on the canvas
  • Describe the scene you want to create in the text box and click generate image

Use Cases

  • Product Photography: Create product lifestyle photos in any backdrop you can imagine, without a physical photoshoot.
  • Synthetic Data Generation: Generate synthetic images of an item in a variety of scenes. Useful when training object detection models.
  • Previsualize Game Assets: Upload your game assets and generate level art around it to previsualize scenes.

Tech Stack

  • ThreeJS to handle loading and viewing 3D models
  • Stable Diffusion 1.5 (inpainting) by stability.ai
  • Replicate to run Stable Diffusion
  • NextJS by Vercel for the front-end
  • Python Flask Server for some backend image processing functions

Install and Run

  1. Clone this repository
git clone git@github.com:Dabble-Studio/3d-to-photo.git
  1. Install necessary packages for the front end
cd 3d-to-photo
npm install
  1. Install necessary packages for the python backend
cd image_proc_server
pip install -r requirements.txt
  1. Run the Flask Server
flask run

  1. Set your Replicate API key. In another terminal, navigate to the root folder and create a file called .env. You can use the .env.example file as a template. Paste your Replicate API key in this line REPLICATE_API_TOKEN=YOUR_API_TOKEN, in place of YOUR_API_TOKEN

  2. Run the NextJS app

npm run dev
  1. Use one of the sample 3D models and drag it into the upload area. Enter a prompt in the text box and click "Generate Image"