Congratulation to the Sketchfab team for the amazing work you are doing here.
I develop computer vision solutions for the web through my company, WebAR.rocks.
I have developped a proof of concept webapp to use Sketchfab viewer for virtual try-on:
But I meet 2 difficulties:
- To update the pose of the glasses, I use the setMatrix function (Viewer API - Functions - Sketchfab). But it is slow. The callback function takes a long time to be called and during this time the pose is not updated. So the movement of the glasses is jerky. Is it possible to update the matrix directly by reference? Or is there a faster way to update the pose of an object in real-time?
- To hide the glasses branches, I need to add an occluder 3D object, i.e. a 3D object with the shape of the face, which will be rendered with a material writting into the depth buffer, but not in the color buffer. The goal is to reproduce the occlusion effect of the head of the user (in this demo for example there is an occluder: WebAR.rocks.face glasses VTO demo). I did not managed to create such a material with Sketchfab.
I think it would be great to use Sketchfab viewer for virtual try-on. Many client need a very high quality rendering, especially for jewelry and eyewear, and your viewer is amazing. I regularly have the feedback of some client asking how they could have both virtual try-on and Sketchfab rendering quality.