Photogrammetry of plants parts/leaves! (Agisoft Photoscan)


(Tinyruin) #1

Hello everybody!
We create the art of our VR game using Photogrammetry and we also want to include some autumn leaves in there.
As I read in the beginning photogrammetry has problems with that (I think it's because of the thickness of the leaf as seen from sideview - can't align that). So far I could fix the aligning errors by putting the plant parts on top of newspaper, so Agisoft has additional "good" reference points to "look at" when aligning the photos. This works fine for rendering one side of the object, but not a result from all sides. Next up my idea was to render - from a leaf for example - both sides as seperate photogrammetry and then combine both chunks using Agisoft photoscan.

However, I don't really understand the different Chunk Alignment techniques of Agisoft Photoscan/their advantages or disadvantages for this approach (from points, cameras or markers). So I wanted to ask you if you have experience with this alignment process.

What would be the "right" chunk alignment method in your opinion for this purpose? And have you scanned plants before and how do you go about it?


(Vlad) #2

For VR game and "flat" surfaces like leaves, photogrammetry is a huge overhead.
You can use http://shadermap.com/home/
and 4-5 images shooting technic for "extract" small details as Normal map or depth map.

Or you can use polarization light technics (http://gl.ict.usc.edu/Research/SpecScanning/) similar to used for Megascan textures.


(Vibrantnebula) #4

To elaborate on Vlad's post, the typical photogrammetry workflow is not a feasible solution for things like foliage and a slightly different approach is required. I'm not a pro at foliage yet but I hope I can help a bit. A leaf is often too thin for current photogrammetry to derive a coherent model and chunking will not help you in this situation. You could merge together two one-sided models in post, however, the output is unsuitable for real-time rendering as it will cripple your performance, and will require a fair amount of work on your end to generate. On top of that, a leaf is a semi-translucent object that owes much of its appearance to subsurface properties so a different technique is required to accurately render it anyways.

The general idea is that you capture surface details along singular planes such as the top of the leaf and generate maps that combine to form "flat" atlases that you can then modify in programs like 3dsmax & blender to create game assets that appear to be 3D but are made up of as few polys as possible. You can have fantastic results using the 4 or 5 shot method Vlad described but as he alluded to, producing higher quality maps that render more realistically will require special techniques that involve highly controlled lighting. I'd recommend checking out one of Megascan's free plant atlases to see what an atlas typically contains. This video shows the theory of creating very basic game assets using an atlas. More advanced assets would slightly subdivide the plane to bend the atlas to induce more of a 3D effect and can be used to create artificial variety and can include animations to make the asset more dynamic. @artbyrens is a good source to follow as he's a beast with foliage.

In terms of photoscan chunking, the manual provides the best explanation of the methods that I've seen to date. The general idea is that you can merge the models by brute force comparing reference points between models but that can be a very lengthy process. You can shorten this process dramatically by providing markers with known GPS coordinates or by feeding in camera locations. In many professional scenarios the camera location is actually GPS metadata generated by the camera and thus can be very accurate; if you use camera positions estimated by photoscan you can have issues merging chunks because the camera positions are estimates subject to reprojection error. In the end these features simply allow for quicker merging.


(Tinyruin) #5

Thank you Vlad and vibrantnebula for the interesting answers :slight_smile: :heart:
The flat surface method is really interesting and I'll try it out c:
I guess for our scale (player is the size of a human thumb), leaves, moss, needs to be more than a combination of planes.
From what I've found by art by rens on Twitter it seems he creates a proper mesh for the foliage: https://twitter.com/ArtbyRens/status/810766554174541825 and he has some cool modifiers he built to create variations of the fern. Really looking forward to his "4k experience"/forest scene.

Thanks for the explaination on the chunks too! You guys are the best <3


(Arc 3 D) #6

For thin surfaces, AGISOFT recommends you look at a new feature at:
http://www.agisoft.com/index.php?id=48

Even before this I have found that Photoscan can handle some thin surfaces if you have enough photos (in single chunk; they recommend against multiple chunks). You usually have to wrestle (reset-realign cycles) the photos into the correct alignment. We have done some paper sculptures and the structures like this. (https://sketchfab.com/ARC-3D)