I use a D5300 for most of my scans, using the 18-55 kit lens for my outdoor/large subject scans (a 85mm macro lens is used for subjects that are fist sized and smaller).
The 85mm macro lens seems to work pretty well for most of the mineral scans I do, but i have had some trouble scanning smaller specimens (like rat mandibles and smaller mineral specimens).
Zircon Crystal: 1.5cm long
Rat Mandible: ~1.7cm long
It would be worth considering image stacking software (like Helicon Focus) if you plan to scan things like insects (unfortunately, i don't have a great deal of experience with image stacking software since i haven't needed to use it for any of my scans so far).
@nebulousflynn has done some great insect models ( https://sketchfab.com/nebulousflynn/collections/zoosphere ), and if you go to the zoosphere website, they give a very brief overview of the workflow involved in terms of photograpghy: http://www.zoosphere.net/project.
I'd imagine the photography stage for something like this to be incredibly time consuming, since you'd be wanting at least 100 images of your specimen (minimum) from enough angles to ensure complete coverage (and each of those 100 images will be the product of 5 or more images processed in Helicon to produce 1 in focus image).
As @russsm and @f1xer have pointed out, using a turntable and a tripod are definitely a must if you want to do complete scans of smaller subjects, and the camera settings are very important too. My go to settings are typically
ISO: 100 (bumped up to 200 for darker subjects, like coal)
F-stop: ~32 ( not sure if this too high)
Exposure time: 3 sec
I use the ceiling lights in the lab I work in because they seem to provide even, flat lighting for my subjects
The turntable is marked at 20° intervals to ensure i take my photos at even spacing and have sufficient overlap (although you need less photos as you photograph your subject from higher angles https://skfb.ly/TDtM)
A sheet of white card-stock is used for a backdrop which seems to remove the need for image masking.
I scanned this specimen at home using a lazy susan as a turntable with a white card-stock backdrop, the 18-55 kit lens, and natural lighting (under the veranda in my backyard). The specimen was roughly 15cm x 7cm x 5cm.
So you can get away with a pretty budget setup in some cases (excluding the DSLR, and processing computer, which aren't exactly 'budget' ).
I've had pretty good results with Agisoft Photoscan Professional, but RealityCapture seems to be a lot faster in terms of processing time, and handles finer details better (like tree branches and foliage, or spines on certain creatures). I have heard that it has some issues aligning photos from turntable scans though. You can get a 3 month RealityCapture license for €99, or €15,000 for lifetime access, just in case you feel like taking out a small mortgage...