I'm not sure about the best burrowing substrate, but I use 50% coconut coir and 50% play sand when my RTs are inside (usually they are outside.) You can go to russiantortoise.org for some good advice.
Russians aren't usually very picky about what they dig in. For younger Russian and Northern Mediterranean tortoises I use a mix of 65% organic potting soil, 20% coconut coir and the rest ground sphagnum moss and leaf litter. I used to use sand mixed in too but found more problems than purpose for it. I keep them awake all winter until they can go out in the spring. For my adult Russians that are only awake for a few months while indoors I use about a 6" layer of hay. Which in about a week gets compacted to about a 3" layer of hay! My adults are either hibernating or outdoors most of the time.
As far as nesting substrate it needs to be something that will retain some moisture and hold together when they're digging in it. It also needs to be deep, 8" minimum, 10-12" even better. Large female Russians dig very deep nest cavities. Potting soil with coconut coir and some sphagnum would be a good choice. Regular top soil would be another good choice.