For my eastern box turtle I make sure the substrate stays very damp (apart from under her light where it is impossible as it dries out the dirt so fast). Keeping the substrate damp will automatically help with the humidity, and if the air ever does get too dry for them they can burrow. Make sure to also have a lot of moss in there as it helps with humidity
For baby box turtles, I put a brick under one side of the plastic tote that they're in so one end is elevated. Then I fill up the lower end with water. Because baby box turtles are so very, very small, they dry out quickly. They MUST be kept hydrated or they will die. So my substrate is quite wet and the habitat is covered.