when to book reservations

You can use the filter "no seat reservation" in the Railplanner app. 

It really depends on where you're heading and when, and how flexible you are with your routes. On peak summer days, it's quite likely for certain trains to sell out, particularly on popular touristic routes. On a random Wednesday in October, you should be able to book a reservation last minute for nearly trains.

For france it usually makes sense to reserve TGVs in advance, as the have dynamic reservation prices (10/20€). on domestic routes, which are based on demand. 

As already said the risk of trains selling out depends on when you plan to travel. 

From my experience reservation free routes in France for longer distances are often not sensible alternatives. (but this is mainly based on my experience with some routes to/from Paris)

Thanks for answering! We are traveling in start August. So would it be two-three days in advance or a week in advance? Or it depends on the routes?

We would like to go to smaller cities, is it then possible to be more spontaneous?

Thanks again

Beginning of August is when all French and Spanish are on holidays. Many trains are already sold out now. Really depends on what routes you want to take.