Frankfurt(Main) is not a part of Baden-Würrtemberg (BW). You cannot use these BW-tickets on ICE and IC. All the rest of public transport you can use (from bus, tram, U-bahn, S-bahn, RE’s, IRE, RB’s, of all local companies in Baden-Württemberg...).

Same for the Quer-durchs-Land (Only RB, IRE, RE and S-Bahn - no local transport), but that you can use all over Germany (- I think?).

Use the transport planner of https://www.bahn.com/en and tick off “Local transport only” to find connections you can take with aforementioned tickets. For IC and ICE you need to buy proper tickets or use your interrail pass.

QdL is valid in the whole of DE on that day (from 9.00 on mo-fr) but ONLY on NahVerkehr=R/S or RE trains=more stops, NOT the sleek white faster IC or ICE. Same-same for Landestickets like B-W: these are ALSO valid on all local bus-tram/metro etc in that land. F is in Hessen land and this also offers its own landesticket, but only for small groups and they are not that cheap. I kind of remember you travel with the family-but forgot details.

HIdden deep in the bowels of the giant bahn.com site are some maps that show to where exactly the Landestickets are valid-B-W to somewhere just above Mannheim ( probably indeed a stop on the hourly RE trains doing MH-F), but I have forgotten which one. Dr Google will probably give the answer in millisecs. And there is no need at all to prebook, so check the locally folders+maps that ly out in any station in B-W. Or check their site bwegt.de. The point is then: from that stop till F will be in the RMV local VERBUND area, and this means that tickets often can only be bought locally from machine-but for the whole trip incl also bustram till final stop. So you have to get out train-find machine-buy tickets-and thus wait for next train (maybe 30 mins, maybe 60). And I understand its pretty cold+snowy right now in DE (not here in tropical BKK). SOME Can also be bought via the DB-navigator app, which then of course you have to install etc.

This trip is in fact not that str8forward on these tickets, as S->MH usually then goes via KArlsruhe=takes more time, as direct trains are only as ICE.

On bahn.de or .com simply use the setting ´Nur Nahverkehr´ to see what it delivers in timings. Any money-conscious German knows that since a very long time. OR see if bahn.com will offer any reasonable SparPreise for tomorrow-if at all probably only at very early or very late trains.