That will be a rather big disappointment-there is simply NO valley at all between these-just an awful lot of former industrial and partly gone into neglect very industrail and not overly interesting cities. Incl. even the very worst big station of all of DE=Germany, Duisburg, but renovation, overdue since 22 yrs, has now started.

THE scenic part of the Rhine Valley is much more south, south of Koblenz, with major lines on both sides of the river. The east one is more used for freight=thundering trains 24/7. Probably best known is RÜdesheim-also for its narrow wine-saufen alley, Bingen also and is slightly larger.  Grmans usually talk about links or rechts (left and right hand) Rheinisch=west and east.

There are frequent (and very often very late) trains on these lines so you do not need to base you in a minor hamlet, distances are not that long. There are also cheaper local passes, incl also buses etc in the whole Land=province. Some HTLs in some towns also provide you with a free local travel pass (Boppard f.e. and some others).