The Nightjet needs ofcourse a reservation even in seated carriage (14€)
Can be done via oebb.at (follow the manual below and dont click on the seat only option. The Nighttrain from Villach to Ljubljana needs a reservation aswell)

You may take the Rome to Trieste Nighttrain and connect in Trieste on one of the few trains to Ljubljana. (more sleep for you)

Hey 

Thanks for your answer. The weird thing is if I search for the trains single that meaning rome-villach and then villach- Ljubljana it shows both trains need reservations but if i search the combination of those it says I don't need reservations for the rome-villach connection, so I thought maybe there is a discount...

Thanks for the Trieste Tipp, I think I'll try that!

@Lit Timetable Trieste - Ljubljana besides these two trains there is a Eurocity arround Lunchtime from Trieste to Ljubljana that goes onwards to Vienna :)

sorry I posted before an answer that was wrong, I have read about Vienna to Ljubljana and not from Rome. 

Like @seewulf says, you can take from Trieste the EC or one of the 2 daily Regional trains to Ljubljana (The EC need a small fee about 2-3€ but reservations optional). You can take the IC Notte nighttrain from Rome to Trieste and than change in Trieste to the Regional train or EC to Ljubljana.