Yes, you need to pay extra. Seat reservations are not free outside the UK (and Norway with 1st class). Sometimes they cost 3€, sometimes a fair bit extra. Depends whether the seat reservations are optional or mandatory. This depends on the route/company. Have a look at this thorough guide: https://www.seat61.com/interrail-and-eurail-reservations.htm

Some advice:

Brussels-Paris: the direct Eurostar trains are expensive and have a passholder quota. You must book ahead of time. Alternatively you can travel for free via Charleroi and Maubeuge (few times a day, slower) or Lille (10€).

Paris-Zurich: the direct TGVs to Switzerland are expensive. However if you hop off before the border, they cost 10-20€. Then continue by TER to Basel. Seat reservations are not a thing within Switzerland.

Zurich-Milan: 13€ for the direct train but you can easily avoid the fee by taking any train to Lugano followed by the hourly RE80 (all reservation-free). Scenic routes would be via Airolo (old Gotthard line) or Chur-Tirano (Bernina) but that’s a bit of a stretch. ;)

Thank you so much for the reply. The input was a very big help. 
 

I am trying to figure out how what is the advantage of 1st class Pass agains 2nd class pass if both of them require for me to pay extra for the seat reservation to get the first class seat? 
 

With a 2nd class pass you can only sit in 2nd class (upgrades are sometimes available but they quickly cost hundreds of €).

Seat reservations mostly cost the same in both classes, sometimes 1st class is slightly more expensive.

Basically you choose 1st class for: quieter and less busy carriages, larger seats, extra legroom, more last-minute availability. In some countries, as a bonus, you might get a free drink or snack but this is not the case in those you mentioned above.

The price difference between 1st and 2nd class passes is much smaller than 1st/2nd class tickets so I'd say (and I'm not the only one) that 1st class passes are great value.

You can’t sit in first class with a second class pass. If you make a first class reservation without a first class ticket / pass, it won’t be valid.

In fact that depends on the train and the available fares. In France for example, this is possible because there's an upgrade reservation available. For the trains mentioned though, such an upgrade doesn't exist.