To measure insertion loss, you connect the channel 1 to the common port of the diplexor and channel 2 to one of the output ports while terminating the unused port with a 50 Ohm load. Move channel 2 to the other output port and terminate the remaining output port with 50 Ohms.
To measure isolation you connect channel 1 to one of the output ports and channel 2 to the other while termination the common port in 50 Ohms. Switch the connections on the output ports to measure the other isolation.
Gary
W9TD