Yes, each light "should" have two wires connected to it. The 12 Volt line and a Ground.
If you remove the lenses, and then the screws holding the fixture into the ceiling, you should be able to gently pull the light away from the ceiling and the two wires normally go up through a hole in the ceiling and attach to the two wires mentioned above. Pull the light gently so that you so not pull the connections apart.
If both connections are good to both lights, then there is most likely a wiring issue up in the ceiling.