That isnt caulk, its sealant tape that is under the molding and squishes out when tightened. Its kind of like butyl tape or "tacky tape". Can't remember the exact name of it.
I would clean it off and not worry about it. Could always put a small bead of non-sag dicor or seam sealer, but i dont think its necessary.