Why not screw your hooks directly into the bottom side of the shelf above each. Small cup hooks should be more than adequate. Loading would be more in shear also.
Not penetrating the walls is a much better move.
You could use adhesive if you cant find the studs. I would use construction grade liquid nails. It will not let go of anything. I have seen 2x6s glued together with this and then have them hold 300lb equipment on sites. It would pull your paneling off before giving way.
i used construction glue when I mounted the coat rack in the garage of our XLR and has worked great. I just put some one the back and then put a few short pin/nails in them to hold until the adhesive dried and then screwed the coat rack on with screws long enough to go through the maple board and in to the wall.