I could see maybe a park being unbalanced if an electrician only wired Red to L1 in the entire park and ran it that way for every single breaker, but I feel that would also have to be the case for the cord and panel in every RV to also be wired the same of L1 being the main line and L2 only for a second AC unit. I doubt many if any worker on an RV knows that THIS cord Red has to be L2 and Black has to be L1 and then on top of that know that L2 is the Right or left side of the cords plug in and then wire EVERY camper the same way across EVERY brand.
BUT I feel a lot of parks are going to be more commercial 3 phase just to be able to supply that sort of load of 50 amp demands and that balancing would have been taken into account for the 240 only parks, ala issues would have presented them selves long ago. as the parks supply on 1 side would have melted not long after opening.
What I suspect happens is Green is ground, White is N and the rest is put on a breaker... with no real rhyme or reason
What I would suspect is you swap the lines and the voltage drop follows it and the issue is more related to the distance, overall loads of the park, distance of your pedestal and or condition of the outlet. Not sure how often you read the voltage on each line but it would be interesting to see some data points VS just the one.
Also what other loads did you have on? hot water heater, microwave, hairdryer.
Might be good to check your RV wiring as well to make sure you have proper wire size, proper TQ settings on connections, wire nuts seated, and if you have a generator transfer switch that it's in a good state.
Not saying maybe that day that park you and a few other campers weren't all on the same line utter possible but we need more data and I think your idea is solid just don't' be surprised if the outcome is less than what you expect. in all parks and situations.