You've encountered the biggest drawback of a residential fridge; they're not easy to keep cold when boondocking. Fridges designed to run on propane consume only a fraction of the power of a residential unit.
If you want to boondock with your current fridge, you'll need to determine its actual power consumption. You'll need an AC ammeter to determine its actual current draw and need to determine exactly what percent of the time the compressor is running. Use this information and add an additional 15% to the power needs to account for an inverter running at 85% efficiency. Once you have your consumption calculated, you can determine the required size of the battery bank.
If you add solar power, you need to have enough watts from the panels to run the fridge and recharge the batteries during the day. You might be surprised at how many watts of solar power you need. Make sure you get an MPPT solar controller and, if you have enough shadow free space on your roof, consider wiring the panels in series instead of parallel. Don't forget that on most solar panels, if one cell is in shadow, the power output of the panel drops to almost nothing.