Not on the ocean, but have you looked at Point Lookout State Park in MD? It is on the spit of land where the Potomac River runs into the Chesapeake Bay. You have water on both sides. Some of the campground loops have water front sites, some are in the trees, some have services, some have none, some allow pets, some don't, some are FHU, some are partial. There are nice updated bath houses, lots of fishing, great nature center for the kids, ranger programs for them, boating, canoeing, crabbing, etc.
There is a lot of history associated with this place. It once housed a Civil War prison camp and a resort community back in the 1800s. There is a lighthouse to tour, beach on the Potomac River and a smaller on the Chesapeake.
Our DDs, 10 and 7, LOVE this campground and we spend each June there. We are also close to DC and do several day trips into the city for sight-seeing.