Trap Pond State Park, in Deleware. Decent campground and right on a pond where you can paddle in and out of the same spot. Nearby, small but enchanting Trussom Pond worth exploring, too.
Shad Landing (Pocomoke River State Park), near Snow Hill, Md. This is a very nice camping park located on the water. You can paddle from the Park or take day trips downriver (see the peep at Pocomoke River Canoe Company for more info on narrow upriver sections) and to tribs such as Dividing Creek or Nasawango Creek.
Close by, on the western shore, Governor Smallwood State Park, on Maryland's side of the tidal Potomac. Nice camping park, nice trails, on the water, and check out the nearby ghost fleet of Mallows Bay (cartop it down there, it's a several miles on the big open river).