Nice little discussion going on here. I'm a canoe guy and I think a canoe would do two things for you. First, canoes are so incredibly versatile and by far the best paddlecraft for kids. The kids can move around. They help out with the paddling, or not. You can haul plenty of stuff. Even enough to camp if you want. North America was explored and settled in canoes. They are just awesome.
Then there's the issue of comfort, and canoes have it over kayaks hands down. Kneeling is SO much more ergonomic for your back, its not funny. Your back stays nice and arched and upright, not scrunched up unnaturally like when sitting a kayak. Be sure to get a good kneeling pad, and maybe some toe blocks if you want. And, of course, you don't have to stay kneeling. Move around; sit back when it suits you. Go ahead and stand up if you want. Strech out those legs. You can do it in a canoe.
The last thing, though, is the "fitness" part. I confess that I've never really felt like a got a "workout" in a canoe. Sure, it'll wear you out, like walking or hiking. But you don't get that hurts-so-good cadrio burn like after a good run or bike ride. Kayaking maybe does that better because you don't really recover between strokes. Even hit n switching with a single blade doesn't quite get the heart rate up as much as doubleblading, in my experience. Hit n switching gets rather tedious. Then again, I find doubleblading kayaks rather tedious. But I suppose it is a better "work out." Anyway, canoes are probably better paddled for the joy of paddling, not for any particular "fitness" program. Good luck!