I have no balance problems with my toggle line being one sided only since they are opposed (port side on bow, starboard on stern) and carrying the kayak balances them out.
I have not noticed any adverse behaviour when holding it just one end, in the water.
Your proposed location looks fine to me since you have your kayak’s ends filled.
In one of my kayaks I had to add a small section of fibreglass tube where the ends of the kayak were hollow: I glassed them in with epoxy paste.
If you were to go for the retractable-handle line maybe consider angling the hole to facilitate the sliding of the rope.
In other kayaks I have joined the loop in the line with PVC tubing to prevent fingers being inserted in the lopp and injured if kayak were to spin (like in a surf wet exit).
Full Size Sail Rig
Pull-Up Strap Handle Kit