is reinforce what others have already said.
1) Surfboards, boats, ships all capsize in the right conditions. Anything that floats can encounter conditions that exceed the performance envelope of your craft. Frankly, if you think you won't capsize, you don't have enough experience to understand the dangers you are in.
2) Prepare for immersion. You will probably find a drysuit more serviceable, comfortable, and warmer, than a wetsuit. Wear wool, not cotton.
3) Read some first hand accounts on hypothermia onset after a capsize. It happens quickly and can overcome the best in a shockingly short time. Judgement and the lost of hand control can occur in less than a minute in some people, over 5 minutes in others. Find out which end of the spectrum you are in and try to do a self rescue in the conditions where you paddle. If you can easily do it, fine. Just remember than any conditions that make cause a capsize will still exist after you rescue and that you will be weaker after each successive capsize.
You ask, "So... am I being an idiot for not wearing a wetsuit, or is it a reasonable thing to do," and the answer to this isn't that cut and dried. I feel you are taking unnecessary risks by paddling alone in conditions where immersion could well be fatal. That you are not taking some reasonable precautions to prevent hypothermia suggests that you are more confident in your boat than paddlers with a great many more years of experience than you. In truth, you are exactly the type of paddler that I would NEVER go on the water with because you endanger everyone around you.
If your lake can produce 2.5 foot waves (which are NOTHING by the way), it can also produce larger waves. Unless you've been out in the worst possible conditions (and no matter what conditions you've been out in, something worse can come along), don't assume that any body of water can't outperform either your boat or your skills.
You say the boat was , "a bit difficult to control," in those conditions. This suggests that a capsize is a WHOLE lot more likely to occur than you think. And by the way, it isn't the boat that capsizes, it's the paddler. The only way a kayak can stay upright in virtually all conditions is because the paddler knows what to do and when to do it. Any boat can, and will, capsize.
I'll be honest and say that I feel you are taking unnecessary risks here, but if you are comfortable with those risks, go out on the water alone, dress in cotton, paddle during storms, wrap your boat with aluminum foil so it conducts electricity better, or whatever else you wish to do. I'll probably say something like, "it was an event just waiting to happen," when I read the incident report.
If you feel that I am being harsh or rude, I don't really care. I've had to many experiences on the water where I've rescued others from their lousy choices and seen what can happen in even warmer water than those you plan to paddle.
I've been in waters cold enough to defeat (to some degree) the thermal protection of a wetsuit and even performed a rescue in those waters. It wasn't fun, and the two scouts we rescued both had mild hypothermia after only about a minute of immersion.
Pardon the wall of text, and either take what I write under advisement or not. Your choice.