Judging from your riding experience, your use case, and your preferences, my answer for your next bike is easy:
Hayabusa!
Just kidding. But you seem to just have gotten comfortable going fast in a straight line on your highway commute and wish for a more sporty feel. The SV is not the best bike for that scenario. Neither is the XSR, albeit slightly better.
What you want is a sports touring bike.
With good weather protection, highway stability, options for luggage, and possibly direct drive those are excellent highway commuters.
Super sports bikes like the GSX-R 750, R6, R1 or the ZX10R are probably the worst bikes you can get to learn on. They need to be ridden on track, that's what they were designed for. And that's why they're nearly double in price as you have noticed. The money is in the premium suspension, brakes, and weight reduction. Improvements that are overkill for street riding.
Don't join the masses of weekend squids who can't ride anything but straight with their show bikes.
There are plenty of options on the used market and since these bikes are usually ridden by more mature riders I wouldn't worry about any issues as long as there is a good maintenance record available. Get a well ridden Japanese model from the late naughts.
Examples are the Honda VFR800, Kawasaki Concourse 1000, Honda ST1300, Yamaha FJR1300, or Triumph Sprint ST. These bikes are made for long distances and are virtually indestructible. With their larger displacement engines tuned for torque not peak power, they have plenty of torque down low, so unlike super sports bikes you can ride them in a relaxed fashion.
These are not the most popular bikes on the used market and there's tons of them around, so they can be had for less than $5k. Remember, mileage doesn't matter nearly as much as maintenance.
You may even keep your SV for weekend rides on the twisties, because that's where it'll shine and where it'll be able to serve as your learner bike for years to come. A heavier, more powerful bike won't be nearly as easy to make progress on. Actually in hindsight, learning to ride corners on a sub 50hp bike would be my advice. The SV is not the best beginner bike either.
I hope you are aware that you're still a beginner with lots to learn.
If you want a new bike look at the 2022 Suzuki GSX-S1000GT. Probably the cheapest of the large sport tourers.