I feel your pain...the only major niggle I have with the 323 is that she does not point well. I have tried everything to improve her performance while pointing, but you can only do so much with this design. When I race, if we have to tack a lot, we are toast (and PHRF doesn't help). When there are fewer tacks, and more reaches, we can (and do) win. It's fun to do that to serious racers with modern boats...
I feel that this is mainly caused by a combination of the shallow draft and the sheeting angle. When pointing - and turning - the shallow keel means that she tends to slip sideways. The shape of the keel and rudder down not give the "bite" or lift that a deeper fin keel would. Tradeoffs....
I have looked at various options for the sheeting angle, but even with my small jib (it gets windy here, so I have an 80%) the leech always hits the fore shroud. Now my smaller jib is cut high, perhaps if I had a specially made small jib with a low foot it would not hit the shroud...not sure.
Play around on a calm day with your sail, mocking up the position of the jibsheets with a cabintop rail - see if you can come up with something. I'd love to hear you thoughts.
In the meantime, just beam reach - she sails very well like that!