We surf as well on longboards (just not nimble enough or even good enough to do otherwise)! I find that while we have a tendency to go straight, surfers with shorter boards get squirrelly in some case on purpose and others not so much. We have a tendency to stray away for high surf density and when it comes to kayaking of surf, I do not plan on doing that around surfers because I would not want that either. In fact, I often see a tension between surfers and body boarders as well!
We do not really plan on long distance trips!