When that has happened to me, it is always an oozing issue. Try more aggressive retraction settings and/or reducing the flow rate. Also, consider changing filament brand/type. I had some cheap makergeeks PLA a while ago that was really bad about this. Settings helped, but switching to a better filament completely solved my problem.