Did another print and with that the problem appeared a bit higher than with previous print so it's not always on the same height. The wall layers when the problems appear is not smooth and flat but a bit rugged like the filament is sticking to the nozzle or something.

Someone else said I might have a problem with my Z-axis and the z-rod. Some kind of common problem with the ender 3. the fit in the motor and in the carriage is not always totally aligned making the travel a bit more heavy than it should. Saw a youtube video where he was able to push the whole z bar down quite easily, I have to turn the motor to lower it.