Ender 3 pro x-axis issue

Attachment 14386 Attachment 14390
Hi, I'm having some issues with larger prints.
When my Ender 3 is printing the first layer the x-axis sometimes goes back to its home position but it continues printing in the y-axis resulting in a lot of noise as it stays in the same height. (It prints everything on top eachother while the printhead is not going up)
It's as if it forgets all about the x-axis.
This problem only seems to occur on bigger prints, the issue doesn't happen after it is done with first layer.
I've tried switching from Cura 4.6 to PrusaSlicer and changing print settings but the problem persists.