Thanks for all your replies! Sadly, I'm not able to get it aligned with the parts supplied.

I think I'm going to try the pressfit print you are suggesting KDog. If that works I will update this post with my a solution process.

If a printed part press fit over the bearing doesn't work I may redesign the bearing mounting parts in place of the wood ones so they are more adjustable like you suggested beerdart.