Puzzles are so fun! My boys started to enjoy them by age 3. They were simple puzzles, but that's perfectly fine. Puzzles are great for developing a mathematical mind. While doing a puzzles, pieces are tried in different ways and are put together and taken apart.
That practice is important later in math. As our children start to do operations with numbers (adding, subtracting, multiplying, and dividing...just to name a few) the ability to take numbers apart and put them together makes is easier do do the math. For instance 47 + 13 = ? I can take apart 47 to be 40 and 7 which are added to 10 and 3. In combining the 40 and 10 I get 50. Then I combine the 7 and 3 and get 10 more for a sum of 60. This is how we add in our heads as adults. But so often in school we learn to add 7 plus 3 and carry the 1.
Puzzles develop our minds to think practically, efficiently, and flexibly. And of course, puzzles are fun!