When creating a puzzle, there is an option to toggle Puzzle Rotation On or Off. This is located under to the Number of Pieces option.

Please note that you can only change the this option during the creation period and not once a puzzle has been started.

To rotate a puzzle piece, simply tap on the piece and it will rotate a quarter turn. If you accidentally started a puzzle with this featured turned on it cannot be changed in game. You will need to return to the home screen to select the puzzle image again.