Twist & Pivot Pattern Puzzle

It's harder than it looks—how DO the pieces come off?