Preheat oven to 350°F. Cream together butter and sugar in a mixing bowl. Add the egg, lemon zest, lemon juice, lemon extract, and buttermilk, and mix until combined.
In a separate bowl, whisk together flours, baking powder, salt, and almond flour until combined. Add the dry ingredients to wet ingredients and stir just until moistened. Fold-in poppy seeds.
Fill muffin cups 3/4 full and bake for 18-20 minutes until lightly golden. A toothpick inserted in the muffin should come out clean. Once removed from the oven, allow to cool in pan for 5 minutes. Remove muffins from the pan and cool on wire racks.