Preheat oven to 350°. Cream together butter and sugar. Add the egg, lemon zest, lemon juice, extract, and buttermilk, and mix well.
In a separate bowl, whisk together flours, baking powder, salt, and almond flour until combined. Add the dry ingredients to creamed mixture and stir just until moistened. Gently fold-in poppy seeds.
Fill muffin cups 3/4 full and bake for 18-20 minutes until lightly golden. A toothpick inserted in the muffin should come out clean. Once removed from the oven, allow to cool in pan for 5 minutes. Remove from pan and cool on wire racks. Cool completely before frosting.
For the Frosting:
Beat butter and cream cheese together until fluffy. Add-in the vanilla and stir to combine. Mix-in powdered sugar on low speed until incorporated. Pipe onto cupcakes and serve immediately.