While buttermilk and olive oil moisten this cake, lemon juice and zest add bold lemon flavour. Lemon olive oil cake for dessert or morning coffee! Cake with olive oil? Yes! Olive oil keeps baked goods fresh for days. Simple desserts like this lemon olive oil cake are more interesting with olive oil's complexity than butter or vegetable oil.