In either Survival or Creative mode, when you destroy a block that's holding an item frame, the item frame will only drop after 4 or 5 seconds. This happens whether the item frame is holding an item/block or empty.