Will the bulb light for the whole time that the capacitor discharges?

Answer: Yes

The capacitor in an RC circuit (Resistor-Capacitance circuit) creates a time delay that decays exponentially. Therefore when the voltage to the bulb is removed, it takes a long time for the capacitor to completely discharge.

Although the brightness of the bulb will diminish as the capacitor discharges, the bulb will still be lit until the capacitor discharges to a voltage that is too low to sustain visible light (to the human eye) from the bulb.