Is it OK to connect to the power supply without installing the battery
Yellow wire labeled "battery" should have constant 12 volts, whether the car is on or off. The "bulb with a wire in the middle" is probably a fuse. (Any direct connection to the battery will normally be fused). "Memory", as you correctly guess, provides a constant power to the stereo head that retains presets etc.