Yes, dirty solar panels restrict the energy from sunlight from being absorbed by the PV cells within the panel. Bird poo, dust, grime, lichen, mould, leaf litter and more can quickly build up on and around your panels leading to damage and reduced efficiency over time.
If you imagine how well a dirty window blocks light from passing through, this is the same concept for dirty solar panels. A clean window lets the light in and the same is true for your panels.
Dirty panels can reduce the efficiency of your residential solar panel output by anywhere from 20-40% depending on the degree of build-up. If you want maximum cost-saving benefits from your solar panels, keeping them clean is key.