Which system would you install to provide active protection and notification of security problems in a network connected to the internet?

I guess the correct answer is intrusion detection system (IDS).

An intrusion detection system (IDS) provides active watching and rule-based responses to uncommon activities on a network. A firewall provides passive security by preventing access from unauthorized traffic. If the firewall were compromised, the IDS would inform you supported rules it's designed to implement.