The establishment clause would prevent

The establishment clause would prevent "the mayor's office from displaying a cross".

Explanation:

In the law of the United States, the establishment clause of the First Amendment to the Constitution of the United States, together with the Free Exercise Clause of that amendment form the constitutional right to religious freedom. The Establishment Clause is a limitation imposed on the US Congress.

These limitations are like:

Illegal to promote theocracy or specific religion with taxes. Prohibits the Free Exercise of Religion by Government. Prohibit Congress from favoring one religion or elevating it over another.