Which term is used in u. s. politics for a significant party outside the two dominant parties?

This would be a third party. With examples like the Reform, Green, and Libertarian parties, these are groups that are typically very influential in local, state, and national politics. While their influence at the top levels of government is lower, they do have the ability to wield power in local and state governments.