Prove that a square matrix is invertible if and only if its columns are linearly independent

Well, if the system is linearly dependent, then there's a vector that's "redundant". That is, it's a linear combination of the other vectors in the system. When you remove it, it is no longer a square matrix, and thus, not invertible.