cell membranes are made of phospholipids which form a hydrophobic barrier. How does water pass through the cell membrane?

Water is a small molecule that easily diffuses through a cell membrane despite the lipid tails. The diffusion of water is what we call "osmosis". Oxygen is a small molecule and it is also nonpolar, so it easily passes through a cell membrane.

It is able to pass through because of the high osmotic pressure difference between the inside and outside of the cell.

Most of the water passes through channels called "aquaporins".