Why couldn't the pilgrims attract more settlers?

The main reason pilgrims often have hard times attracting followers is the fear of change.

It was a hard life in the starting United States, there were tough conditions. but pilgrims had a will to survive, more settlers most likely did not want to go because exploring out and leaving everything behind was tough for some people