You don't have to move them, but I would consider attaching a raised area to your foundation, adding a top layer of better soil, and replanting. That is where I grow my blueberries.