Pavers do not affect the acidity of the soil, so you are pretty safe in planting there. You will have to send us a picture when they grow over and create your canopy, I am looking forward to seeing it.