Car Insurance Costs
From what I understand, using Amazon's in-car delivery should not affect your insurance rates. Because the couriers don't drive your vehicle(s) and are simply dropping items off in a parked car, I don't see any concerning factors an insurance company would worry about — unless a courier vandalizes or steals your car, which seems unlikely. If you have any other questions, don't hesitate to ask.
When you're delivering food, you're effectively using your vehicle for a commercial purpose. Most policies do not cover you to do a side job, such as rideshare or food delivery, without a special endorsement.
In most cases, making deliveries in your personal vehicle would be considered a commercial risk. Even if it is only for a short period, you may not be covered if an accident occurs.