How do I stop my dog from soiling his crate??!!?

I have 3 dogs and recently I started walking them twice a day. We all need the excercise and one is a puppy so I wanted to teach him to walk on a leash. I have noticed that now, about 3 or 4 weeks since we started this routine they all seem to poop while walking. I think the excercise helps their bowels to move and I recommend 2 15-20 min walks per day. Once in the morning after breakfast and then in the evening after dinner.(We have a yard and I'd much rather they would go to bathroom at home, but the benefits of excercise for all of us is worth the inconvenience of picking up poop off the sidewalk.)