I would use automotive weatherstrip cement. It's flexible and sticks to both rough and smooth surfaces extremely well. Permatex is the one that I have, but others make it too.
PL Premium Urethane Construction Adhesive - It will glue anything - forever. Get it at Home Depot at $8 a tube. Don't get anything else, like PL 400, etc. It has to be PL Premium.
Try this website to see what is best for your purpose. I've used it several times with good results. I don't know why I didn't suggest it in my first response.