I used it when it was still called MS Live Cashback. Used it three times. Twice, I got my money. The third time, I never got the Cashback, and to date I don't know why. I asked MS Cashback, they took 10weeks to reply. MS said it was eBay's fault for not sending the info. eBay said it was PayPal for not sending the info. PayPal said it was MS not honoring the Cashback. MS then replied the purchase wasn't eligible because it didn't meet eligibility requirements. When I inquired as to which requirements the reply was that the ineligibility was due the lack of eligibility.
Got that?
I figured I had wasted enough time at this point. I missed out on $16.85. Oh well.
Most people have had success in getting their money. I estimate about 5% of people shopping on eBay run into problems that's not their fault. Another 5% of people also have problems, but that's because they didn't read the fine print, such as not adding another coupon, Buy It Now type auctions etc. Merchants other than eBay seem to have fewer issues, although there are no guarantees. The problem is that this is often difficult to figure out who's at fault - MS or the merchant or any of the other parties involved.
So my advice is to use it as a bonus, not a deciding factor on whether to purchase something. As a protection, you may want to use a credit card, and if you don't get the cashback notification within an hour of the purchase cancel the order and retry.
I have not used it recently, so things may have improved. If cashback ever hits 15% on eBay, I might just go for the a cyclone dust collector