I am presuming if they are living in a dovecote, then they are just ornamental which are allowed their liberty at will. It would be much different if they lived in a loft as you could allow them their liberty at certain times and obviously adjust their feed accordingly, which probably is the problem.
I have not had much dealings with roller pigeons so I could not possibly advise you, but I wouldnt really think taking them away to toss them would really be a good idea.