A Channel 4 Dispatches investigation claimed that his links with the radical group had helped him become the Labour leader of Tower Hamlets council, a position he was ousted from in May 2010.
A Labour spokesman said the decision had been made because of "a number of serious allegations concerning both the eligibility of participating voters and the conduct of Mr Rahman.
“Administrative action has been taken to remove [him] as a candidate pending the investigation."
Tower Hamlets will vote for its first directly-elected mayor next month.
Helal Abbas Uddin, also a Spitalfields and Banglatown councillor, will run in Mr Rahman’s place.