Mrs. Carmody's followers are a congregation of the survivors in the grocery store, loyal to Mrs. Carmody due to their minds snapping under the pressure of the siege mentality caused by the mist, as opposed to David's group.
The group is lead by Mrs. Carmody, and attempts to sacrifice "un-believers" for holy salvation. It only had a few members other than Mrs. Carmody herself at first, but its number of members rapidly increased as the siege mentality, and/or terrifying encounters in the mist that seemingly followed her preachings, caused more of the survivors' minds to snap.
With Mrs. Carmody's death, her followers snapped back to reality, the congregation broke up, and they allowed David's group to leave (and later they allowed to re-enter one of them back into the store without the attempt to harm him). One of them, however, screamed at Ollie, calling him a murderer, suggesting that not all of the followers snapped back to reality and that some of them were still devoted to her.
Most or all of the ex-followers were apparently rescued from the market, as people from the market, many of them Mrs. Carmody's ex-followers, are very briefly glimpsed aboard the rescue trucks at the end.