The Indian government has instructed Meta to implement human review systems in its AI-enabled content moderation processes, according to sources familiar with the matter. During recently concluded meetings with senior Meta executives, including Chief of Global Affairs Joel Kaplan, officials from the Ministry of Electronics and Information Technology (Meity) expressed concerns about the absence of human reviewers in the tech giant's AI-driven content moderation systems. As reported by Business Standard, the government warned that keyword-based takedowns can remove legitimate posts without proper contextual understanding. The issue came up during meetings between August 5-10, following the temporary removal of a Facebook and Instagram post by Prime Minister Narendra Modi during student protests in Delhi.
According to government officials present at the meetings, Meta's AI systems have been trained to detect only 'static' keywords to flag and remove content across all three of the company's platforms. A senior government official explained that this approach can lead to any content being taken down even if it does not violate laws or community guidelines. The government cited a specific example where a government department shared details of law enforcement operations against pro-Khalistan fugitives, but the post was automatically removed by AI systems due to the presence of the word 'pro-Khalistan' without understanding the context. Without the AI system understanding the context, the official said, the post was taken down because of the presence of the term, with a human reviewer potentially preventing the error.
During meetings between August 5-10, Meta executives, including Kaplan, apologized to Union Electronics and Information Technology Minister Ashwini Vaishnaw for the inadvertent 5-hour takedown of Prime Minister Narendra Modi's video addressing students protesting the leak of NEET exam question papers. According to Business Standard, Kaplan was accompanied by Neil Potts, vice-president of public policy at Meta, Rafael Frankel, head of Asia Pacific public policy, and Aman Jain, head of public policy in India. The meetings covered multiple aspects including Child Sexual Exploitative and Abuse Material (CSEAM), deepfake content proliferation, and issues raised by the Grievance Appellate Committee. Meta's global affairs chief Joel Kaplan apologised to Electronics and Information Technology Minister Ashwini Vaishnaw over the restriction.
Meta's content moderation system combines automated technology with human review, with the company's technology helping determine which content should be reviewed by people and in what order. According to Meta's blog explaining its review process, when its systems are highly confident that content violates its rules, they can remove it automatically. Where there is less certainty, the content can be prioritised for review by human teams. The company's automated systems are trained to identify content that may violate its Community Standards, with the process not relying only on a fixed list of keywords but assessing different elements of a post to determine whether they indicate a violation. Meta uses training systems that learn from millions of pieces of current content to help select training data aligned with the company's objectives, improving its ability to detect areas such as hate speech and content that incites violence.
The government's concerns extend beyond Meta to other intermediaries, with officials noting that lack of human agents in AI decision-making systems has been observed across other platforms such as X, which restricts access to posts containing banned keywords. As reported by Business Standard, officials stated they are studying instances where posts have been taken down solely by AI-enabled content moderation systems and may take action based on their findings. The government indicated it might reach out to other intermediaries if necessary to address these systemic issues. Meta has also acknowledged that some moderation decisions can be 'extremely nuanced and contextual', citing bullying as an example where reviewers may need to establish who is being targeted and whether the content is intended to cause harm.
