UN and Red Cross warn machines are close to targeting humans on their own

The United Nations and the International Committee of the Red Cross warned on Tuesday that states are close to allowing machines to select and strike human targets without a person making the final decision. In a joint appeal issued in Geneva, Secretary-General António Guterres and ICRC President Mirjana Spoljaric Egger said the gap between what weapons can now do and what international law currently constrains has widened since their last joint call in 2023.

“We are now dangerously close to crossing a moral red line: the autonomous targeting of humans by machines,” they said. The statement stops short of a blanket ban. It asks governments to negotiate a legally binding instrument with two clear limits: a prohibition on systems that target people, and restrictions on systems whose effects an operator cannot reasonably predict.

Talks on the issue have run for years under the Convention on Certain Conventional Weapons. A group of governmental experts will wrap a three-year mandate in Geneva next week. Guterres and Spoljaric want that process to feed a negotiation that would start in November, when a review conference that meets only once every five years opens in the same city. Russia and the United States have opposed new binding rules, arguing that existing humanitarian law already covers these systems.

What the two organisations want banned

Laurent Gisel, who heads the ICRC’s Arms and Conduct of Hostilities Unit, set out the first category in briefings around the appeal. It covers weapons that, by design or in use, leave an operator unable to anticipate the outcome of force. “If it's not possible to predict it, then how is it possible to limit it?” he asked. The concern is not science fiction. Machine-learning models that adapt after launch can change how they classify objects. A system that behaved one way in testing can behave another way in smoke, dust, electronic interference or a crowded city.

The second category covers anti-personnel systems. Gisel said states already accept that autonomy raises ethical questions because it reduces human control over force. Those questions sharpen when the object of the algorithm is a person. Life is then reduced to data, sensors and machine operations. Existing law already forbids attacks on civilians. The ICRC’s point is that a machine that cannot distinguish a fighter from a bystander, or that cannot weigh surrender, wounded status or the presence of children, cannot apply that law.

Neither organisation confirmed that a fully autonomous system has already killed a person on a battlefield. They did note unconfirmed reports of AI-guided drones in Ukraine, Sudan, Gaza and the Gulf. Semi-autonomous loitering munitions and long-range attack drones are already in daily use. A Ukrainian FP-1 drone, photographed on 16 August, is the kind of system now common on both sides of that war. The next step, they argue, is a system that chooses the target as well as flying to it.

Why the 2023 deadline passed

Three years ago Guterres and Spoljaric asked states to set prohibitions and restrictions by 2026. That year is now almost over. The appeal was not followed by a treaty. Arms programmes continued. The joint statement on Tuesday said development is proceeding at “record speed” and that a race toward greater autonomy is already underway. Scientists and engineers who build the systems have asked for limits. The two organisations said that if the people who write the code want guardrails, governments cannot treat the issue as theoretical.

The legal problem is simple to state and hard to legislate. International humanitarian law requires distinction, proportionality and precautions in attack. Those duties sit with commanders and operators. If a system selects a target after it leaves human hands, the chain of responsibility frays. A state that fielded such a weapon could claim the machine made the choice. A victim would have no clear person to hold to account. That is the “moral red line” the statement names.

The appeal also points to civilian infrastructure. Autonomous systems that hunt radar, vehicles or heat signatures can strike hospitals, power plants or apartment blocks if those objects share a signature with a military target. The organisations said the risk of harm to civilians and civilian sites has grown as these tools have moved from laboratories onto active fronts.

The November window

The November review conference is the nearest large diplomatic date on the calendar. Guterres and Spoljaric called for “political courage” and said incremental discussion is no longer enough. “A failure to act quickly and decisively will cost lives,” the statement said. “Innovation must serve humanity, not endanger it.”

Whether that conference produces a mandate to negotiate will depend on the same split that has stalled the expert group. A large group of states, including many in the global south and several European governments, support a ban on systems that target humans. Major military powers prefer voluntary codes, best-practice manuals or an argument that current law is sufficient if commanders stay “in the loop” at some level. The phrase “in the loop” is itself contested. A human who can abort a strike in theory may not be able to do so in the seconds a drone has to act.

The ICRC has not asked states to freeze research. It has asked them to draw a line before machines are assigned the task of choosing who dies. That line does not exist in treaty form today. The joint appeal is an attempt to force the question onto the agenda while the technology is still, in the organisations’ own words, short of confirmed use against people. Once that use is documented, the politics of a ban become harder. Tuesday’s statement is written as a last warning before that documentation arrives.

States that want rules now have a date. Those that do not will treat November as another round of statements. The weapons programmes will not wait for either outcome.