Methodology
The Civilian Protection Monitor is the first public monitoring dashboard for state policy and practice on civilian harm mitigation and response, based on in-depth qualitative analysis of open source information.
Our analysis is currently focused on the military operations of three countries – the United States, the United Kingdom, and the Netherlands – since the beginning of the so-called ‘War on Terror’ in 2001. These three states were chosen on the basis of their significant military contributions to recent campaigns, including the anti-ISIS coalition in Iraq and Syria, their stated commitments to CHMR, and important lessons emerging from each context on best practices and practices to avoid. We are looking to expand our analysis to include other states in the future.
CPM has three main outputs. The CPM website presents an accessible overview of each of the three monitored countries and their scores across five indicators, from civilian harm tracking, to amends and response for harm. For more detailed information, the country reports contain a comprehensive narrative analysis of the good practices and current gaps, as well as recommendations for concrete actions states can take to improve their performance. A list of all the sources used for our analysis can be found here. Finally, in our news section, we provide up-to-date analysis on developments in policy and practice.
How we work
The evaluation framework
The Civilian Protection Monitor is a collaborative partnership between Airwars and PAX, building on an evaluation framework originally developed by Center for Civilians in Conflict (CIVIC) that has been adapted for practical application. The adapted CPM framework and the country-specific analysis has been peer reviewed by NGOs, subject matter experts, academics, and practitioners to ensure it is fair, comprehensive, and accurate.
All states monitored by CPM were given the offer of a briefing from the CPM team on the framework and our country-specific findings. They were also invited to fact-check the analysis in detail to ensure it accurately reflects their processes.
CPM analysis is based on publicly available sources only. This is to ensure that all analysis is verifiable and replicable. It also ensures that the policies, doctrine, and guidance that are included in analysis have gone through national approval processes and therefore represent a consolidated national approach.
CPM measures state policy and practice across five indicators (each of which have 3-5 sub-indicators). These are:
- The quality of national policies on civilian harm mitigation and response
- States’ practices of tracking civilian harm potentially caused by their operations
- States’ methods for investigating alleged cases of civilian harm
- How transparent states are about their practices,
- How states ensure victims from their military actions have access to redress for harm.
The current indicators were chosen because they are key foundational aspects of CHMR. No state can, for instance, hope to mitigate harm to civilians or respond to harm that has already been caused, if it does not effectively track and investigate allegations of harm. The current indicators have a focus on transparency on civilian harm, and what it tells us about the way each state recognises, responds to, and learns from civilian harm. Future indicators in the CPM will have a stronger focus on monitoring and analysing operational mitigation mechanisms themselves; yet as it stands, stronger transparency is needed across the US, UK, and Netherlands for CPM to be able to assess the nature and quality of such mitigation measures based on open-source material.
The indicators recognise the interconnection between policy and practice, but in scoring and evaluating indicators, we have acknowledged policies and guidance even before they are fully implemented; and conversely, analysed and scored state practices that are not yet fully codified.
Each state is scored according to the CPM leadership framework which ranges from Leader, Emerging Leader, Engaged, Uncommitted, to Regressive. The overall state score is based on the scores of indicators and sub-indicators.
The research process
CPM is based on an intensive research and review process. We review and update CPM analysis and scores according to an annual cycle, taking into account changes made in policy and practice during the calendar year (from January-December), with each new report launched in Q1 of the following year.*
Updating our analysis on an annual basis allows states time to take significant steps between versions of our country reports, and provides an insight into developments (positive or negative) over the long term. In between the annual updates, we conduct up-to-date analysis of changes to policy and practice in our News section.
* In the first year of our publication, this timeline looks slightly different. The cut-off point of our analysis in 2025 was February 1st.
