Civilian Protection Monitor

Methodology

The Civilian Protection Monitor is the first public monitoring dashboard for state policy and practice on civilian harm mitigation and response, based on in-depth qualitative analysis of open source information. 

Our analysis is currently focused on the military operations of three countries – the United States, the United Kingdom, and the Netherlands – since the beginning of the so-called ‘War on Terror’ in 2001. These three states were chosen on the basis of their significant military contributions to recent campaigns, including the anti-ISIS coalition in Iraq and Syria, their stated commitments to CHMR, and important lessons emerging from each context on best practices and practices to avoid. We are looking to expand our analysis to include other states in the future.

© Levi Meir-Clancy/Unsplash, 2021.

How we work


The review period

The CPM country analysis is based on information released publicly within the year under review. For example, the NL assessment for its commitment to civilian protection in 2025 will be judged on documents available within that same calendar year. 

Our analysis focuses on states and militaries’ policies and doctrine which support their efforts to mitigate and respond to civilian harm. This includes the relevant policy framework, civilian harm tracking capabilities, the ability to conduct effective investigations into evidence of harm, the response mechanisms that have been established, and a states’ public reporting. This reflects an underlying principle that having systems in place to understand and respond to civilian harm is an essential marker of good practice when it comes to civilian protection.

The CPM will make certain exceptions where clarification of country policies and practices are released during the months in which the reports are drafted (generally January-April the following year) when these relate to practices in 2025. This will not include any analysis of military campaigns launched during this time or new policies/documents released that are prospective in nature. 

Any major reporting that is likely to shift the overall assessment of countries in question during the calendar year, will be reserved to the following annual civilian protection report. 

The evaluation framework

The Civilian Protection Monitor was co-developed by Airwars and PAX, building on an evaluation framework originally developed by Center for Civilians in Conflict (CIVIC), which has been modified for practical application. The adapted CPM framework and the country-specific analysis has been peer reviewed by NGOs, subject matter experts, academics, and practitioners to ensure it is fair, comprehensive, and accurate.

All states monitored by CPM were given the offer of a briefing from the CPM team on the framework and our country-specific findings. They were also invited to fact-check the analysis in detail to ensure it accurately reflects their processes. 

CPM analysis is based on publicly available sources only. This is to ensure that all analysis is verifiable and replicable. It also ensures that the policies, doctrine, and guidance that are included in analysis have gone through national approval processes and therefore represent a consolidated national approach.

CPM measures state policy and practice across five indicators (each of which have 3-5 sub-indicators). These are:

  1. The quality of national policies on civilian harm mitigation and response
  2. States’ practices of tracking civilian harm potentially caused by their operations
  3. States’ methods for investigating alleged cases of civilian harm
  4. How transparent states are about their practices, 
  5. How states ensure victims from their military actions have access to redress for harm.

The current indicators were chosen because they are key foundational aspects of CHMR. No state can, for instance, hope to mitigate harm to civilians or respond to harm that has already been caused, if it does not effectively track and investigate allegations of harm. The current indicators have a focus on transparency on civilian harm, and what it tells us about the way each state recognises, responds to, and learns from civilian harm. Future indicators in the CPM will have a stronger focus on monitoring and analysing operational mitigation mechanisms themselves; yet as it stands, stronger transparency is needed across the US, UK, and Netherlands for CPM to be able to assess the nature and quality of such mitigation measures based on open-source material.  The indicators on civilian harm tracking and investigations do capture evidence of civilian harm, providing an important indication on whether a state is effectively mitigating harm. 

The indicators recognise the interconnection between policy and practice, but in scoring and evaluating indicators, we have acknowledged policies and guidance even before they are fully implemented; and conversely, analysed and scored state practices that are not yet fully codified. 

Each state is scored according to the CPM leadership framework which ranges from Leader, Emerging Leader, Engaged, Uncommitted, to Regressive. The overall state score is based on the scores of indicators and sub-indicators. 

The research process

CPM is based on an intensive research and review process. We review and update CPM analysis and scores according to an annual cycle, taking into account changes made in policy and practice during the calendar year (from January-December), with each new report launched in the following year. 

Updating our analysis on an annual basis allows states time to take significant steps between versions of our country reports, and provides an insight into developments (positive or negative) over the long term. In between the annual updates, we conduct up-to-date analysis of changes to policy and practice in our News section.