Methodology

Methodology

Jan 24, 2024 13 min read

Cpl. Daniel Hopping, assaultman, Weapons Company, 1st Battalion, 7th Marine Regiment, and a native of Rogers, Arkansas, in Helmand province, Afghanistan, April 28, 2014. U.S. Marine Corps photo by Cpl. Joseph Scanlan/Released

Methodology

The assessment portion of the Index of U.S. Military Strength is composed of three major sections that address America’s military power, the operating environments within or through which that power must be employed, and threats to America’s vital national interests.

The authors of this study used a five-category scoring system that ranged from “very poor” to “excellent” or “very weak” to “very strong” as appropriate to each topic. They chose this approach because it captures meaningful gradations while avoiding the appearance that a high level of precision is possible given the nature of the issues and the information that was publicly available.

Some factors are quantitative and lend themselves to discrete measurement; others are qualitative in nature and can be assessed only through an informed understanding of the material that leads to a judgment call. Further, because conditions in each of the areas assessed are changing throughout the year, any measurement must necessarily be based on the information at hand and viewed as a snapshot in time. We understand that this is not entirely satisfactory when it comes to reaching conclusions on the status of a given matter (especially the adequacy of military power) and will be unsatisfactory for some readers, but we also understand that senior officials in decision-making positions will never have a comprehensive set of inarguable hard data on which to base a decision.

Purely quantitative measures alone tell only part of the story when it comes to the relevance, utility, and effectiveness of hard power. In fact, using only quantitative metrics to assess military power or the nature of an operating environment can lead to misinformed conclusions. Raw numbers are a very important component, but they tell only a part of the story of war. Similarly, experience and demonstrated proficiency are often decisive factors in war, but they are also nearly impossible to measure.

The assessment of the global operating environment in this Index focuses on three key regions—Europe, the Middle East, and Asia—because of their importance relative to U.S. vital economic, diplomatic, and security interests.

For threats to U.S. vital interests, the Index identifies the countries that pose the greatest current or potential threats to U.S. vital interests based on two overarching factors: behavior and capability. The classic definition of “threat” considers the combination of intent and capability, but intent cannot be clearly measured. Therefore, observed behavior (including historical behavior and explicit policies or formal statements vis-à-vis U.S. interests) is used as a reasonable surrogate because it is the clearest manifestation of intent. The countries selected according to these criteria are scored in two areas:

  • The degree of provocative behavior that they exhibited during the year.
  • Their ability to pose a credible threat to U.S. interests irrespective of intent.

Finally, the status of U.S. military power is addressed in three areas: capability (or modernity), capacity, and readiness. All three are fundamental to success even if they are not de facto determinants of success (something we explain further in the section). Also addressed is the condition of America’s nuclear weapons capability, which is assessed in areas that are unique to this military component and critical to understanding its real-world viability and effectiveness as a strategic deterrent. Though they are not scored according to the stated metrics, the chapter on military power includes explanatory overviews of U.S. missile defense and cyber.

Assessing the Global Operating Environment

Not all of the factors that characterize an operating environment are equal, but each contributes to the degree to which a particular operating environment is favorable or unfavorable to future U.S. military operations. In assessing the operating environment, four regional characteristics that are of greatest relevance to the conduct of military operations were considered, and we used a five-point scale that ranges from “very poor” to “excellent” conditions to describe our assessment of the aggregate condition for each region.

The key regional characteristics consisted of:

  1. Alliances. Alliances are important for interoperability and collective defense because allies are more likely than non-allies to lend support to U.S. military operations. Indicators that provide insight into the strength or health of an alliance include whether the U.S. trains regularly with countries in the region, has good interoperability with an ally’s forces, and shares intelligence with nations in the region.
  2. Political Stability. Political stability brings predictability when military planners are considering such things as transit, basing, and overflight rights for U.S. military operations. The overall degree of political stability indicates whether U.S. military actions would be hindered or enabled and reflects, for example, whether transfers of power are generally peaceful and whether there have been any recent instances of political instability in the region.
  3. U.S. Military Positioning. Having military forces based or equipment and supplies staged in a region greatly facilitates the ability of the United States to respond to crises and presumably achieve success in critical “first battles” more quickly. Being routinely present in a region also helps the U.S. to maintain familiarity with its characteristics and the various actors that might try to assist or thwart U.S. actions. With this in mind, we assessed whether or not the U.S. military was well-positioned in the region. Again, indicators included bases, troop presence, prepositioned equipment, and recent examples of military operations (including training and humanitarian operations) launched from the region.
  4. Infrastructure. Modern, reliable, and suitable infrastructure is essential to military operations. Airfields, ports, rail lines, canals, and paved roads enable the U.S. to stage, launch operations from, and logistically sustain combat operations. We combined expert knowledge of regions with publicly available information on critical infrastructure to arrive at our overall assessment of this metric.

The assessment scale included:

  1. Very Poor. Significant hurdles exist for military operations. Physical infrastructure is insufficient or nonexistent, and the region is politically unstable. The U.S. military is poorly placed or absent, and alliances are nonexistent or diffuse.
  2. Unfavorable. A challenging operating environment for military operations is marked by inadequate infrastructure, weak alliances, and recurring political instability. The U.S. military is inadequately placed in the region.
  3. Moderate. A neutral to moderately favorable operating environment is characterized by adequate infrastructure, a moderate alliance structure, and acceptable levels of regional political stability. The U.S. military is adequately placed.
  4. Favorable. A favorable operating environment includes good infrastructure, strong alliances, and a stable political environment. The U.S. military is well placed for future operations.
  5. Excellent. An extremely favorable operating environment includes well-established and well-maintained infrastructure; strong, capable allies; and a stable political environment. The U.S. military is exceptionally well placed to defend U.S. interests.

Assessing Threats to U.S. Vital Interests

To make the threats identified in this Index measurable and relatable to the challenges of operating environments and the adequacy of American military power, Index staff and outside reviewers, working independently, evaluated the threats according to their level of provocation (observed behavior) and their actual capability to pose a credible threat to U.S. interests on a scale of 1 to 5 with 1 representing a very high threat capability or level of belligerency. This scale corresponds to the tone of the five-point scales used to score the operating environment and military capabilities in that 1 is bad for U.S. interests and 5 is very favorable.

Based on these evaluations, provocative behavior was characterized according to five descending categories: benign (5); assertive (4); testing (3); aggressive (2); and hostile (1). Staff also characterized a threat actor’s capabilities according to five categories: marginal (5); aspirational (4); capable (3); gathering (2); and formidable (1). Those characterizations—behavior and capability—form the two halves of the overall threat level.

Assessing U.S. Military Power

Also assessed is the adequacy of the U.S. defense posture as it pertains to a conventional understanding of hard power, defined as the ability of American military forces to engage and defeat an enemy’s forces in battle at a scale commensurate with America’s vital national interests. The assessment draws on both quantitative and qualitative aspects of military forces, informed by an experience-based understanding of military operations and the expertise of the authors and internal and external reviewers.

It is important to note that military effectiveness is as much an art as it is a science. Specific military capabilities represented in weapons, platforms, and military units can be used individually to some effect. Practitioners of war, however, have learned that combining the tools of war in various ways and orchestrating their tactical employment in series or simultaneously can dramatically amplify the effectiveness of the force committed to battle.

The point is that the ability of a military force to locate, close with, and destroy an enemy depends on many factors, but relatively few of them are easily measured. The scope of this specific project does not extend to analysis of everything that makes hard power possible; it focuses on the status of the hard power itself.

This Index assesses the state of military affairs for U.S. forces in three areas: capability, capacity, and readiness.

Capability. Scoring of capability is based on the current state of combat equipment. This involves four factors:

  • The ages of key platforms relative to their expected life spans.
  • Whether the required capability is being met by legacy or modern equipment.
  • The scope of improvement or replacement programs relative to the operational requirement.
  • The overall health and stability (financial and technological) of modernization programs.

This Index focused on primary combat units and such combat platforms as tanks, ships, and airplanes and elected not to include the array of system and component upgrades such as a new radar, missile, or communications suite that keep an older platform viable over time. New technologies grafted onto aging platforms ensure that U.S. military forces keep pace with technological innovations that are relevant to the modern battlefield, but at some point, the platforms themselves are no longer viable and must be replaced. Modernized sub-systems and components do not entirely substitute for aging platforms, and it is the platforms themselves that are usually the more challenging items to field. In this sense, primary combat platforms serve as representative measures of force modernity just as combat forces are a useful surrogate measure for the overall military that includes a range of support units, systems, and infrastructure.

In addition, it is assumed that modernization programs should replace current capacity at a one-to-one ratio. Less than a one-to-one replacement assumes risk, because even if the newer system is presumably better than the older, until it is proven in actual combat, having fewer systems lessens the force’s capacity—an important factor if combat against a peer competitor carries with it the likelihood of attrition. For modernization programs, only Major Defense Acquisition Programs (MDAPs) are scored.

The capability score uses a five-grade scale. Each service receives one capability score that is a non-weighted aggregate of scores for four categories: (1) Age of Equipment, (2) Modernity of Capability, (3) Size of Modernization Program, and (4) Health of Modernization Program. General criteria for the capability categories are:

Age of Equipment

  • Very Weak: Equipment age is past 80 percent of expected life span.
  • Weak: Equipment age is 61 percent–80 percent of expected life span.
  • Marginal: Equipment age is 41 percent–60 percent of expected life span.
  • Strong: Equipment age is 21 percent–40 percent of expected life span.
  • Very Strong: Equipment age is 20 percent or less of expected life span.

Capability of Equipment

  • Very Weak: More than 80 percent of capability relies on legacy platforms.
  • Weak: 60 percent–79 percent of capability relies on legacy platforms.
  • Marginal: 40 percent–59 percent of capability is made up of legacy platforms.
  • Strong: 20 percent–39 percent of capability is made up of legacy platforms.
  • Very Strong: Less than 20 percent of capability is made up of legacy platforms.

Size of Modernization Program

  • Very Weak: Modernization program is significantly too small or inappropriate to sustain current capability or program in place.
  • Weak: Modernization program is smaller than current capability size.
  • Marginal: Modernization program is appropriate to sustain current capability size.
  • Strong: Modernization program will increase current capability size.
  • Very Strong: Modernization program will vastly expand capability size.

Health of Modernization Program

  • Very Weak: Modernization program faces significant problems; too far behind schedule (five-plus years); cannot replace current capability before retirement; lacks sufficient investment to advance; cost overruns include Nunn–McCurdy breach, which occurs when the cost of a new item exceeds the most recently approved amount by 25 percent or more or if it exceeds the originally approved amount by 50 percent or more.1
  • Weak: Modernization program faces procurement problems; behind schedule (three–five years); difficulty replacing current equipment on time or insufficient funding; cost overruns enough to trigger an Acquisition Program Baseline (APB) breach.2
  • Marginal: Modernization program faces few problems; one–two years behind schedule but can replace equipment with some delay or experience some funding cuts; some cost growth but not within objectives.
  • Strong: Modernization program faces no procurement problems; can replace equipment with no delays; within cost estimates.
  • Very Strong: Modernization program is performing better than DOD plans, including with lower actual costs.

Capacity. To score capacity, the Army, Navy, and Air Force (be it end strength or number of platforms) are compared to the force size required to meet a simultaneous or nearly simultaneous two-war or two–major regional contingency (MRC) benchmark. This benchmark consists of the force needed to fight and win two MRCs and a 20 percent margin that serves as a strategic reserve. The Marine Corps is handled a bit differently (see the explanatory note below and a more expanded discussion within its specific assessment).3 A strategic reserve is necessary because deployment of 100 percent of the force at any one time is highly unlikely. Not only do ongoing requirements like training or sustainment and maintenance of equipment make it infeasible for the entirety of the force to be available for deployment, but committing 100 percent of the force would leave no resources available to handle unexpected situations.

Thus, a “marginal” capacity score would exactly meet a two-MRC force size, a “strong” capacity score would equate to a plus-10 percent margin for strategic reserve, and a “very strong” score would equate to a 20 percent margin.

Capacity Score Definitions

  • Very Weak: 0 percent–37 percent of the two-MRC benchmark.
  • Weak: 38 percent–74 percent of the two-MRC benchmark.
  • Marginal: 75 percent–82 percent of the two-MRC benchmark.
  • Strong: 83 percent–91 percent of the two-MRC benchmark.
  • Very Strong: 92 percent–100 percent of the two-MRC benchmark.

Readiness. The readiness scores are derived from the military services’ own assessments of readiness based on their requirements. For many reasons—not least of which is concern about informing a potential enemy’s calculations on sensitive, detailed aspects of a force’s readiness for combat—the services typically classify their internal readiness reporting. However, they do make some public reports, usually when providing open testimony to Congress. Thus, instead of delving into comprehensive reviews of all readiness input factors, the Index relies on the public statements of the military services regarding the state of their readiness, caveated to an extent by the accumulated personal experience of the authors and noted as such in the text when appropriate.

It should be noted that even a “strong” or “very strong” score does not necessarily indicate that 100 percent of the force is ready; it indicates only that the service is meeting 100 percent of its own readiness requirements. Often, these requirements assume that a percentage of the military at any one time will not be fit for deployment. Thus, even if readiness is graded as “strong” or “marginal,” there is still a gap in readiness that will have significant implications for immediate combat effectiveness and the ability to deploy quickly. Anything short of meeting 100 percent of readiness requirements therefore assumes risk and is problematic.

In addition, a service’s assessment of its readiness occurs within its size or capacity at that time and as dictated by the Defense Strategic Guidance, National Military Strategy, and related top-level documents generated by the Administration and senior Defense officials. It does not account for the size-related “readiness” of the force to meet national security requirements, which is assessed as needed by this Index. Consequently, for a service to be assessed as “very strong” would mean that 80 percent–100 percent of its existing force meets its requirements for being “ready” even if its size is less than the size required to meet the two-MRC benchmark. It is important that the reader keep this in mind when considering the actual readiness of the force to protect U.S. national security interests against the challenges presented by threats around the world.

Readiness Score Definitions

  • Very Weak: 0 percent–19 percent of service’s requirements.
  • Weak: 20 percent–39 percent of service’s requirements.
  • Marginal: 40 percent–59 percent of service’s requirements.
  • Strong: 60 percent–79 percent of service’s requirements.
  • Very Strong: 80 percent–100 percent of service’s requirements.

Endnotes

[1] 10 U.S. Code Ch. 325: Cost Growth–Unit Cost Reports (Nunn–McCurdy), § 4371(a)(3)(A)(1) and (B)(1), https://uscode.house.gov/view.xhtml?req=granuleid%3AUSC-prelim-title10-chapter325&edition=prelim (accessed August 9, 2023).

[2] 14 U.S. Code § 1135: Acquisition Program Baseline Breach, https://uscode.house.gov/view.xhtml?req=granuleid:USC-prelim-title14-section1135&num=0&edition=prelim (accessed August 9, 2023).

[3] As noted in the introduction to the chapter assessing military power, the three large services (Army, Navy, and Air Force) are sized for global action in more than one theater at a time. The Marines, by virtue of overall size and most recently by direction of the Commandant, focus on one major conflict while ensuring that all Fleet Marine Forces are globally deployable for short-notice, smaller-scale actions. Having assessed that the Indo-Pacific region will continue to be of central importance to the U.S. and noting that China is a more worrisome “pacing threat” than any other competitor and that the Joint Force lacks the ability to operate within the range of intensely weaponized, layered defenses featuring large numbers of precision-guided munitions, the Corps is reshaping itself to optimize its capabilities and organizational structures to meet this challenge. This Index concurs with this effort but assesses that the Corps will still need greater capacity to succeed in war in the very circumstances for which the Marines believe they must prepare. Consequently, we assess the Marine Corps’ capacity against a one-war metric.