Defense Innovation Scores Big: The Surprising Truth About Its Impact

How Can We Measure if Defense Innovation Works?

Safe

• Source Information:
• My position as a professor of educational methodology is a unique one within professional military education: Rather than substantive expertise in topics like national security, military operations and tactics, I contribute my expertise in the scholarship of teaching and learning. I explore questions like, “How does game-based learning develop students’ strategic thinking skills?” and, “How is the seminar learning environment influenced by different student demographics?” These are the types of answerable questions that are missing from recent discussions around structural reforms, instructional strategies, and like topics.
• Jim Golby argues that professional military institutions should emphasize applied social science research. I agree, but our efforts to achieve “intellectual overmatch” in professional military education systems could be further bolstered if we also include applied research on professional military education itself. Doing so would allow the U.S. military’s schoolhouses to make evidence-based curricular, instructional, and even administrative decisions.

Addressing the Challenges in Measuring Defense Innovation

The lack of meaningful measurement in defense innovation is a significant challenge in the field.

• Applied research is actionable scholarly inquiry. Its experimental design follows the scientific method: posing a research question, sharing a hypothesis, testing that hypothesis, analyzing data, and communicating results. Applied research is rigorous and transparent in its methods, and because of this its findings are not only testable but also often reproducible.
• Applied research helps us to understand experiences, behaviors, and relationships beyond discrete, highly controlled variables and measured effects. Context is key. One has only to read RAND’s recent report on culture and competition across the military services to understand that no two professional military education institutions, classrooms, or (especially!) individual students are identical.
• Further, it is unwise to relegate an understanding of what goes on within military learning environments solely to the analyses often seen as part of standardized institutional assessments like end-of-course surveys or student evaluations of teaching. These surveys yield reactionary data. Rather, they reflect students’ satisfaction with any given quality of a learning event (most commonly, teacher performance). Student evaluations of teaching have also been increasingly exposed as biased against educators identifying as women as well as racial and ethnic minorities. While it is important for faculty and schoolhouse leaders to know if students are satisfied with their educational experiences, it is more important to know what and how they learned from these experiences. Asking students to rate their own knowledge of curricular concepts in end-of-course surveys is a shallow data point that is more helpful as a reflection activity than as a measure of learning.
• Instead, we can use applied educational research to capture students’ true demonstration of learning through their own behaviors and dialogue, including the quality and vocabulary of the questions they ask. In this way, applied educational research can supplement the data we are already capturing from experiential learning activities and formative and summative assessments like tests, practical exercises, and capstone requirements.

Real-World Applications of Applied Research in Defense Innovation

The application of applied research in defense innovation offers several real-world examples.

• The U.S. Navy has used applied research to develop advanced sensors and communication systems. For example, the Navy’s Advanced Sensor Suite (ASS) has been tested in various environments and has demonstrated improved performance over traditional sensors. This success can be attributed to the use of applied research in sensor design and testing.
• The U.S. Air Force has used applied research to develop advanced fighter aircraft systems. For example, the F-35 Lightning II has been tested in various environments and has demonstrated improved performance over traditional fighter aircraft. This success can be attributed to the use of applied research in aircraft design and testing.
• The U.S. Army has used applied research to develop advanced logistics systems. For example, the Army’s Logistics Modernization Program (LMP) has used applied research to develop advanced logistics systems that can improve supply chain efficiency and reduce costs. This success can be attributed to the use of applied research in logistics system design and testing.

In each of these examples, applied research has played a critical role in driving innovation and improving performance. These examples demonstrate the potential of applied research in defense innovation.

Conclusion

In conclusion, the lack of meaningful measurement in defense innovation is a significant challenge in the field. To address this challenge, we need to incorporate applied research into defense innovation efforts. This can be achieved through the use of regular, methodologically rigorous experiments in the classroom to test educational strategies. By doing so, we can better understand what works and what does not in defense innovation, and develop more effective solutions to drive innovation and improve performance.