 |
 |
|
|
Sense & Respond, Program Manager, Light-Armored Vehicles, United States Marine Corps We are carrying out an evaluation of the Program Manager, Light Armored Vehicles Sense & Respond Phase II demonstration prototype project. This evaluation was conducted by researchers from the College of Information Sciences & Technology at Penn State University. The evaluation period was from the project's inception in the spring of 2005 through May of 2007. The objectives of the evaluation project included the following: 1. Research
and development of a metrics framework for Sense & Respond and
similar ground vehicle autonomic logistics systems. The Sense & respond evaluation project has resulted in development of a rich, empirically-grounded metrics framework for assessing, predicting, and designing to the information needs of ground vehicle operators, maintainers, commanders, and analysts. The metrics framework is, we believe, unique in that it is tailored to specific roles, scenarios, and system components and how they come together in support of the Marine Corps mission and priorities. We were instrumental in the development of a set of scenarios describing how Sense & Respond is designed to be used in LAV operations. These scenarios were, we believe, essential to help project partners communicate and design to the requirements of the system. Going forward these scenarios should be revisited and refactored during the S&R Phase II demonstration. The evaluation methodology and associated software tools provide a resource for future evaluations of very large-scale distributed systems for ground vehicles and other Marine Corps platforms. Central to the methodology is the use of a metrics framework mapped to roles, scenarios, and components. The methodology includes both analytical and empirical techniques and focuses especially on the utility (benefits, costs, risks) of a systems project relative to the organizational mission. The evaluation project experienced mixed success with respect to its core objective of providing both subjective and objective measures of Sense & Respond project success. While we are able to provide a relatively thorough subjective account of the project, our objective measures are lacking. This is primarily a function of the very limited amount of time the system was actually available to end-users for trial. Still, we believe that the evaluation results reported here represent very valuable input to the future Sense & Respond and similar projects. About 100 project stakeholders have been interviewed about the utility of the Sense & Respond system and the most important benefits and risks attending the project. Finally, based
largely on extensive study of project stakeholder perceptions of
and experiences with Sense & Respond Phase II, we are able to
provide recommendations for the project going forward. Some of these
are included in this draft report but the full set of recommendations
is still pending analysis of data gathered late in the project lifecycle. |