This content is not included in
your SAE MOBILUS subscription, or you are not logged in.
Designing the Design Space: Evaluating Best Practices in Tradespace Exploration, Analysis and Decision-Making
Technical Paper
2022-01-0354
ISSN: 0148-7191, e-ISSN: 2688-3627
Annotation ability available
Sector:
Language:
English
Abstract
Determining the validity of the design space early in the conceptualization of a project can make the difference between project success and failure. Early assessment of technical feasibility, project risk, technical readiness and realistic performance expectations based on models with different levels of fidelity, uncertainty, and technical robustness is a challenging mission critical task for large procurement projects. Tradespace exploration uses model-based engineering analysis, design exploration methods, and multi-objective optimization techniques to enable project stakeholders to make informed decisions and tradeoffs concerning the scope, schedule, budget, performance and risk profile of a project. As the intersection with a number of project stakeholders, tradespace studies can provide a significant impact upon the direction and decision-making in a project. Yet, the act of studying the tradespace is data intensive, subject to variability, uncertainty complexity and ambiguity (VUCA). These properties of the data in the tradespace present challenges in both conducting tradespace studies but also in presenting the results of tradespace studies to a variety of stakeholders. In this work, data collected from the literature concerning tradespace exploration, analysis and decision-making practices is reviewed and analyzed to identify the best practices and common pitfalls experienced by organizations conducting tradespace studies. Using a series of interviews conducted with members of a tradespace study group, the authors then proceed to use these best-practices and common issues to evaluate the tradespace study group to reveal avenues for improvement and areas of excellence within that group. Through this paper, other organizations can perform similar self-assessments to improve their capabilities with respect to tradespace studies.
Authors
- Julia Daniels - Clemson University
- Cameron J. Turner - Clemson University
- John Wagner - Clemson University
- Nafiseh Masoudi - Clemson University
- Malena Agyemang - Clemson University
- Gregory Hartman - US Army GVSC
- Denise Rizzo - US Army GVSC
- David Gorsich - US Army GVSC
- Annette Skowronska - US Army GVSC
- Rachel Agusti - US Army GVSC
Citation
Daniels, J., Turner, C., Wagner, J., Masoudi, N. et al., "Designing the Design Space: Evaluating Best Practices in Tradespace Exploration, Analysis and Decision-Making," SAE Technical Paper 2022-01-0354, 2022, https://doi.org/10.4271/2022-01-0354.Also In
References
- Beesemyer , J.C. , Ross , A.M. , and Rhodes , D.H. An Empirical Investigation of System Changes to Frame Links between Design Decisions and Ilities Procedia Computer Science 8 2012 31 38 10.1016/j.procs.2012.01.010
- Bertoni , M. , Bertoni , A. , Broeze , H. , Dubourg , G. et al. Using 3D CAD Models for Value Visualization: An Approach with SIEMENS NX HD3D Visual Reporting Computer-Aided Design and Applications 11 3 2013 284 294 10.1080/16864360.2014.863492
- Bertoni , M. , Eres , H. , and Isaksson , O. Criteria for Assessing the Value of Product Service System Design Alternatives: An Aerospace Investigation Functional Thinking for Value Creation Berlin Heidelberg Springer 2011 141 146 10.1007/978-3-642-19689-8_26
- Bhattacharya , S. , Nagaraju , V. , and Fiondella , L. , Spero , E. , and Ghoshal , A. Rotorcraft Tradespace Exploration: Incorporating Reliability Engineering 5th Annual SERC Doctoral Students Forum (SDSF) Washington, DC November 7-8 2017
- Blackburn , T.D. , Mazzuchi , T.A. , and Sarkani , S. Using a TRIZ Framework for Systems Engineering Trade Studies Syst. Engin. 15 3 2011 355 367 10.1002/sys.21199
- Boehm , B. , et al. 2013 https://apps.dtic.mil/sti/citations/ADA590751
- Bond , C.A. , Mayer , L.A. , McMahan , M.E. , Kallimani , J.G. , and Sanchez , R. 2015 http://www.rand.org/t/RR701
- Browne , D.C. , Fullmer , D.D. , Slusser , D.S. , Stone , R. , and Terrell , G.R. Improving DoD Acquisition with Set-Based Design 19th Annual Systems Engineering Conference Springfield, VA October 27, 2016
- Cataldo , G. , et al. 2014 http://hdl.handle.net/1721.1/92407
- Chattopadhyay , D. , Ross , A.M. , and Rhodes , D.H. A Practical Method for Tradespace Exploration of Systems of Systems AIAA SPACE 2009 Conference & Exposition, American Institute of Aeronautics and Astronautics 2009 10.2514/6.2009-6638
- Chattopadhyay , D. , Ross , A.M. , and Rhodes , D.H. Demonstration of System of Systems Multi-Attribute Tradespace Exploration on a Multi-Concept Surveillance Architecture 7th Annual Conference on Systems Engineering Research 2009 (CSER 2009), Loughborough University April 20-23 2009
- Chau , D. and Patel , M. Engineered Resilient Systems: A Practical Armament Example and Study 19th Annual Systems Engineering Conference Springfield, VA October 26, 2016
- Chhabra , J.P.S. and Warn , G.P. Sequential Decision Process For Tradespace Exploration By Bounding Probabilistic Decision Criteria Using Mean-Risk Analysis Volume 1: 37th Computers and Information in Engineering Conference, American Society of Mechanical Engineers 2017 10.1115/DETC2017-68112
- Cilli , M.V. and Parnell , G.S. Systems Engineering Tradeoff Study Process Framework INCOSE International Symposium 24 1 2014 313 331 10.1002/j.2334-5837.2014.tb03151.x
- Comes , T. , Hiete , M. , Wijngaards , N. , and Schultmann , F. Enhancing Robustness in Multi-Criteria Decision-Making: A Scenario-Based Approach 2010 International Conference on Intelligent Networking and Collaborative Systems, IEEE 2010 10.1109/INCOS.2010.16
- Comes , T. , Hiete , M. , Wijngaards , N. , and Schultmann , F. Decision Maps: A Framework for Multi-Criteria Decision Support Under Severe Uncertainty Decision Support Systems 52 1 2011 108 118 10.1016/j.dss.2011.05.008
- Davenport , T. and Edwards , J. Tradespace Exploration of MBSE and MBE Integrated Workflows 2018 Global Product Data Interoperability Summit Phoenix, AZ 17 20 2019
- Davison , P.L. Tradespace Exploration for Space System Architectures: A Weighted Graph Framework 2014 http://hdl.handle.net/1721.1/90773
- Dillon-Merrill , R.L. , Parnell , G.S. , Buckshaw , D.L. , Hensley , W.R. Jr. et al. Avoiding Common Pitfalls in Decision Support Frameworks for Department of Defense Analyses Military Operations Research 3 2 2008 19 31 https://www.jstor.org/stable/43920755
- Durbach , I.N. and Stewart , T.J. A Comparison of Simplified Value Function Approaches for Treating Uncertainty in Multi-Criteria Decision Analysis Omega 40 4 2012 456 464 10.1016/j.omega.2011.08.004
- Ender , T. SERC-2014-TR-043 2014 10.21236/ada603143
- Ender , T.R. 2016
- Ender , T.R. and Goerger , S.R. ERS Tradespace Toolset 19th Annual Systems Engineering Conference Springfield, VA October 26, 2016
- Felix , A. Standard Approach to Trade Studies: A Process Improvement Model that Enables Systems Engineers to Provide Information to the Project Manager by Going Beyond the Summary Matrix 2004 INCOSE Mid-Atlantic Regional Conference (MARC2004) Arlington, VA November 2-4 2004
- Fitzgerald , M.E. and Ross , A.M. Artificial Intelligence Analytics with Multi-Attribute Tradespace Exploration and Set-Based Design Procedia Computer Science 153 2019 27 36 10.1016/j.procs.2019.05.052
- Girerd , A.R. A Rapid, Flexible Approach to Tradespace Definition and Exploration 2005 IEEE Aerospace Conference 2005 10.1109/AERO.2005.1559725
- Henry , S.M. , Waddell , L.A. , and DiNunzio , M.R. The Whole System Trades Analysis Tool For Autonomous Ground Systems 2016 Michigan Chapter NDIA Ground Vehicle Systems Engineering and Technology Symposium Novi, MI August 2-4, 2016
- Hitomi , N. Multiobjective Optimization For Space Systems Architecture: Applying And Extracting Knowledge 2018 10.7298/X4PN93WB
- Holland , J.P. Engineered Resilient Systems: Power of Advanced Modeling and Analytics in Support of Acquisition 18th Annual NDIA Systems Engineering Conference Springfield, VA October 28, 2015
- Le Moigne , J. et al. Tradespace Analysis Tool For Designing Constellations (TAT-C) 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS) 2017 10.1109/IGARSS.2017.8127168
- Kelley , D. , Goerger , S.R. , and Buchanan , R.K. Tradespace Tool Survey - Evaluating Tradespace Exploration for Engineered Resilient Systems 2016 Conference on Systems Engineering Research (CSER16) Huntsville, AL 2016
- Lamassoure , E.S. , Wall , S.D. , and Easter , R.W. Model-Based Engineering Design for Trade Space Exploration throughout the Design Cycle Space 2004 Conference and Exhibit 2004 10.2514/6.2004-5855
- MacCalman , A. , Kwak , H. , McDonald , M. , Upton , S. , Grider , C. , Hill , R. , Wood , H. , and Evangelista , P. Illuminating Tradespace Decisions Using Efficient Experimental Space-Filling Designs for the Engineered Resilient System Architecture 2015 https://apps.dtic.mil/sti/citations/AD1007274
- MacCalman , A.D. , Sanchez , S.M. , McDonald , M.L. , Goerger , S.R. , and Karl , A.T. Tradespace Analysis for Multiple Performance Measures 2016 Winter Simulation Conference (WSC) 2016 10.1109/WSC.2016.7822340
- Malan , C. and Pretorius , L. Methods and Metrics for Design of Value Robust Systems ISEM 2011 Proceedings, Stellenbosch South Africa September 21-23 2011
- Martin , P.J. Developing the ERS Collaboration Framework 19th Annual Systems Engineering Conference Springfield, VA October 26, 2016 2016
- McGowan , D. and Lichty , N. Framework for Developing Model-Based Buy Packages 2019 Global Product Data Interoperability Summit Phoenix, AZ September 16-19, 2019
- McManus , H.L. and Schuman , T.E. Understanding The Orbital Transfer Vehicle Trade Space AIAA Space 2003 Conference & Exposition 2003 10.2514/6.2003-6370
- McManus , H.L. , Richards , M.G. , Ross , A.M. , and Hastings , D.E. A Framework for Incorporating “ilities” in Tradespace Studies AIAA SPACE 2007 Conference & Exposition 2007 10.2514/6.2007-6100
- Mixon , E.R. , Church , J.Q. , Niles , K.D. , and Garton , T.W. Engineered Resilient Systems: TradeAnalyzer v4.0 User Guide 2019 https://apps.dtic.mil/sti/pdfs/AD1076452.pdf
- Nickel , J. 2010 https://dspace.mit.edu/handle/1721.1/63022
- Nickel , J. , Ross , A.M. , and Rhodes , D.H. Cross-Domain Comparison of Design Factors in System Design and Analysis of Space and Transportation Systems 6th Conference on Systems Engineering Research Redondo Beach, CA April 4-5 2008
- Nickel , J. , Ross , A.M. , and Rhodes , D.H. Trading Project Costs and Benefits in Multi-Attribute Tradespace Exploration 7th Annual Conference on Systems Engineering Research 2009 (CSER 2009), Loughborough University April 20-23 2009
- Oster , C. Evolving Lockheed Martin’s Engineering Practices through the Creation of a Model-Centric Digital Tapestry 2011 Frontiers in MBSE Workshop Phoenix, AZ January 30-31 2011
- Parnell , G.S. , Cilli , M.V. , and Buede , D. Tradeoff Study Cascading Mistakes of Omission and Commission INCOSE International Symposium 24 1 2014 332 346 10.1002/j.2334-5837.2014.tb03152.x
- Pernin , C.G. , Axelband , E. , Drezner , J.A. , Dille , B.B. , Gordon , J. , Held , B.J. , McMahon , K.S. , Perry , W.L. , Rizzi , C. , Shah , A.R. , Wilson , P.A. , and Sollinger , J.M. 2012 https://www.jstor.org/stable/10.7249/j.ctt3fgzv9
- Price , R.M. A Capability-Based Framework for Applying Value-Driven Design to Systems with Multiple Value-Producing Scenarios 2016 https://hdl.handle.net/1969.1/157752
- Qiao , L. , Efatmaneshnik , M. , and Ryan , M. A Combinatorial Approach to Tradespace Exploration of Complex Systems: A CubeSat Case Study INCOSE International Symposium 27 1 2017 763 779 10.1002/j.2334-5837.2017.00392.x
- Rader , A.A. , Ross , A.M. , and Rhodes , D.H. A Methodological Comparison of Monte Carlo Simulation and Epoch-Era Analysis for Tradespace Exploration in an Uncertain Environment 2010 IEEE International Systems Conference 2010 10.1109/SYSTEMS.2010.5482433
- Ricci , N. , Fitzgerald , M.E. , Ross , A.M. , and Rhodes , D.H. Architecting Systems of Systems with Ilities: An Overview of the SAI Method Procedia Computer Science 28 2014 322 331 10.1016/j.procs.2014.03.040
- Rigas , E. and Spero , E. Systems Tradespace Analysis: Assessment of Current Capabilities and Future Directions 15th Annual NDIA Systems Engineering Conference San Diego, CA October 24, 2012
- Ross , A.M. and Hastings , D.E. The Tradespace Exploration Paradigm INCOSE International Symposium 15 1 2005 1706 1718 10.1002/j.2334-5837.2005.tb00783.x
- Ross , A.M. and Rhodes , D.H. Architecting Systems for Value Robustness: Research Motivations and Progress 2008 2nd Annual IEEE Systems Conference 2008 10.1109/SYSTEMS.2008.4519011
- Ross , A.M. Multi-Attribute Tradespace Exploration With Concurrent Design as a Value-Centric Framework for Space System Architecture and Design 2003 http://hdl.handle.net/1721.1/82788
- Ross , A.M. 2006 http://hdl.handle.net/1721.1/35089
- Ross , A.M. , Hastings , D.E. , and Warmkessel , J.M. Multi-Attribute Tradespace Exploration as Front End for Effective Space System Design Journal of Spacecraft and Rockets 41 1 2004 20 28 10.2514/1.9204
- Ross , A.M. , McManus , H.L. , Rhodes , D.H. , and Hastings , D.E. Revisiting the Tradespace Exploration Paradigm: Structuring the Exploration Process AIAA SPACE 2010 Conference & Exposition 2010 10.2514/6.2010-8690
- Ross , A.M. , Rhodes , D.H. , and Fitzgerald , M.E. Interactive Value Model Trading for Resilient Systems Decisions Procedia Computer Science 44 2015 639 648 10.1016/j.procs.2015.03.035
- Salado , A. 2018 https://dair.nps.edu/handle/123456789/2735
- Simpson , T.W. , Miller , S. , Tibor , E.B. , Yukish , M.A. et al. Adding Value to Trade Space Exploration When Designing Complex Engineered Systems Syst. Engin. 20 2 2017 131 146 10.1002/sys.21384
- Sitterle , V.B. , Freeman , D.F. , Goerger , S.R. , and Ender , T.R. Systems Engineering Resiliency: Guiding Tradespace Exploration within an Engineered Resilient Systems Context Procedia Computer Science 44 2015 649 658 10.1016/j.procs.2015.03.013
- Smead , K. A Descriptive Guide to Conducting Trade Space Analysis 2015 https://apps.dtic.mil/sti/pdfs/ADA625386.pdf
- Smead , K. Trade Space Analysis: Rotational Analyst Research Project 2015 https://apps.dtic.mil/sti/pdfs/ADA625386.pdf
- Smith , E.D. , Son , Y.J. , Piattelli-Palmarini , M. , and Bahill , A.T. Ameliorating Mental Mistakes in Tradeoff Studies Syst. Engin. 10 3 2007 220 240 10.1002/sys.20072
- Smith , E.D. , Son , Y.J. , Piattelli-Palmarini , M. , and Bahill , A.T. Cognitive Biases Affect the Acceptance of Tradeoff Studies Decision Modeling and Behavior in Complex and Uncertain Environments 21 New York, NY Springer 2008 227 249 10.1007/978-0-387-77131-1_10
- Spaulding , T.J. 2003 Tools For Evolutionary Acquisition: A Study of Multi-Attribute Tradespace Exploration (Mate) Applied To The Space Based Radar (SBR) https://apps.dtic.mil/sti/citations/ADA416698
- Specking , E. , Parnell , G. , Pohl , E. , and Buchanan , R. Early Design Space Exploration with Model-Based System Engineering and Set-Based Design Systems 6 4 2018 2018 45 10.3390/systems6040045
- Specking , E. , Parnell , G. , Pohl , E. , and Buchanan , R. Evaluating a Set-Based Design Tradespace Exploration Process Procedia Computer Science 153 2019 185 192 10.1016/j.procs.2019.05.069
- Spero , E. , Avera , M. , Valdez , P. , and Goerger , S. Tradespace Exploration for the Engineering of Resilient Systems 2015 10.1016/j.procs.2014.03.072
- Spero , E. , Bloebaum , C.L. , German , B.J. , Pyster , A. et al. A Research Agenda for Tradespace Exploration and Analysis of Engineered Resilient Systems Procedia Computer Science 28 2014 763 772 10.1016/j.procs.2014.03.091
- Stump , G. , Lego , S. , Yukish , M. , Simpson , T.W. et al. Visual Steering Commands for Trade Space Exploration: User-Guided Sampling with Example ASME J. Comput. Inf. Sci. Eng. 9 4 2009 044501 10.1115/1.3243633
- Technology, Planning and Integration for Observation (TPIO) Trade-Space Analysis Guide
- The Road to Interoperability 2019 Global Product Data Interoperability Summit Phoenix, AZ September 16-19, 2019
- Tibor , E. Visualization-Based Decision Support for Value-Driven System Design 2014 10.31274/etd-180810-3815
- Ullman , D.G. and Spiegel , B.P. Trade Studies with Uncertain Information INCOSE International Symposium 16 1 2006 1294 1307 10.1002/j.2334-5837.2006.tb02813.x
- Unal , M. , Warn , G.P. , and Simpson , T.W. Introduction Of A Tradeoff Index For Efficient Trade Space Exploration Volume 2B: 41st Design Automation Conference, American Society of Mechanical Engineers 2015 10.1115/DETC2015-46895
- Unal , M. , Warn , G.P. , and Simpson , T.W. Quantifying the Shape of Pareto Fronts During Multi-Objective Trade Space Exploration ASME J. Mech. Des. 140 2 2018 021402 10.1115/1.4038005
- Witus , G. and Bryzik , W. Progress Toward a DoD Ground Vehicle Tradespace and Affordability Analysis Framework Procedia Computer Science 44 2015 537 546 10.1016/j.procs.2015.03.028
- Xiong , Y. , Duong , P.L.T. , Wang , D. , Park , S. et al. Data-Driven Design Space Exploration and Exploitation for Design for Additive Manufacturing ASME J. Mech. Des. 141 10 2019 101101 10.1115/1.4043587
- Yoe , C. Trade-Off Analysis Planning and Procedures Guidebook U.S. Army 2002