这篇论文提出了REAL框架,专门针对ML系统,用失败驱动对齐,自动驾驶案例很具体。
该论文提出了REAL(Requirements Engineering for mAchines that Learn - and Fail)框架,用于系统化机器学习系统的需求工程。REAL基于三个原则:将数据、模型和系统的需求交织在一起;利用失败来驱动替代需求的探索;以及迭代且可追溯的需求精化。作者通过自动驾驶案例演示了框架的有效性,并提供了在线复制包。
From Failure to Alignment: A Requirements Engineering Framework for Machine Learning Systems
Organisations designing, developing, and deploying machine learning systems (MLS) need to be able to check that these systems are trustworthy, and communicate this clearly to their stakeholders, be they different categories of users, engineers, or wider society. By focusing on stakeholders, Requirements Engineering is well positioned to drive the design and engineering of MLS that align with the needs of their stakeholders. Yet, we still need a systematic process for modelling and reasoning about requirements for MLS that is driven both by stakeholders' needs and constraints for MLS development. This paper proposes a framework entitled REAL (Requirements Engineering for mAchines that Learn - and Fail) to help develop MLS that align with stakeholders' needs by adopting a requirements engineering approach. This model-based framework is based on three principles. First, weaving together requirements for data, models, and the system as a whole. Second, using failure to drive the exploration of alternative requirements. Third, iterative and traceable refinement of MLS requirements. We demonstrate the proposed framework using an example from autonomous driving and show that REAL supports the development of MLS that better align with stakeholders' requirements. A replication package is available online.