System tests in automated production system (aPS) engineering in production automation are often performed under high time pressure and in an uncomfortable on-site environment at the customer’s premises. This can lead to a lack of documentation, uncertainty of test adequacy and inefficient or inadequate testing in case of changes. The rising complexity in today’s production automation systems and missing software tool support in this field aggravate these situations.
The aim of this thesis was to tackle these problems by developing an approach for a more structured and efficient system testing process. Its foundation is a guided system testing method, which partially automates the testing process, while including a human tester by giving step-wise instructions via a Human Machine Interface when necessary. Extending this foundation by analyzing data acquired during test execution, the approach allows for exploratively identifying untested system behavior by visualizing system test coverage. Furthermore, in case modifications are performed on a previously tested aPS, a test case prioritization method supports the tester in quickly finding unintentionally introduced faults into the system.
The approach was developed in accordance with industrial requirements, which were compiled in cooperation with experienced experts from reputable companies active in the field of aPS engineering. Based on these requirements, the approach was evaluated in an industrial case study and expert evaluation, showing the approach’s industrial applicability.