A Methodology for Testing Simulated Environments

Abstract

Currently, testing is the most widely used technique to check the validity of complex systems. Its application to software is a critical aspect in the development of systems. However, there are several difficulties for applying these techniques, such as high economic and computational cost. One of the main difficulties when applying testing techniques is to obtain an appropriate test suite. This is especially difficult when the size of the system under test is large, like HPC clusters or cloud systems. The main goal of this paper is to provide mechanisms that allow to evaluate the suitability of test suites to check distributed systems, in an inexpensive and efficient way. In this work, we propose a methodology to integrate mutation testing techniques with simulation techniques. For modelling and simulating distributed systems, the SIMCAN [1] simulation platform has been used. It is intended to unify, in one framework, functionalities of different tools, such as simulators, mutation frameworks and tools for generating tests. This work also presents some experiments for analysing the suitability of test suites over different distributed applications. Each of these applications has been modelled with SIMCAN, and different experiments have been carried out by applying mutation testing techniques on these models.