Impact of model notations on the productivity of domain modelling: an empirical study

Abstract

Context: The intensive use of models is a cornerstone of the Model-Driven Engineering (MDE) paradigm and its claimed gains in productivity. However, in order to maximize these productivity gains, it is important to adequately select the modeling formalism to be used. Unfortunately, the MDE community still lacks empirical data to support such choice. Objective: This paper aims at contributing to filling this gap by reporting an empirical study in which two types of domain model notations, graphical vs. textual, are compared regarding their efficiency and effectiveness during the creation of domain models. Method: A quasi-experiment was designed in which 127 participants were randomly classified in four groups. Then, each group was randomly assigned to a different combination of notation and application. All the participants were students enrolled in the 6th semester of the Computer Engineering degree at the University of Alicante. The statistical procedure applied was a two-factor multivariate analysis of variance (two-way MANOVA). Results: The data shows a statistically significant effect of notation type on the efficiency and effectiveness of domain modelling activities, independently from the application being modelled. Conclusions: The joint examination of our results and those of previous studies suggests that, in MDE, different tasks call for different types of notations. Therefore, MDE environments should offer both textual and graphical notations, and assist developers in selecting the most suitable one depending on the task being carried out. In particular, our data suggest that domain model creation tasks are better supported by graphical notations. To augment the validity of the conclusions of this paper, the experiment should be replicated with different subject profiles, notations, domain model sizes, tasks and application types.