Measuring Intrinsic Motivation in Pair Programming

Table of Links

Abstract and 1. Introduction

2 Research Settings

3 Methods

3.1 Instruments and 3.2 Quantitative Analysis

3.3 Thematic Analysis

4 Results

4.1 Quantitative Results

4.2 Qualitative Results

5 Discussion and 5.1 Answering the research questions

5.2 Threats to validity and 5.3 Limitations and generalizability

6 Conclusion, Acknowledgments, and References

3 Methods

3.1 Instruments

Recent studies, such as [6, 7], advocate the use of instruments coming from psychology and related fields for systematic studies in the IT discipline and emergent “behavioral software engineering” field [13].

Intrinsic motivation related to the programming role was measured using a self-report questionnaire Intrinsic Motivation Inventory at the end of each 10-minute-long round of which there were six in each session. The questionnaire was developed and first used by Ryan in a series of psychological experiments in the 1980s [15]. Its first subscale that was used consists of seven Likert-scale items, which are factor-analytically coherent and stable across various tasks, conditions, and settings, and assesses participants’ subjective experience related to target activity in laboratory experiments [17].

The final questionnaire deployed in the experiments also contained the Big Five personality measurements. All instruments were implemented in their original English version.

3.2 Quantitative Analysis

We used scripts written in R language v4.2.2 and open-source libraries for statistical tests and computations.

Descriptive statistics were used to describe the sample and provide a demographic overview. Subsequently, the Shapiro-Wilk test evaluated the reported intrinsic motivation values in various programming roles for their normality. Assuming the normal distributions, a one-way parametric test of variance (ANOVA) was used to verify the statistical significance of the derived relations between the chosen programming role and reported intrinsic motivation.

3.3 Thematic Analysis

The inquiry framework presented in this article uses thematic analysis developed for use within a qualitative paradigm, subjecting data to analysis for commonly recurring themes. It is theoretically flexible, and the author has chosen the inductive (bottom-up) way of identifying patterns in the data instead of the deductive (top-down) approach. Inductive analysis was used to code the data without trying to fit it into a pre-existing coding frame or the researcher’s analytic preconceptions.

Seven steps by [5] were applied flexibly to fit the research questions and data: transcribing, becoming familiar with the data, generating initial codes, discovering themes, reviewing themes, defining themes, and writing up. Transcribing (step 1) can be seen as a critical phase of interpretative data analysis, as the meanings are created during this thorough act [4]. A professional tool (Descript, v52.1.1) was used for initial pre-processing to generate basic transcription from voice recordings and to remove filler words like “um.” Final transcripts were imported into a computeraided qualitative data analysis tool (MAXQDA, v22.3.0), coded, and analyzed for themes (steps 2-6)