Does Programming with a Partner Make Learning Easier?

Table of Links

Abstract and 1. Introduction

2 Research Settings

3 Methods

3.1 Instruments and 3.2 Quantitative Analysis

3.3 Thematic Analysis

4 Results

4.1 Quantitative Results

4.2 Qualitative Results

5 Discussion and 5.1 Answering the research questions

5.2 Threats to validity and 5.3 Limitations and generalizability

6 Conclusion, Acknowledgments, and References

2 Research Settings

The research was set in two advanced software engineering course classrooms taught by the author of this paper and took place over five classes, with the first being a training session. Afterward, the research continued with seven individual online interviews until the point of saturation was reached [4].

The students were subjected to four laboratory sessions of 60 minutes of net programming time each. The (N = 35) subjects paused every 10 minutes to self-report their motivation with a seven-item questionnaire and rotated roles in their pair. Each round, they received a new programming task but oftentimes continued the previous one because the tasks formed a continuum. Each session lasted a total of 90 minutes, of which 30 were used for describing the tasks and completing the questionnaires.

The partners in the experimental group were designated either “pilot,” who controls the keyboard and codes, or “navigator,” who conceptualizes the solution and looks for defects. The participants in the control group worked on the same tasks in the “solo” role.

Each individual’s motivation related to the performed role was reported after every round using a standardized seven-item questionnaire, including when programming solo in the control group. The Big Five personality test at the beginning of each session measured the subject’s personality. Analysis of the personality results was not included in this preliminary study’s scope, but the results were confidentially given to the subjects and indirectly queried in the qualitative interviews.

The subjects were instructed on how to pair-program during a training session that preceded the other four. Then, the subjects worked on predefined tasks in a static order that helped them create their individual semestral work. It was an adventure game with a graphical user interface written in Java. During the experimental sessions, students have (i) implemented the observer time for real-time updating, (ii) added a contextual menu, (iii) animated the characters in the game, and, finally, (iv) cooperated with other pairs in their six-member teams to develop the foundations of their second, more complex semestral work.

Concerns of experimental design.

The task difficulty was of primary concern to the experimental design’s validity, which is why the first two sessions contained a “control group.” In each session, the motivational levels for each 10-minute round were compared in both the control (solo programming) and the treatment group (pairing) to confirm that there was, in fact, no correlation between the specific tasks and self-reported intrinsic motivation. This is consistent with [18] and our qualitative results.

The second concern, the subjects were of various abilities. If performances were measured, the subjects’ abilities could have diminished the measured effect, like in [3], as performance = ability x motivation [12]. We avoided that by measuring intrinsic motivation instead, which does not directly relate to ability unless the task is outside the individual’s perceived competency levels [16].

Lastly, the uncertain personal compatibility of pairs posed a threat. Pairs were allocated pseudo-randomly and irrespective of personality, like in two previous experimental studies, one with 564 students, where 90 % of pairs reported compatibility [11], and another with 1350 students, and 93 % reported compatibility [19].