Developers’ Insights on Game Design Elements: Pros and Cons

Table of Links

Abstract and 1 Introduction

2. Background and Related Work and 2.1. Gamification

2.2. Game Design Elements and 2.3. Gamification Effects

2.4. Related Work on Gamification Negative Effects

3. Systematic Mapping and 3.1. The Research Questions

3.2. Search Strategy and 3.3. Inclusion and Exclusion Criteria

3.4. Applying the Search Strategy

3.5. Data Extraction

4. Systematic Mapping Results

5. Focus Group: Developer Perception on the Negative Effects of Game Design Elements

5.1. Context and Participant Characterization

5.2. Focus Group Design

5.3. The Developers’ Perception on The Negative Effects

5.4. On the Perceived Usefulness, Ease of use and Intent of Adoption of Mapped Negative Effects

5.5. Participant Feedback

6. Limitations

7. Concluding Remarks

7.1. Future Research Directions

Acknowledgements and References

5.3. The Developers’ Perception on The Negative Effects

We asked participants to assign one vote to the awareness degree on the negative effects reported in the literature and to provide us with comments on pros and cons for using the game design elements under discussion. We have collected these comments through post-it notes added by the participants in the session’s virtual mural (as illustrated in Figure 8). In order to analyze these comments, we first watched the video and audio records and transcribed them into plain text. Thereafter, we analyzed all post-it comments written by the participants and associated transcription quotes. In the following subsections, we summarize the comments that emerged for each group of game design elements.

5.3.1. Badges and Rewards

Figure 9 illustrates the comments made by the participants. In this group of game design elements, three participants reported being mostly unaware, and only one participant reported being mostly aware about the negative effects. We detail each comment as follows grouped by positive and negative ones.

Positive comments. The participants mentioned a total of two positive comments. Increases Extrinsic Motivation. Participant P1 mentioned that he was unaware about the negative effect of reducing intrinsic motivation. On the other hand, the participants mentioned that the use of these game design elements, in isolation or together with others, is important to increase extrinsic motivation. As mentioned by P1 and agreed by P3, as follows.

P1: “when we see that a player has many badges, what we expect as developers is that another player becomes more motivated, [...] the perception is that it increases the extrinsic motivation. However, I was unaware that [the use of badges and rewards] could reduce intrinsic motivation.”

P1: “We made some simulations using badges as part of the challenges, I would say that yes, it had a return on the motivation.”

Distinguished Badges to improve engagement quality. Another positive point highlighted by the participants was the use of distinguished badges to improve engagement quality. In this sense, participant P3 mentioned that one of the strategies used to motivate users of the VazaZika software was the use of distinguished badges, which could only be won by the user under specific conditions, associated with challenges and time intervals.

P3: “[...] the idea of distinguished badges instead of [normal] badges that you can get doing a task anytime, [is that] you have distinguished badges associated with a certain time period, a certain challenge. So the idea would be to motivate certain periods associated with each badge.”

Negative comments. The participants mentioned a total of four negative comments. Super motivation. An interesting negative point raised by participants P2 and P3 concerns the super motivation of users in performing tasks, aimed only at winning badges and rewards. In this case, P3 also mentioned that super motivating users can lead to the introduction of false data into the software. For instance, in gamified software where the user earns badges for each check-in performed.

P3: “[...] In the VazaZika software, we had a badge that the user would only earn if he was in a certain location in a specific period of time [...]. So it was possible to increase the [data] quality, because we knew that the user who was completing this challenge was reporting real mosquito breeding sites.”

Although the presence of super motivated users has a negative effect, participant P2 also mentioned that this effect can be minimized through the definition of validation mechanisms.

P2: “[..] in programming sites in which the user is super motivated, writing code and sending a lot of code every time. The control mechanism is good [because] it tests and verifies that the code is working. Thus, the super motivated user is not a problem in this case. It would only be a problem if the control mechanism was bad.”

Validation mechanisms to avoid cheating. Another negative point mentioned by the participants was the need to introduce validation mechanisms to avoid cheating. As mentioned by participants P1 and P4.

P1: “In order to avoid false data, we used the vote (up and downvote) to verify and validate if that reported mosquito breeding site was really real or not.”

P4: “[...] we had a worry that the award by itself, without any type of control mechanism, could be harmful. In our context, at the health area, that involved the work of health public agent, government actions [...], a greater care was needed to ensure the fidelity of information.”

Cheating to Badges. Another point mentioned, and strongly related to super motivation, and the presence of validation mechanisms is the practice of cheating to badges, intending to earn more rewards as mentioned by the participant P2 as follows.

P2: “[...] the player was misrepresenting the app data because he was super motivated to win the badge. However, I do not think that this avoids the bigger problem that is the false data that are entering the software, this is the big context.”

Excessive focus on rewards. Finally, the participants mentioned that the use of badges and rewards can generate an excessive focus on earning rewards. According to participant P4, this may be due to the gamification software being designed to generate rewards for users after performing simple and repetitive tasks, generating a deficiency in the game’s logic.

P4: “So, maybe it is related to certain gamified software that have more focus in provide rewards than the game logic.”

P4: “[So], if the idea is to [provide] fun, entertain and attract [the players], we need to be more careful in giving rewards for each performed task by the player. This cannot go unnoticed. We need game logic too.”

5.3.2. Competitions, Challenges and Goals

Figure 10 illustrates the pros and cons mentioned by the participants for Competitions, Challenges and Goals. In this group of game design elements, two participants reported a mostly unaware, and two participants reported mostly awareness about the negative effects. We detail each comment as follows grouped by positive and negative comments.

Positive comments. The participants mentioned a total of three positive comments as follows. Communitary (social) integration. P4 mentioned that the use of competition, challenges, and goals can increase community integration. More particularity, the participant mentioned the use of these game design elements in a particular community, for instance, the citizens of a particular neighborhood can increase integration between those involved. As we can see below.

P4: “The context here influences a lot. I would add this type of competition as an advantage. So, if you are gamifying something that probably has a purpose of getting a result for a certain community, e.g., a industry, but specifically in our case citizens within a neighborhood or region, I see this as an opportunity for integration.”

Cooperative tasks can help integrate the community. In line with the previous positive reason, participant P2 mentioned that the use of cooperative competitions, i.e., in which players come together to achieve the same goal, is more motivating than just creating competitions between players.

P2: “if you create competitions that aren’t necessarily between players, but between players around a common goal, you can help players come together to achieve those goals. In VazaZika, we’ve done that by enabling team creation.”

Team effort might reduce individual efforts. Complementary, Participant P3 mentioned that depending on the type of challenge and its difficulty level, is essential that the challenge be done with two to three people. In this sense, the creation of cooperative tasks helps to reduce the effort to achieve a common goal.

P3: “What I think goes against the extra effort, is that depending on the challenge, you can not do it alone, you need two to three people”

Negative comments. The participants mentioned a total of five negative comments, as follows. Perceived extra effort. Participant P1 has mentioned that extra effort is a negative effect when challenges require physical effort. In the context of the VazaZika software, citizens had to report and validate real cases of mosquito outbreaks in loco. The participant also mentioned that even though the challenge is done in teams, the extra effort was still perceived by the citizens.

P1: “Most of our challenges required a physical effort since the citizens had to go to a location to report or validate the existence of a mosquito breeding site, either individually or in teams. We were aware that there was an extra effort because they would have to move around.”

Can demotivate casual players. Another interesting negative reason mentioned by participant P2 concerns the demotivation of casual players to participate in competitions that attract frenetic or experienced players. In other words, a casual player may feel demotivated to compete with frenetic players, whose experience level is constantly increasing due to the time spent these players in performing gamified activities in the software.

P2: “You have a frenetic community of [players] who set the bar high, and for a [casual player] to participate in competitions is simply demotivating, because he cannot play casually in these competitions.”

Rules might demotivate participation. Another problem associated with the demotivation of casual players is related to the definition of rules that can restrict the participation of some players in collaborative challenges. For instance, rules that involve the availability of collaborative challenges in a given time interval may restrict the participation of players who are in different time-zones. As mentioned by Participant P3 as follows.

P3: “Another problem is that there are some rules in competitions that make it difficult for players to participate, e.g., players who are in a completely different time zone and who want to compete with their friends from Brazil who are in another time zone. In this sense, a rule involving time restrictions discourages a player from participating in the competition.”

On the other hand, participant P4 mentioned that the definition of rules that restrict the participation of players is not always a bad design. That is the case of challenges that involve the participation of a community (a favela), or a small group of adolescents in a public school. Such challenges, although they have geographic restrictions, also stimulate the accomplishment of tasks or challenges pertinent to a certain location. For instance, a challenge to report mosquito outbreaks in a region with a public health policy deficiency.

P4: “For a community geographically, a favela, adolescents from a public school [...] it would be a stimulus, to prevent diseases [...]”

Rules may stop competitors to participate in the way they want. In line with the previous comment, participant P3 mentioned that some rules in addition to demotivating can stop players that want to participate in challenges. For instance, rules related to the minimum quality of participants for challenges. In this case, the software may be forcing challenges with certain characteristics that a player does not necessarily want as mentioned by participant P3 as follows.

P3: “I remember that in the VazaZika software , some competitions could only be held by a team, and the team had a minimum amount of people. [...] So, the software was forcing you to enter a competition with certain characteristics that you don’t necessarily want.”

Confusion to reach the goal as a team. In fact the lack of understanding can be a negative effect. More specifically, participant P1 mentioned that although instructions for completing a challenge were explicitly provided to players, in some cases they could not understand what needed to be done to complete a challenge as a team.

P1: “The lack of understanding could indeed have this negative effect. We tried to make it explicit what they had to do to complete a challenge, but they still couldn’t understand.”

5.3.3. Leaderboards and Rankings

Figure 11 illustrates the pros and cons mentioned by the participants for Leaderboards and Rankings. Similar to the previous group of game design elements, two participants reported a mostly unaware, and two participants reported mostly awareness about the negative effects. We detail each comment as follows grouped by positive and negative comments.

Positive comments. The only positive comment mentioned was Can create pro players, by P2, where he argued that players with a higher level of ability in the game would be able to be beneficial outside it.

P2: “Per example, trying to get the context of a gamified system, the HackerRank, (...) that is a programming questions’ system, the pro players there... for you to be as good as them, you have to ’waste your life’ writing code and submitting questions all the time, and being approved on the tests and gaining points.”

P2: “But the pro players in those systems are so good in algorithms that they would easily be approved on certain business interviews, you understand? So, there’s a lot of business monitoring those pro players, let’s say that. So for the industry that is monitoring the rankings, this could be a positive point. Not considering all the negative effect of the thing that can stress you, stuff like that. For the corporate world this can help somehow.”

Negative comments. There are four negative comment, starting with Depends on the profile of the players (collaborative or competitive), the participant P3 mentioned that depending on the type of player, the use of leaderboards or rankings can have a negative effect. For instance, unlike players with a competitive profile, who are motivated to get the best positions, collaborative players can feel unmotivated.

P3: “[...] there are those that will consider Leaderboards and Rankings as something that motivates them, ”I want to be the first”, but there are others with a more collaborative profile, their reaction will be ”look, I don’t want to participate and be the first”, so it depends a lot of the [...] players’ profile.”

Cooperative harassment. Participant P4 has mentioned that the use of leaderboards and rankings in a cooperative environment, mainly in the private sector, can lead to harassment or bullying of the player. In this case, participant P4 argued that in these environments, a player can use leaderboards and rankings as a form of harassment, both for those who are at the top of leaderboards/rankings and for those who are not.

P4: “[...] in the public sector and mixed economy, this worry with harassment, the person feeling excluded, diminished, or even demotivated. And that has a lot to do with the negative effect.[...] They see that as a way to reduce the other, both who is higher in the rankings/leaderboards and who is lower.”

Limits of quantitative approaches (do not consider particular situations). In line with the Cooperative harassment, participant P4 mentioned that a negative aspect is the use of leaderboards and rankings heavily based on quantitative approaches. P4 argued that in situations in which a player is unable to perform tasks in the software, he will lose positions in the ranking, which would be considered unfair. For instance, in cooperative environments when a player takes sick leave, his actions are no longer quantified.

P4: “[...]exceptions will always happen because we are human and there will be situations where the person will fall through the rankings and that will be considered unfair, as an example, a medic license and the person went away. And the metrics are [...] quantitative, and the rankings are mostly quantitative.”

Can create an unfair competition. Is related to the demotivation of beginner players in achieving leadership or a similar position to players who are at the top of the leaderboard. In other words, depending on the beginner player’s current level, he may never reach the leadership as mentioned by participant P1 as follows.

P1: “So I think this is a cons point, because depending on what level you are, you will never reach the lead.”

5.3.4. Points, XPs and Virtual Currencies

Figure 12 illustrates the comments made by the participants for Points, XPs, and Virtual Currencies. About the awareness degree, three participants reported a mostly unaware, and one participant reported mostly awareness about the negative effects. We detail each comment as follows grouped by positive and negative comments.

Positive comments. Two positive comments were made by participant P3: Build a reputation mechanism and It increases reliability. Participant P3 exemplified that in systems such as Stack Overflow, a gamified system of questions and answers about programming problems, the use of points helps to build a reputation in terms of the player’s level of experience in response to specific answers. At the same time, responses from highly reputable players are considered highly reliable.

P3: “For instance, let’s think about the case of Stack Overflow, that is a gamified system where sometimes you want to search an answer to a solution, then you have two possibilities, which one you choose first? The one of the user with the highest XP. [...].”

Negative comment. The participants P2 and P4 mentioned that There are games that allow paying to win, demotivating others as a negative reason. In this sense, P2 discussed the negative effects of virtual currencies in allowing players to obtain advantages over other players (pay to win).

P2: “It’s a real option. And then you have games with a mechanic of paying to have advantages upon other players. This creates a demotivation. It’s what people calls as pay to win.”

Participant P4 also argued that the use of virtual currencies, which facilitates the practice of pay to win, can discourage players who continually perform actions in the software in order to advance in the game.

P4: “I agree a lot with P2. I can’t see the gamification context, it does not occur to me that it would be worthy to give coin or the opportunity to pay to play, I can’t see it, because usually the serious game, you will gamify a thing there to fix an issue, how are you going to add payment there? Why when you pay to achieve something without a fight for that, so that is done only to brag about the achievement. But in a serious game you have to do things, you have to solve problems. So, how can you in this context use a coin, not only as money but being able to pay to gain the money? Then in this pay to win, I agree completely with P2, and I can’t see, can’t glimpse in the serious game context that you would pay to win.”

5.3.5. Feedback and Achievements

Figure 13 illustrates the comments made by the participants for Feedback and Achievements. In this group of game design elements, two participants were completely unaware, one participant was mostly unaware, and one participant as mostly aware of the negative effects. We detail each reason as follows grouped by positive and negative comments.

Positive comments. Public feedbacks may serve as hints for other users and Achievements-driven players can play and contribute more. P1 added that public feedbacks would be advantageous to engage the users. In other words, users can feel motivated, realizing that other users are performing tasks and being rewarded for them.

P1: “So, there’s that question about various gamification elements, but in truth, it has the same meaning. I think, this was our case, in relation to the list of activities that the citizen would be able to do, I don’t know if you all agree, but when we say to the other citizens that that player, that citizen, did an activity with success, I believe that it is a valid feedback form to other participants, even as a way to engage. So-and-so is reporting small flies at that region. Hey I’m also going there because theoretically there’s a high concentration, or something like that.”

Difficult to perceive feedback as negative. Participant P3 mentioned that the use of feedback should be considered in any software. In gamified software , its use is more intensive because when the user performing a task with with reward, the user must always be informed.

P3: “[...] Feedback is something that must be considered in any software development. [...]. So, as a gamified software this should be even more prominent, because, you start doing a lot of different achievements and you don’t get any response from the software that you’re changing levels and getting a badge. So it’s a little difficult for me to see that lack of effect is a negative feedback effect.”

Negative comments. Excessive feedback might be a problem and Massive use of feedback. The Participant P3 have mentioned that the excessive use of feedback results in a negative and intrusive effect for the user of the software .

P3: “Maybe it can be bad when you receive too much feedback. For instance, I logged into the software , and I get a notification, anything you do on the software gets feedback. So maybe there is a negative and intrusive effect.”

5.3.6. Avatar

Figure 14 illustrates the comments made by the participants for Avatars. Three participants were completely unaware, and only one participant was completely aware about the negative effects. We detail each comment as follows grouped by positive and negative comments.

Positive comments. The participants have mentioned a total of four positive and one negative comments as follows. Basic characteristics of any game, enables customizing the users. P1 and P3 have mentioned Avatar is a basic characteristic of any gamified software . Even more so today, in which everyone wants to differentiate themselves, and in this case an avatar can express that difference.

P3: “In the same way that XP is considered a basic feature of any game, Avatar is too. Even more so nowadays, when everyone wants to differentiate themselves, and you don’t have an avatar that can express that difference. For me or at least for most users it is a core feature of the software.”

Additionally, participant P1 mentioned that from the end user’s point of view, there is no extra effort in selecting an avatar’s characteristics. P1 also mentioned that setting up an avatar is normally part of the initial setup of any game.

P1: “From the end user’s point of view, I don’t think it takes too much effort. Also, as P3 mentioned, if you were going to play some other game, or even some gamified software, a step in the tutorial is to create your avatar.”

Self-expression brings engagement. The participant P4 have mentioned that in gamified software s that focus on a younger audience, the use of avatars leads to greater engagement, mainly due to the capacity for self-expression.

P4: “You know, it’s a thing that it must exist, because some like it, mainly the young public.[...]”

Development effort might pay off (e.g., engagement). Participant P2 has mentioned that despite the development effort required to add an avatar as a feature into the software, the effort might pay off, especially if the avatar increases user engagement in the software.

P2: “Understood. Because, even if it demands effort from the developer to create the activity, if me manages to engage (...), even if just 10% or 20% of the users, it would be worthwhile.”

Negative comment. The participants P1 and P4 mentioned that an Extra effort was indeed considered in the gamified software, and only basic avatars were used. More specifically, P1, mentioned that the creation of sophisticated avatars requires the customization of different items, e.g., hair, clothes, skin color, which consumes development time (one of the main constraints of the VazaZika project).

P1: “From the developer’s point of view, in VazaZika we didn’t have a sophisticated avatar where you could choose your hair, clothes, skin color, etc. One of the comments for this was extra effort, and time restrictions.”

Participant P4 also mentioned that the extra effort is also related to the constant need to make new items available for the avatar, which are often driven by external trends, associated with the need for a professional with design skills.

P4: “Who chooses and customizes an Avatar is an identity mechanism, the person will continually update the Avatar. Something happened, I’ll update the avatar. Changed a trend, a fashion, I’ll update. It’s how I change clothes. I update the avatar all the time.”

P4: “(...)because it’s something that demands much more design, so it must have a designer, someone that would work with that part.”

5.3.7. Quizzes

Figure 15 illustrates the comments made by the participants. Two participants were mostly unaware, and two mostly aware of the negative effects. We detail each comment as follows grouped by positive and negative comments.

Positive comment. The participants mentioned one positive comment: Might Help To Achieve Educational Purposes Participant P1 mentioned that, although quizzes were not added to VazaZika software, the quizzes were considered in the early phases of VazaZika software design. The same participant also mentioned that an advantage of using quizzes is enabling achieving educational purposes.

P1: “I think that the advantage is when you want to gamify in this educational sense. So, in our software the advantage would be that we would be making the citizen be more conscious about the peculiarities of the small fly and its focus. So, how to combat the focus? Which is the time period with more incidence? [...]”

Negative comments. The participants mentioned a total of three negative comments as follows. Boring task. P1 mentioned that compared to other gamified tasks, answering questions can demotivate players.

P1: “[...] only answering questions really is a bit boring, if compared with other tasks that the game would offer.”

Frustration after getting wrong answers. In line with P1, participant P3 elaborated on the possible negative effect of using quizzes is the frustration when a user gives a wrong answer to a question, generating pressure to get it right.

P3: “First that I don’t like quizzes, but there are cases where you have to do, right, you don’t have a choice. But there’s that pressure of having to get it right, and when you don’t, you stay with that [...] frustration, and I always see it as a negative effect.”

P3: “Even more when you are in an educational software, where the person answering the quiz wants to get it right and when it doesn’t, or has the pressure to, or gets frustrated.[...]”

Risk of repetition. The use of traditional quizzes may have the risk of repetition, resulting in the lack of motivation of the players as pointed by P3 as follows.

P3: “The repetition risk that I added here, its obvious that there are a lot of mechanisms to avoid [it], but a more conventional quiz where you raffle questions in a small group always have that.”