Crafting an international survey from conception to feedback to launch.
This project was completed during my summer internship as a User Researcher for an unannounced project at Bungie. Bungie is a well-known game developer (and now publisher), who created the Halo franchise, as well as the Destiny franchise. Most recently, they have revitalized their Marathon IP and transformed it into an extraction-shooter.
In my role, not only was I tasked with moderation, analysis, and reporting for in-person playtests for this same unannounced project, but to add to the meaningful work I had completed, I was able to create and launch an international survey. This survey was informed by previous internal research, which was incorporated in the research plan and draft survey, and this survey was created to reach an international audience. Unfortunately, my internship was finished before responses were returned, so I was unable to do any analysis or reporting for this project.
Overall, my time spent on this survey was approximately 6 weeks.
Objectives:
Unfortunately, due to the sensitive nature of the information and that the project is unannounced, I cannot go into detail on the objectives of the research at hand.
Process:
To familiarize myself with the internal research that had already been completed, I conducted a scan of all previous work completed related to this current project. Everything from research plans to stakeholder presentations were reviewed to get a better understanding of the goals of the project.
Next, in consultation with stakeholders and fellow user researchers, I drafted a research plan outlining what success looks like based on responses we could receive from the survey. The research plan outlined screener criteria, as well as the number of participants required, broad hypotheses, and also included some expectations for feedback based on prior research.
From here, I outlined the screening criteria for what the research team was looking for in terms of respondents. Again, this was created in consultation with stakeholders, and was used as a launching board for creating the screener. I created the screener as well as its accompanying logic to accurately screen out participants at several points throughout the initial questions of the survey.
After the screener criteria and logic was set, I constructed a draft list of survey questions that took the team’s objectives and hypotheses and adapted them into concrete questions. As this was an international survey, the focus was primarily on quantitative questions to save on time for translation as well as potential open-ended coding during analysis. Additionally, many questions utilized logic to ask follow-on questions based on previous responses. Stakeholders were able to add their input, which greatly aided the content of the questions that would be launched.
As this project would continue after the completion of my internship, I worked ahead to create a brief analysis plan and documentation based on feedback expectations and hypotheses. The documentation included an analysis spreadsheet to quickly move the survey results into a spreadsheet that allows for filtering of key results.
Finally, a draft presentation for key stakeholders was created. The presentation was based on templates that were used for previous research presentations, and included several sections that were relevant to the survey.
Key Learnings:
- Stakeholder management is essential with large research projects. This includes both internal and external stakeholders; keeping folks involved and in-the-loop helps speed up feedback and allows for better flexibility in the face of challenges.
- Third-party vendors, especially those that are new relationships, can add complexity. Not only are the vendor and the researcher working together, they are also building rapport and learning how each other work.
- Deferring to expertise is vital. During many points in this research, I was unsure how best to proceed – do I go with what I know? Do I go with what has been done before? Do I go with what is the most time efficient? I am extremely lucky that I had support from an amazing research team and mentor.
Reflections:
What went well – I had an amazingly supportive team that allowed me to get outside of my comfort zone and try new things within my research. It also allowed me to flex my abilities, especially in terms of coding survey logic and translating stakeholder needs into questions that can lead to actionable outcomes.
What could be improved – Keeping on top of both the research team, the unannounced project team, as well as the third-party vendor turned out to be quite difficult. Looking back, setting clearer timeline expectations and making sure adequate time is set aside for review is absolutely vital.
Overall, I believe that my work had a hugely positive effect within the ongoing work on Bungie’s unannounced project. I am incredibly proud of the work that I was part of, and am excited to see how the research team continues my work moving forward.
