From Zero to a National Data Set in 2 Weeks: Reflections on a COVID-19 Collaborative Survey Project

Abstract

In March 2020, like much of the rest of the world, we went into lockdown. A week into our new reality, we decided to do a survey study about how people were experiencing the COVID-19 pandemic. In this piece, we describe what motivated us to do the study, how we went about it, and what others can learn from our experiences.

This piece presents our experiences collaborating on a COVID-19 study. We share our lessons learned about the logistics of working on something so in-the-moment in a large group, which was novel to some of the researchers involved. We reflect on why people at different life and career stages may want to embark on such an undertaking, how different perspectives can be integrated at varying phases of the study, and what challenges scholars may encounter when getting involved in something of this sort. First, we present some overall impressions and then each reflect on some personal specifics.

We are members of the Internet Use and Society Division in the Department of Communication and Media Research at the University of Zurich. Other than Will who was an incoming postdoctoral scholar in Chicago at the time of the pandemic, we are located in Zurich where a soft lockdown began on 16 March 2020. We spent the first few days finding our bearings. Most of us are not from Switzerland so frequent communication with far-away loved ones was top of mind as was obsessively following the news. As communication scholars who study Internet uses, this experience was not only personal, it also prompted research questions. How was the pandemic affecting people’s interpersonal communication? How were people keeping abreast of all the government rules and health recommendations? How were people of different life stages and home circumstances coping? But also: did we have the wherewithal to launch into a study on all this or was it more realistic to focus our energies on handling this unprecedented situation?

At first, we seemed headed for the latter approach. Eszter put out some feelers to see whether people were interested (completely optional!) in collaborating on a study. Reactions were lukewarm. Continued exchanges a few days later suggested interest from some team members, but hesitation to commit given the uncertainty about the workload. These were legitimate concerns; we were all facing a big unknown not just with respect to this possible study, but so much else in our lives. First lesson learned: to the extent possible, leadership should communicate expectations up front.

After more discussion, on 24 March, we decided to launch into the project, which would result in a 15-min draft survey a week later, and data collected on 1,374 American adults by 8 April. In the meantime, we secured funding for a Swiss version of the survey that we would field in three national languages (German, French, Italian) plus would go on to collecting data in Italy, too. Translating surveys to different languages and adapting questions to different cultures and national contexts was not trivial resulting in considerable additional work. A month later, we would field one more survey in the United States.

At the start, our prior experiences with surveys varied widely (from considerable to none). Survey quality was always top-of-mind, which requires drawing on prior literature for instrument development. Of course, this is not always possible with completely new phenomena, but even then, drawing on related questions from prior studies is important. To that end, we scoured versions of the US General Social Survey, the European Social Survey, the Swiss Household Panel, among others, in addition to relying on our own previous measures and input from numerous experts. Second lesson learned: expertise in the methods used is essential, but not everybody on the team has to have it. With the right guidance, everyone can contribute in important ways.

It is worth repeating: researchers must not compromise on quality in the interest of speed. The data we would collect would serve as the basis for many publications, including op eds, all of which may have policy implications and inform scholarly debates. The quality of an instrument includes a consideration of respondent fatigue. An endless survey is not an option, partly because that compromises data quality, but also out of ethical considerations. Our work took a lot of effort, not just because of the short timeline, but because we were committed to doing it well.

A note on logistics. We had already been using Slack for years so opted for that system for team communication. We created a separate project-focused Slack to organize the many components of the study such as instrument development, data collection, translations, calls for papers, links to other studies, data visualizations to disseminate information on social media, and separate publications that would result. At the time of this writing, in less than 2 months, the platform had accumulated over 4,000 messages. The shared system allowed us to keep a clear overview and record of all communication, work asynchronously, and track progress, all without an overload of email correspondence. Third lesson learned: give careful thought to what tools and communication methods meet project needs and researcher sanity best.

We also held group video chats several times a week. Our usual weekly group meeting was required to attend; additional joint coffee breaks were optional. These gave us a chance to talk about things unrelated to the project, check in to see how everyone was doing, hear about people’s family situations, and share tips on how we were all coping during lockdown. In these, we attempted to recreate some of the serendipitous interactions we usually have when working in neighboring offices. Fourth lesson learned: always make time for informal social exchange.

In the end, we all had to set aside other projects to devote time to what we all believed would be an important social scientific effort: learning how people were experiencing life in COVID-19 times. Although such worldwide phenomena may be rare (we hope so!), local sudden opportunities for research do arise (e.g., natural disasters, conflict) so our lessons learned should be applicable to varying potential future endeavors. In the rest of this piece, we share our personal reflections on the experience. We touch upon how, for both junior and senior scholars, involvement in such collaborative projects not only has research benefits, but also broader personal and professional advantages concerning collaboration, communication, and balancing competing priorities.

Eszter: For me, the project ended up being an extremely helpful distraction from following the news obsessively, which did not seem to do anything good for my mental health. Those working in healthcare were more than pulling their weight as were those in essential positions. As a social scientist, I felt that studying the social aspects of the pandemic was my way of contributing productively to the situation. In fact, suddenly, academics of all stripes were interested in how people were using digital media, as communication scholars specializing in Internet use research, it was important to have our expertise represented at the table. Overall, I am extremely proud of the work we achieved together and to have done it in a way that accommodated different team members’ needs.

Hao: Our first open conversation about who would be interested and comfortable working on a COVID-19 project was invaluable as it set the tone for the weeks that followed. While working hard, we checked in daily to see how everyone was coping. We tried to extend the same consideration to our respondents as we designed the survey. Being mindful of the emotional state people could be in amid the crisis meant carefully debating appropriate question wording, survey length, and data collection timing. Never before had I understood the importance of such nuance so well, as now I was experiencing the crisis myself. I also faced challenges. I had hoped to send out some manuscripts for review in the Spring, an always-urgent goal as a junior scholar aspiring to hold a professorship one day. But this now had to take lower priority. Eventually I realized that this was OK; these are challenging times for everyone. In fact, I am immensely grateful that I still got to do my job (unlike many others) and give back to society with our project.

Jaelle: As the most junior member of the team currently finishing my master’s degree, the project offered new experiences. I appreciated that everyone’s opinions and ideas were valued. Most team members took a lead in their topic of expertise, but everyone else was able to give input. This provided helpful insight into all aspects of survey creation. Designing the surveys at incredible speed was impressive, but also took much work. Since I am still taking classes, I had to do this amid transitioning to virtual instruction and to working from home. On the one hand, the project was a good distraction from worrying about the pandemic; on the other, it added some anxiety to an already stressful situation. This project has the questionable honor of being my longest-ever workday. There was always someone else working just as long though so I knew I was never alone in my efforts. Even with the pressure of wanting to get the survey out quickly, keeping a work–life balance was encouraged, including breaks. No one ignored the importance of the work, but neither did we forget that this was a hard situation for all of us.

Jonathan: I am in the first year of my doctoral studies. Before I started working on the project, I did not have much experience with surveys other than what I had learned in classes. At the beginning, I was rather overwhelmed by the sheer amount of information. Although I was mainly involved in one part of the study, I could hardly keep up with the constant flow of information about research questions, research models, related literature, survey items, and much more. I was missing the usual opportunity of asking colleagues in the office spontaneously for input. Every exchange had to be done through mediated communication. That felt complicated and more challenging for me than our usual way of in-person interactions. It sometimes prevented me from asking questions that I thought might seem silly. I do not think I would experience the same problems so strongly again because our current Slack channel could serve as a helpful step-by-step guide. I already use it as a kind of wiki to reconstruct all the different steps and decisions that we made in the process of creating the survey.

Will: As an ethnographer and generally solo author, what first struck me was the complexity of the teamwork this project required. Slack was instrumental, but our process was ultimately a matter of team ethos. Our approach was that anyone with a paper idea from the data could create a Slack channel for it and others could self-select into each others’ initiatives. We could each join a particular part of the study based on interests and then start contributing. Even on papers I did not plan to author, I read several channels for the pure benefit of learning more about the overall project. It was informative to see how we could construct survey questions to resonate with respondents during the outbreak of a novel pandemic. Digital scholars are used to the objects of our attention always changing. In the time of COVID-19, we faced the additional challenge of comprehending rapid changes in the way people think and act in the world.

Amanda: When Eszter first suggested a possible COVID-19 study, schools had just closed leaving my partner and me struggling to balance work and child care. Being involved in a new project, as important as it sounded, seemed utterly impossible. After a few weeks, the “new normal” set in and we found our groove. As the team completed the initial surveys and began additional data collection, I became involved. It was entirely different from past collaborations—work moved at lightning speed (it needed to) and urgency was palpable. After a morning homeschooling, I would return to dozens of messages on multiple Slack channels hashing out questions. Balancing time pressure while orchestrating multiple layers of logistics and collaborators took considerable decision-making prowess from our leadership. There were heartbreaking moments when we made decisions about cutting questions. I discovered that one can become highly attached to questions only conceived days (hours?) earlier. Working toward a high-quality survey meant that individual questions became less important and spurred refinements that improved content. (Could it be that our decision-making processes were mirroring the same worldwide tensions we were facing: protecting the public good versus infringing on individual rights?) Overall, documenting people’s COVID-19 experiences through survey work created many truly rewarding research moments.

Gökçe: Since this is the last year of my PhD, time is of the essence. I joined the project later than others, prioritizing my dissertation completion. Having listened to project updates during our weekly team meetings had me curious about how the group could manage to accomplish so much in so little time so when they decided to do another wave of data collection, I decided to join. All that was needed now was to come up with more creative ways to keep my cat entertained and away from my keyboard. I expected that joining the project would result in more interactions with team members, what I had not realized up front was all the excitement and struggle we would share. The project brought us all together and made me feel closer to the others despite physical distancing. I am glad I was able to fit in this experience of virtual research collaboration as I learned so much, which I now know is possible even while living the “new normal.”

Overall, while we hesitated at first, we are very happy to have embarked on this adventure. It gave our group something shared to work on during this difficult time and allowed us to contribute to an understanding of the pandemic. See our resulting publications at webuse.org/covid.