Research Methodologies, Data Collection, and Analysis at MailChimp: A Case Study

Introduction

MailChimp develops a web application that allows users to create, send, and track email newsletters (see Figure 1). Many different industries use MailChimp’s email platform to communicate to their particular audiences in a variety of ways: e-commerce and retail stores send messages promoting products or sales; non-profit organizations spread awareness of their causes; artists showcase their work; authors share their words; and entertainers promote their events. Though most often thought of as a marketing tool, MailChimp—at its most basic description—is a publishing platform from which its users can write and distribute information online.

Figure 1.

MailChimp is an online service that allows customers to design, send, and track emails

(©2013, MailChimp. Used with permission).

With millions of customers using MailChimp to share information, sell products, promote events, and more, knowing who these users are—and why and how they use MailChimp—is paramount. The MailChimp Research team plays a crucial role in gathering and sharing information that will help the company understand and improve the MailChimp user experience. Though this chapter focuses on software in general and MailChimp in detail, the methodologies and practices described herein apply to other industries and information services.

Research Methods

Each year, MailChimp’s directors announce several broad themes for the company to focus its energies on—such as “Mobile,” “Automation,” or “International.” These themes are kept intentionally broad, so that each team within the organization can interpret and tackle them as they see fit. Designers and developers, for example might build new tools or features that address those themes; the Knowledge Base team might expand their support documents to include more information on those tools; the marketing team might focus on how to talk about the year’s themes with current and prospective customers. Much of the work conducted throughout the year can be directly tied to one of the year’s themes.

The MailChimp Research team relies on these themes to guide research projects and maintain focus throughout the year. For instance, when Mobile was one of the company’s annual themes, the Research team made mobile usage and behavior a focus or underlying topic in their research projects. They surveyed customers who use the MailChimp mobile apps, included mobile-related questions in customer interviews, and gathered information on the different apps customers rely on to conduct business throughout their day.

This chapter uses case study examples from MailChimp’s research workflow to explain and describe research methods for collecting, analyzing, and reporting on results.