Toolkit Applied

An organization was looking to understand and address two things about the non-users of its evidence on quality professional learning (those in their target audience who were not engaging with their resources):

1. What are the barriers preventing school districts from finding their resources?
2. What are the barriers preventing school districts from trusting their resources enough to interacting with them?

In our commitment to bolster this organization and enact tangible change, we crafted a bespoke methodology employing essential tools from our toolkit. Here, we present a brief synopsis before delving into the finer nuances.

Approach

We initiated diagnostic interviews with the organization’s staff to gain a comprehensive understanding of their needs. Additionally, we conducted exploratory interviews with school district leaders who are responsible for purchasing professional learning (PL) services (buyers).

These interviews involved usability tests of the organization’s website and aimed to qualitatively evaluate perceptions of the organization, focusing on key trust factors.

Using insights from these diagnostics, we developed a survey to delve deeper into trust levels. Subsequently, we utilized these survey findings to design and implement three experiments , specifically focusing on identifying the most impactful imagery and language for online content and email campaigns.

Impact

Based on our diagnostic findings, we identified a lack of awareness among buyers regarding the organization’s professional learning offerings. Although some resources garnered high trust levels compared to the broader PL ecosystem, buyers expressed the lowest level of trust in the organization’s ability to fulfill agreements with schools diligently and cater to specific school needs as a PL vendor.

To address this, we conducted behavioral experiments to assess buyers’ trust in various messaging and communication strategies. In so doing, we demonstrated that optimizing email bodies (design and language) can yield up to 2x more link clicks, and that adjusting subject lines can raise open rates by up to 6%. Guided by these insights, we collaborated with the organization to design an optimized email campaign to attract new users.

Quality seals or quality certifications are used in different ways across the professional learning, core curriculum, and education technology (EdTech) ecosystems. An organization providing EdTech product quality certifications was looking to answer the following questions about all district buyers in general:

1. How do district buyers use product quality certifications in their procurement decisions?
2. How can the organization enhance the impact (awareness and engagement) of their quality seal/certificate?

To support this organization, we went through a series of tools to both refine the questions further and systematically address them. Below we highlight a summary of these activities, then dive into the details.

Approach

We interviewed key staff in the organization to better understand their needs and their process, including the intended plan of engagement with the organization’s quality seal. For example, what would the organization prefer that district buyers do upon seeing the seal? What are the resources (or additional information about product certifications) tied to the seal?

We assessed the experience of district buyers on the organization’s website through usability tests to better understanding users’ broader barriers utilizing their resources, then conducted an exploratory survey to understand the experience of district buyers on engaging with EdTech quality seals (product certifications) generally.

Impact

The exploratory interviews and survey revealed that considerable potential for converting neutral district buyers into active users of product certification, with availability and accessibility identified as the primary barriers. We implemented a 5-second test on various alternative badge designs to strengthen engagement with product certification.

The findings reveal that badges designed by TDL outperform the current design, exhibiting positive attributes like clarity of message (e.g. the organization’s original badge was more often mistaken for an EdTech provider logo), reliability, professionalism of the issuing organization, and trustworthiness of the issuing organization. This was accomplished by making the visuals more intuitive and language on the badge more easy to understand. Furthermore, our MaxDiff experiment identified that in conference settings, the organization should emphasize transparency, guideline-alignment, and relevance to all districts, regardless of their unique needs.

Numerous organizations, particularly those of smaller or recent establishment, face the challenge of broadening their user base, especially when district buyers lack awareness of their organizational presence. An organization in this predicament was looking to understand and address two things with all district buyers but especially about district buyers in specific states of interest:

1. What are the biggest barriers preventing school districts from finding their resources?
2. What are the barriers preventing school districts from trusting the organization’s resources enough to interact with them?

To help this organization expand their reach, we leveraged key tools to better understand the information-seeking behaviors of their target audiences.

Approach

Our approach commenced with diagnostic interviews targeting buyers in the organization’s states of interest. These interviews delved into their information-seeking behaviors, adoption processes, and specific needs related to the organization’s resources, aiming to discern whether the organization could address these needs. Data from these interviews informed the design of a survey, enhancing generalizability and uncovering preferences without relying solely on self-reported data.

Within these surveys , A/B tests were conducted to ascertain optimal channels and assets for the organization’s marketing efforts. Subsequently, insights from the survey guided the organization in reevaluating its strategy to align more effectively with user preferences in the identified states of interest.

Impact

The exploratory interviews and survey revealed that considerable potential for converting neutral district buyers into active users of product certification, with availability and accessibility identified as the primary barriers. We implemented a 5-second test on various alternative badge designs to strengthen engagement with product certification.

The findings reveal that badges designed by TDL outperform the current design, exhibiting positive attributes like clarity of message (e.g. the organization’s original badge was more often mistaken for an EdTech provider logo), reliability, professionalism of the issuing organization, and trustworthiness of the issuing organization. This was accomplished by making the visuals more intuitive and language on the badge more easy to understand. Furthermore, our MaxDiff experiment identified that in conference settings, the organization should emphasize transparency, guideline-alignment, and relevance to all districts, regardless of their unique needs.