Edtech Company

No one knows an edtech product better than the teachers and students who use it everyday. Piloting your edtech product with real teachers and students is the best way to understand how – and how well – it actually works. Through piloting, you can:

  • Gain precious insight into how teachers and students use your product, and how it can be improved
  • Understand what implementation of your product looks like in actual classrooms, and how this changes from teacher to teacher, school to school, and district to district
  • Acquire valuable efficacy data, and learn whether your product actually produces the outcomes you intend

Ready to get started? Check out our tips and resources below.

Overview

Why pilot?

When it comes to education innovation, the term “pilot” can refer to many different things. In this toolkit, we focus on what the members of the Learning Assembly refer to as “implementation” and “efficacy” pilots. (To see the Learning Assembly’s definitions of a range of pilot types, view our Evaluation Taxonomy here.)

“Implementation” pilots are conducted to evaluate how an edtech product, an instructional practice, or combination of the two, can be implemented and adapted to meet the needs of a specific classroom, school or other learning environment. For example, how can an edtech tool be used as part of a personalized learning station rotation model to augment math or literacy instruction?  Or, how can an edtech product help increase student engagement during whole-group instruction?

“Efficacy” comes into the mix when you want to know whether the product or practice actually produces the outcomes intended – for example, increased student engagement or agency, increased reading or math proficiency, improved critical thinking skills, etc.

What can we learn?

With the findings from an implementation pilot, you can better understand how to implement your edtech product or instructional practice in a particular classroom, school or other learning environment.

With the findings from an efficacy pilot, you can understand whether a product or practice actually has the result you intended, from improvement in student growth to increased student agency.

With what edtech products?

You can pilot products from across the product development continuum, from recently launched products to more mature, established ones. Pre-launch products, or products still in development, are not a good fit for implementation or efficacy pilots.

How long will it take?

This depends on the product or practice you want to pilot. You’ll need enough time for a full implementation to measure the specific learning objectives, depending on your pilot design.

Pilot Process
Plan

Be clear on intent

Be sure of your pilot objective, and that piloting is the best way to meet that objective. While many schools that take part in pilots are looking for a product that will meet the needs of their teachers and students, pilots should not simply be a vehicle for sales. Implementation and efficacy pilots require a significant investment of time and resources, but the reward is informed feedback on how to improve your product, an understanding of how to implement your product well in varying classroom environments, and valuable efficacy data.

Find a research partner

There’s a lot to consider when planning for and conducting an implementation or efficacy pilot, and university or support organization partner can bring valuable expertise in research design, data collection and analysis. Results published by a third-party pilot partner with strong research expertise can serve as valuable validation, and insights gleaned from their findings can inform product improvement.

Do a data check

Be clear on what data you need to capture, and how you will securely share this data with school teams and other research and support partners. Validate the guidelines from your school and district partners up front, and test share data files prior to piloting to allow time to address any bugs or gaps.

Plan for support

Educators need support to implement new instructional practices and edtech tools, and pilots work best when product implementation is consistent. Decide up front on a plan for support and troubleshooting both in advance of and during a pilot – how much staff time will this require, vs. what can you commit? Will support be virtual or in-person? Etc.

Resources for Planning

Each Learning Assembly member organization has their own, unique approach to piloting. Below are resources that these organizations have used while planning for their respective pilots, including general pilot program overviews, applications, and planning checklists. Have an idea for a needed resource? Let us know at info@learningassembly.org.

EdTech Pilot Data Worksheet

Digital Promise

This worksheet will help you think about what data you need to collect in to determine whether the ed tech product is meeting your school or districts' identified goals.

EdTech Pilot Goals Worksheet

Digital Promise

This worksheet will help you think about your goals for the ed tech pilot and how those goals align with your school and/or district's vision for teaching and learning.

EdTech Pilot Checklist

Digital Promise

A checklist to help a school or district plan for a successful edtech pilot.

Examples of Team Goals

LearnLaunch

Included in this document are examples of three team goals: one elementary and two middle school level.

Fuse RI Early Adopter Coaching Methods

Highlander Institute

This document describes the format and nature of the coaching that early adopters receive for blended and personalized instruction, which they receive in addition to being provided access to an edtech product in their pilots.

Highlander Institute's "Our Process"

Highlander Institute

This document provides a brief overview of Highlander Institutes’ process for facilitating edtech pilots and supporting educators to adopt the best practices of personalized learning.

Instructional Goals & Metrics Guide

iZone NYC

This guide helps teachers identify and formalize their classroom need by walking them through a process to create their inquiry question and identify the metrics they will use to assess if the product solved their specific problem.

LEAP Pilot Network Overview

LEAP Innovations

The LEAP Pilot Network is an intense 18 month experience for school teams to implement a personalized learning pilot and edtech tool in their building. It incorporates a mix of change management, professional development, research, market vetting, and in-classroom coaching to ensure a high fidelity implementation can be achieved to inform what should or should not be scaled.

MassNET Pilot Cycle

LearnLaunch

A brief infographic explaining the LearnLaunch MassNET piloting process.

Planning Your Implementation Guide

iZone NYC

This guide will help educators plan their pilot implementation through discussion questions and an organizer to keep everyone on track.

Professional Development Scope and Sequence

LEAP Innovations

The LEAP Pilot Network provides intensive professional development on personalized learning design and implementation. We utilize a workshop-seminar model to personalize our sessions, allowing teams to select programming that best meets your school’s interests and needs.

Recruiting Early Adopters

Highlander Institute

This document dives deeply into the second phase of the Blended and Personalized Learning Development Framework: Recruiting Early Adopters. It contains guidance for identifying, motivating, and supporting educators who are interested in adopting the best practices of blended and personalized instruction.

School Pilot Prep Checklist

LEAP Innovations

This checklist is for schools and covers all the “to do’s” before piloting, from IT resources, training, communication, and scheduling. Reviewing this checklist requires engagement with a wide range of school staff and vendor partners.

Back To Top
Partner

Create - or subscribe to - a shared vision

To be done well, implementation and efficacy pilots require a big commitment from all parties involved. Be sure that your school partners are ready to implement your edtech product with fidelity. Likewise, make sure that you and your product team fully understand and are committed to your school partner’s learning objective for the pilot – improving literacy skills, for example, rather than tech for tech’s sake.

Check for readiness

Ensure that you have the capacity – your team and your technology – to fully support a pilot across the classrooms, schools, and even districts required. On the school side, be sure of at least a baseline comfort with technology, or plan for extra supports for teachers who lack familiarity.

And infrastructure

Not all schools have the same degree of tech readiness and infrastructure. Ensure that school partners have the capacity needed to effectively pilot within the pilot’s specific context. What bandwidth is required? Is a particular type of device required? Does a classroom need to be 1:1? Etc.

Resources for Partnering

Each Learning Assembly member organization has their own, unique approach to piloting. Below are resources that these organizations have used while selecting partners for their respective pilots, including applications and rubrics for review. Have an idea for a needed resource? Let us know at info@learningassembly.org.

Company Pilot Prep Checklist

LEAP Innovations

This checklist is for edtech companies and covers the basics for engaging in pilots.

Company Review Rubric

iZone NYC

Rubric used by iZone staff, DOE staff, and community partners to review product applications in school year 2015-16 prior to additional review and selection by school teams.

Company and Teacher Application Questions

Silicon Valley Education Foundation

These are the application questions for teacher teams and edtech companies that apply to participate in the SVEF iHUB pilots.

EdTech Strategy Review

LEAP Innovations

This document helps teams make sure they are asking all the right questions about IT, edtech, and academic integration. It is designed to be worked through collaboratively with the Director of IT/Ops and academic leadership.

LEAP Pilot Network Company Application

LEAP Innovations

Each year, LEAP has an open call for edtech companies to apply for the next cohort of the Pilot Network. The application consists of two parts, a written portion and three 2-minute videos.

LEAP Pilot Network School Application

LEAP Innovations

School teams must apply to participate in the LEAP Pilot Network and become part of the cohort, this simple application helps check for key characteristics that set the teams up for success.

Product Considerations List

iZone NYC

This list helps teachers review products against common criteria and narrow their list of potential pilot products. Everything from bandwidth to free time to learn the ins and outs of the product are considered in this list.

Sample Master Services Agreement

LEAP Innovations

LEAP Innovations has a Master Services Agreement that is signed between LEAP and any edtech company that participates in our Pilot Network program. It is designed to provide clear and transparent expectations for both parties during the course of the engagement and establishes the edtech company as a sub-contractor for the piloting organization.

Sample Professional Services Agreement

LEAP Innovations

LEAP Innovations has a Professional Services Agreement that is signed between LEAP and any school that participates in our Pilot Network program. It is primarily used with our schools that look to us to provide a service agreement. It is designed to provide clear and transparent expectations for both parties during the course of the engagement.

Sample School Application

LearnLaunch

These are the application questions LearnLaunch uses to ask schools when they apply to participate in the MassNET Project.

Sample School Commitment Letter

LearnLaunch

This letter of commitment explains the LearnLaunch MassNET project and outlines the roles and responsibilities of all participants.

School Team Structure and Expectations

LEAP Innovations

This document outlines the roles and expectations for each member of the pilot team. It is designed to give strong ownership to middle-tier leader (Pilot Lead).

Short Cycle Evaluation Challenge Company Q&A

iZone NYC

An explanation of the Short-Cycle Evaluation Challenge pilot, specifically focused on edtech company roles, benefits, responsibilities.

Short Cycle Evaluation Challenge Teacher Q&A

iZone NYC

An explanation of the Short-Cycle Evaluation Challenge pilot, specifically focused on teacher roles, benefits, responsibilities.

Vendor On-boarding Call Sample Agenda

Citizen Schools

A brief agenda to plan for a quick check-in with vendor representatives to share high level feedback and plan for future collaboration.

Back To Top
Match

Match well

To ensure strong matches – and strong implementations – both school teams and edtech companies should be well prepared in advance of finalizing pilot partnerships. Make sure you have a clear understanding of the school’s learning objectives, and that your product is a good fit to meet these.

Also, buy-in is critical for successful partnerships. Ideally, all members of a school team – teachers and school leaders – will be 100 percent on board with the objectives of the pilot and the product selected. But this is not always the case. Be aware of differing perspectives, and be prepared to differentiate support.

Resources for Matching

Each Learning Assembly member organization has their own, unique approach to piloting. Below are resources that these organizations have used while supporting smart matches between school teams and edtech companies, including strategies for aligning on a pilot vision and goals and event-planning guides. Have an idea for a needed resource? Let us know at info@learningassembly.org.

Match Day Deck for EdTech Partners

LEAP Innovations

At Match Day, school teams meet with edtech company representatives whose products passed our curation process to select which tool they will pilot. Prior to Match Day, we review this deck with invited companies to set expectations about the pilot process and prepare for the day.

Product Dossier Template

iZone NYC

Pilot organizations or districts can use this template to create a one pager with the most important information about products being considered for a pilot.

Back To Top
Implement

Provide dedicated support

Even with careful planning, school teams require significant support throughout the pilot period. The easier you can make piloting for teachers, the smoother implementation will be. Make sure to provide a dedicated pilot contact to help trouble-shoot and conduct trainings as needed. Likewise, encourage school teams to appoint a “pilot lead” to serve as your point of contact.

Consider context

In providing for supports, consider the unique context and needs of your partner school, teachers, and their students. Get to know your partners well, and plan ahead for potential implementation challenges that may be unique to their circumstances. Understand the classroom mechanics already in place, and work together with educators to make implementing your product work smoothly within them.

Resources for Implementing

Each Learning Assembly member organization has their own, unique approach to piloting. Below are resources that these organizations have used while implementing their respective pilots, including guides for teacher interviews and focus groups, observation protocols, and resources for capturing student feedback. Have an idea for a needed resource? Let us know at info@learningassembly.org.

Blended & Personalized Learning Development Framework

Highlander Institute

The BPL Development Framework outlines a multi-year process for districts to engage in as they move toward adopting blended and personalized best practices.

Blended Learning Best Practices Walkthrough Tool

Highlander Institute

This is the tool Highlander Institute staff and observers use to collect data during classroom observations.

Classroom Observation Protocol

iZone NYC

This document helps companies or other stakeholders record observations meaningfully when they visit a classroom that is piloting a product. It was developed as a template for use by iZone staff, company representatives, and the Center for Children & Technology research team.

LEAP Learning Framework

LEAP Innovations

The LEAP Learning Framework is based on research and educator experience to define the key elements of a learner's experience in a personalized learning setting. It sets the research agenda for LEAP Innovations and is used for guiding classroom, school, and institution level design.

LEAP Personalized Learning Surveys

LEAP Innovations

The LEAP Personalized Learning Surveys are designed to measure the teaching practices and classroom conditions that enable personalized learning. The surveys ask teachers about their practices and students about their classroom experiences.

MassNet Beginning of Year Teacher Interview Protocol

LearnLaunch

This brief interview protocol focuses on teacher readiness to use technology.

MassNET Classroom Observation Protocol

LearnLaunch

To be used in conjunction with scripted observation notes, this protocol highlights key issues for reflection and measurement when observing classrooms.

MassNET End of Year Teacher Interview Protocol

LearnLaunch

This brief interview protocol focuses on teacher reflections on technology and personalized learning.

Quick Check with Students

iZone NYC

This document helps teachers try out a product with a small group of students to find and plan for any issues they might run into when they introduce and pilot the product with the whole class.

Sample Implementation Plan

Highlander Institute

Highlander Institute uses implementation plans to clearly outline objectives and work in service to those objectives. It is also used as a record of all services provided to a school or district team.

Site Observation Planning Protocol & Summary Tool

Citizen Schools

A document to guide and collect data during site observations by a member of the national team

Teacher Focus Group Protocol

LearnLaunch

This protocol has key questions for teacher focus groups.

Teacher Monthly Ratings

LearnLaunch

These logs take approximately 5 minutes and are a good way to track teacher experiences and product ratings over time as well as collect open-ended responses.

Teacher/Product Team Meeting Template

Silicon Valley Education Foundation

This meeting template is for teacher teams within a school site (or within a district) that are evaluating the same product.

Tuning Protocol

iZone NYC

This document is a version of our tuning protocol, a structured process for giving and receiving feedback to refine teams’ work, used by companies and teacher teams during the second workshop of the pilot.

Back To Top
Share

Consider your audience

Pilot results will be interesting not just to you and your partners, but also to other educators facing similar issues, working in similar contexts, or with similar questions. Share results broadly, allowing other edtech companies, teachers, administrators, and the field at large to learn from your experience.

Resources for Sharing

Each Learning Assembly member organization has their own, unique approach to piloting. Below are resources that these organizations have used while sharing out the results from their respective pilots, including sample reports. Have an idea for a needed resource? Let us know at info@learningassembly.org.

Evaluating EdTech

Digital Promise

This tool will help district leaders evaluate studies on edtech product effectiveness to decide whether it is necessary to run a pilot.

Pilot Study Brief Template

Digital Promise

This template can be used to succinctly share pilot results within a district and with external peers and partners.

Sample Pilot Report

Highlander Institute

This is a sample report that summarizes the data collected in a half-year edtech pilot.

Sample Product Interim Progress Report

LearnLaunch

This is an example of a supplemental product report that we send to product providers in the fall, after the year of use. It is provided in combination with our overall evaluation report and is intended to provide additional information specific to that product. Other product analyses are also included in our overall report.

Sample Pilot Template

Silicon Valley Education Foundation

This is a sample report that summarizes the data collected in a half-year edtech pilot.

School Leader Interview Questions

Citizen Schools

This document helped gather high level feedback from school leaders about user satisfaction, instructor competency, and challenges/lessons learned for each product tested.

Back To Top