School/District Team

Finding what works for one team of teachers can be the first step in finding what will – or won’t – work across an entire school or district. With the right support and structures in place, piloting can be a great way for school and districts teams to get started with education innovation. Through piloting, you can:

  • Try personalized learning practices on a smaller scale, and iterate as you go
  • Identify promising edtech products to address your specific needs, and see how they work in your specific context, in real time
  • Better understand how to effectively implement personalized learning and education technology
  • Inform purchasing decisions

Ready to get started? Check out our tips and resources below.

Overview

Why pilot?

When it comes to education innovation, the term “pilot” can refer to many different things. In this toolkit, we focus on what the members of the Learning Assembly refer to as “implementation” and “efficacy” pilots. (To see the Learning Assembly’s definitions of a range of pilot types, view our Evaluation Taxonomy here.)

“Implementation” pilots are conducted to evaluate how an edtech product, an instructional practice, or combination of the two, can be implemented and adapted to meet the needs of a specific classroom, school or other learning environment. For example, how can an edtech tool be used as part of a personalized learning station rotation model to augment math or literacy instruction?  Or, how can an edtech product help increase student engagement during whole-group instruction?

“Efficacy” comes into the mix when you want to know whether the product or practice actually produces the outcomes intended – for example, increased student engagement or agency, increased reading or math proficiency, improved critical thinking skills, etc.

What can we learn?

With the findings from an implementation pilot, you can better understand how to implement an edtech product or instructional practice in a particular classroom, school or other learning environment.

With the findings from an efficacy pilot, you can understand whether a product or practice actually produced the result you intended, from improvement in student growth to increased student agency.

With what edtech products?

You can pilot products from across the product development continuum, from recently launched products to more mature, established ones. Pre-launch products, or products still in development, are not a good fit.

How long will it take?

This depends on the product or practice you want to pilot. You’ll need enough time for a full implementation to measure the specific learning objectives, depending on your pilot design.

Pilot Process
Plan

Find a research partner

As a school or a district team, consider working with a university or piloting organization partner. There’s a lot to consider when planning for and implementing this kind of pilot, and these organizations can bring expertise in research design, data collection and analysis, instructional practice design, and selecting/vetting edtech products.

Be specific

Be clear on the learning objective of the pilot, and understand exactly what you want to learn and how exactly you’re going to measure it, from the specific teaching practices to the efficacy of an edtech tool. Again, a knowledgeable research partner can be critical to ensuring that your pilot design is truly aligned to what you want to learn.

Do a data check

Be clear on what data you need to capture, and how edtech companies will securely share this with school teams and other research and support partners. Likewise, be clear about what data you will share back with partners – researchers, piloting organizations, and edtech product partners as applicable – and how you’ll do so. Establish clear guidelines up front, and test share data files prior to piloting to allow time to address any bugs or gaps.

Plan for support

Educators need support to implement new instructional practices and edtech tools. Decide up front on a plan for this support, including professional development prior to starting pilots, to allow school teams a chance to come together and develop the practices, and/or decide on the tool they want to pilot. Hands-on support during the pilot period will also be critical, both from researchers or organizations supporting instructional practice, and the edtech companies supporting and trouble-shooting edtech implementation.

Resources for Planning

Each Learning Assembly member organization has their own, unique approach to piloting. Below are resources that these organizations have used while planning for their respective pilots, including general pilot program overviews, school and company applications, and planning checklists. Have an idea for a needed resource? Let us know at info@learningassembly.org.

Cohort Support Discussion Focus Areas

Citizen Schools

An overview of key discussion topics for pilot leaders meeting across school contexts to share practice and provide feedback to each other.

EdTech Pilot Data Worksheet

Digital Promise

This worksheet will help you think about what data you need to collect in to determine whether the ed tech product is meeting your school or districts' identified goals.

EdTech Pilot Goals Worksheet

Digital Promise

This worksheet will help you think about your goals for the ed tech pilot and how those goals align with your school and/or district's vision for teaching and learning.

EdTech Pilot Checklist

Digital Promise

A checklist to help a school or district plan for a successful edtech pilot.

Examples of Team Goals

LearnLaunch

Included in this document are examples of three team goals: one elementary and two middle school level.

Fuse RI Early Adopter Coaching Methods

Highlander Institute

This document describes the format and nature of the coaching that early adopters receive for blended and personalized instruction, which they receive in addition to being provided access to an edtech product in their pilots.

Highlander Institute's "Our Process"

Highlander Institute

This document provides a brief overview of Highlander Institutes’ process for facilitating edtech pilots and supporting educators to adopt the best practices of personalized learning.

Instructional Goals & Metrics Guide

iZone NYC

This guide helps teachers identify and formalize their classroom need by walking them through a process to create their inquiry question and identify the metrics they will use to assess if the product solved their specific problem.

Introductory Guide to Education Technology for NYC Educators

iZone NYC

We created this guide to support educators with small-scale implementation of instructional edtech tools by sharing insights, tips, and lessons learned from real-life cases of integrating education technology.

LEAP Pilot Network Overview

LEAP Innovations

The LEAP Pilot Network is an intense 18 month experience for school teams to implement a personalized learning pilot and edtech tool in their building. It incorporates a mix of change management, professional development, research, market vetting, and in-classroom coaching to ensure a high fidelity implementation can be achieved to inform what should or should not be scaled.

MassNET Pilot Cycle

LearnLaunch

A brief infographic explaining the LearnLaunch MassNET piloting process.

Planning Your Implementation Guide

iZone NYC

This guide will help educators plan their pilot implementation through discussion questions and an organizer to keep everyone on track.

Professional Development Scope and Sequence

LEAP Innovations

The LEAP Pilot Network provides intensive professional development on personalized learning design and implementation. We utilize a workshop-seminar model to personalize our sessions, allowing teams to select programming that best meets your school’s interests and needs.

Recruiting Early Adopters

Highlander Institute

This document dives deeply into the second phase of the Blended and Personalized Learning Development Framework: Recruiting Early Adopters. It contains guidance for identifying, motivating, and supporting educators who are interested in adopting the best practices of blended and personalized instruction.

Sample Time Strategies

LEAP Innovations

Many shifts for personalized learning require new layouts of time. This resource contains common tips and tricks for creative scheduling.

Back To Top
Partner

Create a shared vision for educators

To be done well, implementation and efficacy pilots require a big commitment from both school leadership and classroom teachers. Ensure the pilot feels like an important initiative for the school from both sides – this removes  barriers and allows teachers to feel empowered to move forward. Also, be sure everyone is on the same page around the learning objective for the pilot – improving literacy skills, for example, or enabling student choice, rather than tech for tech’s sake.

Pick promising edtech partners

Given the length and intent behind implementation and efficacy pilots, start with promising edtech products that are specifically meant to enable the pilot’s learning objective. Look not just at the product, but the team behind it, checking for education and content-area expertise. Check for stability (will this product still be available in five years?), ability to scale (can it effectively serve an entire direct or districts?), and the depth and breadth of content available (does it address the needs of a wide range of learners, and for a long enough period of time?).

Check for readiness

Among the team that will be piloting, be sure of at least a baseline comfort with technology, or supports for teachers who lack familiarity. Also, make sure your edtech product partner has the capacity to fully support a pilot across the classrooms, schools, and even districts required. An interest in research on the company side is also key – while you are likely looking for a product that will ultimately meet the needs of their teachers and students, pilots should not simply be a vehicle for sales.

And infrastructure

Not all schools have the same degree of tech readiness and infrastructure. Ensure you have the capacity needed to effectively pilot within the pilot’s specific context. What bandwidth is required? Is a particular type of device required? Does a classroom need to be 1:1? Etc.

And innovation

To pilot effectively, schools must consider not just the piloted practices and products themselves, but also the policies and structures that must be put in place to enable innovation. This includes adjusting schedules or building structures to allow for teacher collaboration, opening lines of communication to leadership, reviewing and adjusting grading policies, and instilling an overall willingness to and vision for change.

Resources for Partnering

Each Learning Assembly member organization has their own, unique approach to piloting. Below are resources that these organizations have used while selecting partners for their respective pilots, including school and company applications, as well as rubrics for review. Have an idea for a needed resource? Let us know at info@learningassembly.org.

Company Pilot Prep Checklist

LEAP Innovations

This checklist is for edtech companies and covers the basics for engaging in pilots.

Company Review Rubric

iZone NYC

Rubric used by iZone staff, DOE staff, and community partners to review product applications in school year 2015-16 prior to additional review and selection by school teams.

Company and Teacher Application Questions

Silicon Valley Education Foundation

These are the application questions for teacher teams and edtech companies that apply to participate in the SVEF iHUB pilots.

EdTech Strategy Review

LEAP Innovations

This document helps teams make sure they are asking all the right questions about IT, edtech, and academic integration. It is designed to be worked through collaboratively with the Director of IT/Ops and academic leadership.

Evaluating EdTech

Digital Promise

This tool will help district leaders evaluate studies on edtech product effectiveness to decide whether it is necessary to run a pilot.

LEAP Pilot Network Company Application

LEAP Innovations

Each year, LEAP has an open call for edtech companies to apply for the next cohort of the Pilot Network. The application consists of two parts, a written portion and three 2-minute videos.

LEAP Pilot Network School Application

LEAP Innovations

School teams must apply to participate in the LEAP Pilot Network and become part of the cohort, this simple application helps check for key characteristics that set the teams up for success.

Product Considerations List

iZone NYC

This list helps teachers review products against common criteria and narrow their list of potential pilot products. Everything from bandwidth to free time to learn the ins and outs of the product are considered in this list.

Sample Master Services Agreement

LEAP Innovations

LEAP Innovations has a Master Services Agreement that is signed between LEAP and any edtech company that participates in our Pilot Network program. It is designed to provide clear and transparent expectations for both parties during the course of the engagement and establishes the edtech company as a sub-contractor for the piloting organization.

Sample Professional Services Agreement

LEAP Innovations

LEAP Innovations has a Professional Services Agreement that is signed between LEAP and any school that participates in our Pilot Network program. It is primarily used with our schools that look to us to provide a service agreement. It is designed to provide clear and transparent expectations for both parties during the course of the engagement.

Sample School Application

LearnLaunch

These are the application questions LearnLaunch uses to ask schools when they apply to participate in the MassNET Project.

Sample School Commitment Letter

LearnLaunch

This letter of commitment explains the LearnLaunch MassNET project and outlines the roles and responsibilities of all participants.

School Pilot Prep Checklist

LEAP Innovations

This checklist is for schools and covers all the “to do’s” before piloting, from IT resources, training, communication, and scheduling. Reviewing this checklist requires engagement with a wide range of school staff and vendor partners.

School Team Structure and Expectations

LEAP Innovations

This document outlines the roles and expectations for each member of the pilot team. It is designed to give strong ownership to middle-tier leader (Pilot Lead).

Short Cycle Evaluation Challenge Company Q&A

iZone NYC

An explanation of the Short-Cycle Evaluation Challenge pilot, specifically focused on edtech company roles, benefits, responsibilities.

Short Cycle Evaluation Challenge Teacher Q&A

iZone NYC

An explanation of the Short-Cycle Evaluation Challenge pilot, specifically focused on teacher roles, benefits, responsibilities.

Vendor On-boarding Call Sample Agenda

Citizen Schools

A brief agenda to plan for a quick check-in with vendor representatives to share high level feedback and plan for future collaboration.

Back To Top
Match

Match well

To ensure strong matches – and strong implementations – prepare both your piloting team and edtech company partner in advance of finalizing the partnership. School teams should be on the same page around the learning objective of their pilot, and how potential edtech pilot partners might meet these needs. Edtech partners should also have a clear understanding of your learning objectives.

Also, buy-in is critical for successful partnerships. Make sure all of your piloting team members – school leaders and classroom teachers – have a say in company selection. And support strong decision-making – don’t get taken in by all the bells and whistles a product has to offer. Stay true to the agreed-upon learning objectives, and which edtech product best aligns to those needs.

Resources for Matching

Each Learning Assembly member organization has their own, unique approach to piloting. Below are resources that these organizations have used while supporting educator partners in selecting edtech products to pilot, including strategies for aligning on a pilot vision and goals, tips for educators on selecting edtech, and event-planning guides. Have an idea for a needed resource? Let us know at info@learningassembly.org.

Match Day Deck for EdTech Partners

LEAP Innovations

At Match Day, school teams meet with edtech company representatives whose products passed our curation process to select which tool they will pilot. Prior to Match Day, we review this deck with invited companies to set expectations about the pilot process and prepare for the day.

Match Day Overview

LEAP Innovations

As part of the Pilot Network process, LEAP matches its educators with vetted edtech products. This document is a walkthrough of LEAP’s process for establishing relationships between companies and pilot teams on Match Day, recommended timetables for the event, and considerations and rationale for the format.

Pitch Games Event Checklist

Silicon Valley Education Foundation

Event checklist for planning a "shark tank" style event.

Product Dossier Template

iZone NYC

Pilot organizations or districts can use this template to create a one pager with the most important information about products being considered for a pilot.

Product Review Guidelines

iZone NYC

This document serves at a comprehensive quick-start guide for teacher teams to compare products ahead of making a selection for a pilot. It shares product considerations and best practices for reviewing as a team.

Back To Top
Implement

Be prepared

Before pilots begin, the teachers and school leaders involved in the pilot need dedicated time to plan. Structured professional development sessions, workshops, or simply common planning time provide teams an opportunity to come together. During these opportunities, teams should discuss learning objectives, design and refine instructional practices to be piloted, plan for integration and implementation challenges, consider school-wide impact, and thoughtfully choose the edtech products that best meet their needs.

Require dedicated support

Even with careful planning, school teams require significant support throughout the pilot period. Make sure edtech companies provide a dedicated pilot contact to help trouble-shoot and conduct trainings as needed. Likewise, appoint a “pilot lead” within your piloting team to serve as the contact point for the edtech company, and to ensure collaboration and fidelity of implementation on the school side.

Resources for Implementing

Each Learning Assembly member organization has their own, unique approach to piloting. Below are resources that these organizations have used while implementing their respective pilots, including guides for teacher and company interviews, observation protocols, and resources for capturing student feedback. Have an idea for a needed resource? Let us know at info@learningassembly.org.

Blended & Personalized Learning Development Framework

Highlander Institute

The BPL Development Framework outlines a multi-year process for districts to engage in as they move toward adopting blended and personalized best practices.

Blended Learning Best Practices Walkthrough Tool

Highlander Institute

This is the tool Highlander Institute staff and observers use to collect data during classroom observations.

Blended Lesson Planning Template

Highlander Institute

Coaches at Highlander Institute use this lesson planning template to co-plan blended lessons with early adopter teachers. It encourages teachers to group their students intentionally, create multi-modal learning activities, and anticipate the technical and management challenges that may arise in a blended classroom.

Classroom Observation Protocol

iZone NYC

This document helps companies or other stakeholders record observations meaningfully when they visit a classroom that is piloting a product. It was developed as a template for use by iZone staff, company representatives, and the Center for Children & Technology research team.

Entrance Tickets

LEAP Innovations

In this spotlight of a Learner Demonstrated strategy, Mr. Scott Frauenheim, School Director, and Ms. Connie Scalzetti, teacher at CICS West Belden, provide step by step details on how she uses daily entrance tickets to help students demonstrate their level of proficiency and determine their own learning plan for the day.

Goal Setting

LEAP Innovations

Educators at Lovett Elementary School use regular student-teacher conferences focused on student data and goals to build a Learner Led environment and drive student engagement. These resource show you how it works in Ms. Moody-Shumaker’s classroom.

LEAP Learning Framework

LEAP Innovations

The LEAP Learning Framework is based on research and educator experience to define the key elements of a learner's experience in a personalized learning setting. It sets the research agenda for LEAP Innovations and is used for guiding classroom, school, and institution level design.

LEAP Personalized Learning Surveys

LEAP Innovations

The LEAP Personalized Learning Surveys are designed to measure the teaching practices and classroom conditions that enable personalized learning. The surveys ask teachers about their practices and students about their classroom experiences.

MassNet Beginning of Year Teacher Interview Protocol

LearnLaunch

This brief interview protocol focuses on teacher readiness to use technology.

MassNET Classroom Observation Protocol

LearnLaunch

To be used in conjunction with scripted observation notes, this protocol highlights key issues for reflection and measurement when observing classrooms.

MassNET End of Year Teacher Interview Protocol

LearnLaunch

This brief interview protocol focuses on teacher reflections on technology and personalized learning.

Quick Check with Students

iZone NYC

This document helps teachers try out a product with a small group of students to find and plan for any issues they might run into when they introduce and pilot the product with the whole class.

Sample Implementation Plan

Highlander Institute

Highlander Institute uses implementation plans to clearly outline objectives and work in service to those objectives. It is also used as a record of all services provided to a school or district team.

Sample Time Strategies

LEAP Innovations

Many shifts for personalized learning require new layouts of time. This resource contains common tips and tricks for creative scheduling.

Site Observation Planning Protocol & Summary Tool

Citizen Schools

A document to guide and collect data during site observations by a member of the national team

Student Choice

LEAP Innovations

Educators at Patrick Henry Elementary School use student choice during literacy rotations – with teacher check-ins – to create an environment where students own their learning paths. These resources show you how it works in Ms. Figueroa’s classroom.

Teacher Focus Group Protocol

LearnLaunch

This protocol has key questions for teacher focus groups.

Teacher Monthly Ratings

LearnLaunch

These logs take approximately 5 minutes and are a good way to track teacher experiences and product ratings over time as well as collect open-ended responses.

Teacher/Product Team Meeting Template

Silicon Valley Education Foundation

This meeting template is for teacher teams within a school site (or within a district) that are evaluating the same product.

Tuning Protocol

iZone NYC

This document is a version of our tuning protocol, a structured process for giving and receiving feedback to refine teams’ work, used by companies and teacher teams during the second workshop of the pilot.

Back To Top
Share

Consider your audience

Pilot results will be interesting not just to the school and edtech companies involved, but also to other educators facing similar issues, working in similar contexts, or with similar questions. Share results broadly, allowing other teachers, administrators, and the field at large to learn from your experience.

Resources for Sharing

Each Learning Assembly member organization has their own, unique approach to piloting. Below are resources that these organizations have used while sharing out the results from their respective pilots, including sample reports. Have an idea for a needed resource? Let us know at info@learningassembly.org.

Pilot Study Brief Template

Digital Promise

This template can be used to succinctly share pilot results within a district and with external peers and partners.

Sample Pilot Report

Highlander Institute

This is a sample report that summarizes the data collected in a half-year edtech pilot.

Sample Product Interim Progress Report

LearnLaunch

This is an example of a supplemental product report that we send to product providers in the fall, after the year of use. It is provided in combination with our overall evaluation report and is intended to provide additional information specific to that product. Other product analyses are also included in our overall report.

Sample Pilot Template

Silicon Valley Education Foundation

This is a sample report that summarizes the data collected in a half-year edtech pilot.

School Leader Interview Questions

Citizen Schools

This document helped gather high level feedback from school leaders about user satisfaction, instructor competency, and challenges/lessons learned for each product tested.

Back To Top