The simple answer is that it is important to know what works, so that effective efforts can be replicated and unsuccessful efforts discarded or modified. Yet perhaps the more important answer to this question is that it is important for policymakers, leaders, teachers, and stakeholders to "think" like an evaluator, and continually strive to understand if what they are doing is working, and to continually improve - to be "improvement-oriented."
Evaluation findings are often politicized and interpreted in different ways. Even when the results of an evaluation appear to be accurate and point towards a particular action or change in policy, various stakeholders, groups, and organizations often have different perspectives as to the direction that should be taken as a result of specific evaluation findings. While it is always important that educators and policymakers look to pertinent research when making decisions about how to best improve public schools, it is equally, if not more crucial that educators evaluate and think critically about their work on an ongoing basis.
Public education has a history of adopting new strategies without carefully examining the research for these strategies, or building in mechanisms to evaluate the effectiveness of new strategies. It is when educators fail to actively assess and evaluate the effectiveness of potentially innovative ideas that "innovations" becomes "the latest fad" and are easily dismissed by parents, teachers, educational leaders and policymakers. The spread of innovation has the potential to lead to transformational change, but doing so entails that all educators become conscious evaluators of their own work and practice.
Public education is filled with numerous policies and regulations that are intended, in one way or another, to ensure that students receive an equitable and high-quality education. We have policies to ensure fairness and equitable access and opportunity for all students, policies to regulate the use of federal and state funds, and policies that serve as incentives and sanctions to motivate districts, schools, teachers, and students to improve.
This policy environment, broadly defined as the myriad of rules, regulations and written expectations within which educators and students live and make decisions, is not always as coherent as educators may desire. Rather, the collection of policies and regulations tend to create what can best be characterized as policy noise, which serves to hinder educators from working effectively and making decisions that best meet the needs of their organizations, their teachers, and students. Coupled with the reality that individuals are inclined to interpret policy based on their own perspective and mental mindset, it easy to understand how educational leaders often feel constrained and unable to find clarity within the policy noise that serves as their everyday working environment.
"Truly innovative and sustainable change requires a re-examination of how education policies impact improvement efforts, and renewed efforts to construct a productive and enabling policy environment conducive to innovative change."
What is the focus of our evaluation services?
Evaluation design and data analysis: We work with clients to conceptualize and design high quality evaluations and data analysis strategies that are meaningful, practical, and designed to capture the information needed to address and accomplish client objectives and goals.
During this work, we: (1) engage clients (e.g., teams from state departments of education, districts, schools, and constituents) in an iterative and collaborative process that provides time for individuals to discuss the purpose, goals, and questions that will provide the foundation for the evaluation design; and (2) convene a set of facilitated and semi-structured conversations and meetings are used to collect information and provide opportunities for clients to articulate their theory of action for the strategy, intervention, or policy being implemented, and inform the evaluation design that will best assess the impact of their work.
Formative and impact evaluation: We design and conduct evaluations of education policies, interventions, and strategies, with a particular focus on evaluations of district and school improvement strategies and state-level initiatives.
Our evaluation services are customized, and typically includes (1) an initial evaluation design session in order to clarify evaluation questions and identify and agree upon data collection strategies and methodologies, followed by (2) formal evaluation activities (e.g., data collection, surveys, focus groups and interviews, quantitative and qualitative data analysis, dissemination) designed to meet client needs while ensuring the integrity and rigor of evaluation activities and findings.
All evaluation design and consultation involves work towards the following outcomes:
What is the focus of our policy analysis work?
We work with state and district leaders to jointly analyze policy, in some instances conducting a policy audit, focused on identifying areas of policy coherence (e.g., situations in which policy works to support improvement efforts) and areas of policy fragmentation (e.g., situations in which policies and regulations tend to constrain improvement efforts). We use our analysis of the policy landscape to design specific action strategies for how policy might be adapted, modified, or communicated in ways that will enhance overall policy coherence.
Policy Analysis and Audit activities occur over a period of four months, and include: (1) the co-development of an analytic policy framework for understanding the relationship among policies and regulations, and their intended and real impact; (2) an historical analysis of policy shifts and trends over time, with respect to the particular focus or need of the client; and (3) a detailed analysis of how local, state and federal policy intersect, as a means of exploring and illuminating areas of policy coherence and areas of policy fragmentation.
Policy Implementation activities involve ongoing work with a state or district policy design team (or an existing state or district leadership team), focusing on how to construct, modify, and communicate new or existing policies to build upon areas of policy coherence (strengths) and minimize areas of policy fragmentation (for instance, by reducing the number of planning requirements that schools must meet).
What are the outcomes of this work?
Capacity building outcomes include: