These features of the ELTS need to be taken into account when interpreting the results. For example, while teacher responses offer insight to the culture of a building, they are still subjective data points akin to interviewing individual teachers. C-PEER and researchers took great care in designing this instrument in order to ensure that the data are reliable and valid across elementary schools.
Effective Learning Leader Survey
The Effective Learning Leader Survey (ELLS) was also co-designed with C-PEER and doctoral candidates/researchers as a way to better understand the role that participation in teacher leadership networks plays in supporting and retaining effective teachers in urban schools. Researchers wanted to understand how opportunities for collaboration and leadership (within and beyond the classroom) can increase teacher efficacy and effectiveness for STEM-foundational thinking, while improving the retention of highly effective teachers. The ELLS was designed to be completed in less than 20 minutes, and is approximately 30 questions (see Fig. 16). ELLS questions related to the STEM-foundational thinking and instructional activities utilized a four point Likert Scale, (1 is “Not at all true,” and 4 is “Very much like my school”), as well as a text response and multiple choice. The ELLS was also administered online using the Qualtrics Survey Software. Participants who took the survey were de-identified through the use of a participant constructed, confidential identification code, which allowed researchers to connect responses across schools without providing any identifiable personal information. Instructions for completion of the survey were accessed by the subjects at the provided URL. The results of the survey were compiled and analyzed by C-PEER.
Student Perception Survey
The Student Perception Survey (SPS) is designed to provide important feedback regarding teacher behaviors and the classroom environment. SPS results can point to strengths and opportunities for greater growth for teachers’ pedagogical practice. I focused primarily on how students perceived STEM-foundational instructional activities in the classroom. The survey (see Table 1) is comprised of 30 questions and can be administered in 45 minutes. According to the school district’s LEAP Handbook, “the SPS is administered once per year in the late fall to students in grades 3-12,” in order for administration and teachers to use results from the survey to make adjustments to instructional practices (LEAP, 2015). Responses are scored on a four point Likert Scale where 1 is “Never,” and 4 is “Always.” For this focus-study, the responses for items under the SPS construct of Facilitates Learning were analyzed. The results of the survey were compiled by the school district. Results for the participant schools provided by the district were analyzed for variance and correlation by C-PEER. The STEM-foundational thinking questions were derived a variety of sources (see Technical Report) (Seidel, et al. 2016).
LEAP Framework for Effective Teaching
Researchers specific to analyzing STEM-foundational thinking, examined results for teachers in participating schools on nine indicators on the framework for effective teaching (LEAP, 2015). From the Framework for Effective Teaching Observation Domain, these include:
- LE.1: Demonstrates knowledge of, interest in and respect for diverse students’’ communities and cultures in a manner that increases equity (Positive Classroom Culture and Climate).
- LE.3: Implements high, clear expectations for students’ behavior and routines (Effective Classroom Management).
- I.1: Clearly communicates the standards-based content-language objective(s) for the lesson, connecting to larger rationale(s) (Masterful Content Delivery).
- I.2: Provides rigorous tasks that require critical thinking with appropriate digital and other supports to ensure students’ success (Masterful Content Delivery).
- I.3: Intentionally uses instructional methods and pacing to teach the content-language objective(s) (Masterful Content Delivery).
- I.4: Ensures all students’ active and appropriate use of academic language (Masterful Content Delivery).
- I.5: Checks for understanding of content-language objective(s) (High-Impact Instructional Moves).
- I.6: Provides differentiation that addresses students’ instructional needs and supports mastery of content-language objective(s) (High-Impact Instructional Moves).
- I.7: Provides students with academically-focused descriptive feedback aligned to content-language objective(s) (High-Impact Instructional Moves
- I.8: Promotes students’ communication and collaboration utilizing appropriate digital and other resources (High-Impact Instructional Moves
 | ||
Figure 15. Sample questions from teacher survey
|
|
Q#
|
Item
|
SPS Category
|
|
1
|
My teacher listens to me.
|
Supports Students
|
|
2
|
My teacher helps me understand my mistakes so that I can do better next time.
|
Facilitates Learning
|
|
3
|
My teacher makes sure that the class rules are clear.
|
High Expectations of Students
|
|
4
|
My teacher makes learning interesting.
|
Facilitates Learning
|
|
5
|
In my teacher's class, I have to work hard.
|
High Expectations of Students
|
|
6
|
Q06: My teacher explains what we are learning and why.
|
Facilitates Learning
|
|
7
|
My teacher ignores me (reverse-coded).
|
Supports Students
|
|
8
|
My teacher wants me to think about things I learn and not just memorize them.
|
Facilitates Learning
|
|
9
|
My teacher encourages me to share my ideas.
|
Facilitates Learning
|
|
10
|
My teacher makes sure that we all treat each other with respect.
|
High Expectations of Students
|
|
11
|
My teacher helps me learn new things.
|
Facilitates Learning
|
|
12
|
My teacher uses examples in class that I understand.
|
Facilitates Learning
|
|
13
|
I like the way my teacher treats me.
|
Supports Students
|
|
14
|
In my teacher's class, we learn to correct our mistakes.
|
Facilitates Learning
|
|
15
|
My teacher hurts my feelings (not used in scoring).
|
Filtering Use Only (not used in scoring)
|
|
16
|
My teacher checks to make sure I understand.
|
Facilitates Learning
|
|
17
|
In my teacher's class, I have to think hard about the work I do.
|
High Expectations of Students
|
|
18
|
My teacher believes in me.
|
Supports Students
|
|
19
|
My teacher makes sure that students do what they're supposed to be doing.
|
High Expectations of Students
|
|
20
|
My teacher only accepts my best effort.
|
High Expectations of Students
|
|
21
|
My teacher is good at explaining things that are hard to understand.
|
Facilitates Learning
|
|
22
|
I get bored in my teacher’s class (not used in scoring).
|
Filtering Use Only (not used in scoring)
|
|
23
|
My teacher explains things in different ways.
|
Facilitates Learning
|
|
24
|
My teacher makes sure that students in this class behave well.
|
High Expectations of Students
|
|
25
|
In my teacher's class, I have to explain my answers.
|
Facilitates Learning
|
|
26
|
My teacher is nice to me when I need help.
|
Supports Students
|
|
27
|
My teacher makes sure I do my best in school.
|
High Expectations of Students
|
|
28
|
The rules in my teacher's class are fair.
|
Supports Students
|
|
29
|
My teacher knows when the class does not understand.
|
Facilitates Learning
|
|
30
|
My teacher cares about me.
|
Supports Students
|
Table 1. Questions from student perception survey with coding categories
