Methodology (Mixed Methods Comparative Cases)

In collaboration with the Center for Practice Engaged Education Research (C-PEER), I analyzed available and relevant trend data for participating schools and school districts, archival data (e.g.: school schedules and team and committee workflows), data from common teacher and leader survey developed for the project (including asking teachers about 21st century teaching methods identified from the extant literature), and extant data from the schools’ district about students’ perceptions of school and evaluation data about teachers. We focused on understanding “effective learning community” systems through the lens of STEM-foundational thinking. Effective school learning communities, both inside the classroom, among teachers, and in relationship to school leadership work together to support or hinder a STEM mindset in students, especially students of color. We collected data to help schools understand how school structures and resources (e.g.: time, curriculum/instructional programs, equity of access, procedures), climate (e.g.: trust, leadership systems. STEM culture in buildings), and personal aspects (e.g.: teachers’ efficacy, student persistence and motivation) intersect. While each of these has been the subject of research in focused, disconnected studies, our collaborative approach with C-PEER brought data from each element of the system together for a comprehensive look at the interacting factors for improving access and opportunity to STEM curricula.

For our triangulation research design we use a mixed-methods, multi-site, comparative case study, using both quantitative and qualitative processes, in order to measure STEM foundational thinking

and instructional activities at the elementary school level. In some instances, statistical conclusions were limited by small sample size, but we received a very high response rate from school teachers and leaders. Where survey data did not lend themselves to standard quantitative analysis, they could still be considered as qualitative findings. Specifically, I am answering: (a) What elementary school structures support students in STEM curricular areas? (b) Do these supports differ for sub-groups of students, i.e. students of color, students in poverty, and English language learners? (c) What are the components of elementary STEM opportunities to learn that foster interest, participation, and academic success in STEM content areas, especially for marginalized students of color? The modified cultural historical activity theory (CHAT) framework, which undergirds this study allows us as researchers to triangulate our data points and understand what STEM-oriented activities (object) and goal-directed actions lead to STEM foundational thinking. Activity systems analysis is a method that helps to capture multi-mediational processes in human activity (Engeström, 1987, 1999). For example, in researching school structures at the elementary school level that give STEM access and opportunity to students of color, we need to capture information that will inform us about teachers’ mediational processes. 


A triangulation design is the best choice of methodology in order to ascertain what practices are in place in effective learning organizations for recruiting and engaging students of color in STEM curricula. The scope of prior research focuses on separate lines of inquiry (e.g.: STEM perspectives, STEM frameworks, Critical Race Theory, Culturally Responsive Education). By using a triangulation, mixed-methods comparative case study design, we were able to modify an established framework (CHAT), collect several quantitative and qualitative data points, and analyze elementary schools for STEM foundational thinking. For example, something in the rules/policies corner conflicts with something in the students’ ability to take on the role of a collaborative peer with others in their classroom. Researchers chose participating schools in an urban public school district based on a range of academic performance. Researchers used the Colorado Department of Education (CDE) School Performance Framework (SPF) to identify elementary schools based on student achievement and student growth in the 2015-2016 academic school year (ASY).

Planned Study Design

Our proposed research design was to use a mixed-methods, multi-site, comparative case study, using both quantitative and qualitative processes. Examining both quantitative and qualitative methods allowed for a more complete analysis of the research questions and findings (Tashakkori & Teddlie, 1998), as well as provide a broader basis for generalization of results (Simons, 1996). For example, this study intended to answer: (a) What elementary school structures support students in STEM curricular areas? (b) Do these supports differ for subgroups of students, i.e. students of color, students in poverty, and English language learners? (c) What are the components of elementary STEM opportunities to learn that foster interest, participation, and academic success in STEM content areas, especially for marginalized students of color? Our mixed-methods approach was guided by these research questions, and “ultimately reflect[ed] a value of both subjective and objective [STEM] knowledge” (Johnson & Onwuegbuzi, 2004; Tashakkori & Teddlie, 2003). We planned to use data sources centered on (a) recruiting schools to develop a focused priority research agenda; (b) conducting trend analyses of participating schools; and (c) a collaborative analysis of the research questions and recommendations for further action. It is important to note that mixed methods studies, such as this one, are strengthened when research teams are comprised of individuals from a variety of disciplines (Simons, 1996). This allows researchers to engage in a “mixed methods way of thinking” while discussing “different ways of seeing, interpreting, and knowing” about the data (Sammons, 2010, p. xi). Our C-PEER research team includes doctoral candidates from various K12 backgrounds including expeditionary learning, professional development, distributive leadership, and coaching teams. 

          Planned instrumentation. The C-PEER team planned to use instruments organized with specific constructs. For example, teacher surveys will measure the shared values and vision of teaches, as well as support conditions and relationships within the building. We also intended to collect information from both teachers and administration using the (Teaching, Empowering, Leading, and Learning) TELL Colorado surveys. These anonymous instruments would allow researchers to assess teaching conditions within school buildings, as well as throughout participating districts. Since these surveys would be designed to support school and district improvement planning, as well as inform policy decisions, our hope is that they would be extremely reliable and valid measures. It is our intention to design this study that will yield high-quality evidence for educators and school districts.

Chapter III. Mixed Method Comparative Case Study Design

Presently, there is no established agreement on the proper methodology of integrating STEM in elementary schools. However, I felt that interdisciplinary STEM education will be most successful at the elementary level, due to the fact that students spend most of their academic day with the same teacher in all content areas. Research indicates that traditionally underserved, minoirtized population of students engage in problem-based learning activities (e.g.: STEM-foundational thinking, instructional activities and assessments) will exhibit an increased performance in their overall academic achievement, critical thinking skills, and cooperative learning strategies (Cole, 1995; Tharp, R.G., Doherty, R. W., Echevarria, J., Estrada, p., Goldenberg, C., Hillberg, R.S., & Saunders, W.M., 2004). Therefore, we invited seven urban elementary schools from a large urban district to

participate in research designed to understand the school systems that may support STEM-foundational thinking and activities. We focused on the elementary school level because it is in these early academic years where students find their interests in STEM either helped or hindered. Our research analyzed school performance and process data (both quantitative and qualitative) and each school received a report designed to help them incorporate the findings into school improvement efforts. Seven elementary schools agreed to participate. Participation with this project intended to simultaneously help individual schools learn with and from other sites engaged in similar work. Led by the Center for Practice Engaged Education Research (C-PEER) at the CU Denver School of Education and Human Development, this technical assistance brought research expertise to support school improvement efforts. It helped fill gaps in local staff time, bridge challenges accessing performance and process data, and provide access to additional resources and learning from other school sites.