Qualitative Procedures

First, data was classified into categories on the Activity System Frameworks, using content analysis and constant-comparative structural coding (e.g., see Saldana, 2013). From the case study analysis protocol (e.g., see Yin, 2014) we looked for patterns of variables and analyzed the likelihood of designated outcomes based on the qualitative variable patterns. Researchers used a concept-matching approach (Kane & Trochim, 2007) to categorize the Activity System elements (rules/policies, community, tools, roles) into membership in common variable sets. Next, researchers analyzed the set-membership across case study sites (n=7) in relation to designated outcomes (in this case, STEM-foundational thinking and instructional activities, student achievement growth, both generally and for subgroups of students; and social-emotional supports of students in classrooms, as indicated by the student perception surveys). This “fuzzy-set qualitative comparative analysis” method (Ragin, 2000; Ragin, 2008) allowed researchers to assess which context and activity system variable combinations improved the likelihood of particular outcomes of interest (e.g.: STEM-foundational thinking). Finally, this, combined with the Activity System qualitative analysis, which looks within and across the “triangle” framework for conflicts among variables allowed researchers to highlight potential levers for systemic change within each of the urban elementary schools and classrooms. All survey responses were coded utilizing the appropriate scale and entered into SPSS for analysis. Reports were generated through SPSS to identify key findings in the quantitative data.

Conclusion

This study examines the relationship between STEM foundational thinking and instructional activities present in the Colorado elementary schools. Qualitative and quantitative data were collected and analyzed during the fall semester of 2015 through the spring semester of 2016 from seven participating schools and analyzed in order to answer the research question. Participants included teachers, instructional staff and school leaders, who participated in the Effective Learning Teacher Survey (ELTS), and the Effective Learning Leader Survey (ELLS). Extant data examined included the framework for effective teaching (LEAP) data, Student Perception Survey (SPS) results, the Colorado Department of Education (CDE) School Performance Framework (SPF), and each school’s Unified Improvement Plans (UIP), relevant trend (qualitative data), and archival structure data (e.g.: school schedules and team and committee workflows). An analysis of variance (ANOVA) and a principal components analysis (PCA) were utilized to find relationships and patterns among the variables. The purpose of this research was to ascertain what practices are in place for recruiting and engaging students of color in STEM curricula, as well as recommendations for creating a culturally relevant school culture (e.g.: an effective learning organizations). The findings of this study will contribute toward an understanding of how best to integrate STEM-foundational thinking and instructional activities into mainstream classroom curricula, so as to provide increased access and opportunity for traditionally underperforming students of color.

Methodology (Mixed Methods Comparative Cases)

In collaboration with the Center for Practice Engaged Education Research (C-PEER), I analyzed available and relevant trend data for participating schools and school districts, archival data (e.g.: school schedules and team and committee workflows), data from common teacher and leader survey developed for the project (including asking teachers about 21st century teaching methods identified from the extant literature), and extant data from the schools’ district about students’ perceptions of school and evaluation data about teachers. We focused on understanding “effective learning community” systems through the lens of STEM-foundational thinking. Effective school learning communities, both inside the classroom, among teachers, and in relationship to school leadership work together to support or hinder a STEM mindset in students, especially students of color. We collected data to help schools understand how school structures and resources (e.g.: time, curriculum/instructional programs, equity of access, procedures), climate (e.g.: trust, leadership systems. STEM culture in buildings), and personal aspects (e.g.: teachers’ efficacy, student persistence and motivation) intersect. While each of these has been the subject of research in focused, disconnected studies, our collaborative approach with C-PEER brought data from each element of the system together for a comprehensive look at the interacting factors for improving access and opportunity to STEM curricula.

For our triangulation research design we use a mixed-methods, multi-site, comparative case study, using both quantitative and qualitative processes, in order to measure STEM foundational thinking

and instructional activities at the elementary school level. In some instances, statistical conclusions were limited by small sample size, but we received a very high response rate from school teachers and leaders. Where survey data did not lend themselves to standard quantitative analysis, they could still be considered as qualitative findings. Specifically, I am answering: (a) What elementary school structures support students in STEM curricular areas? (b) Do these supports differ for sub-groups of students, i.e. students of color, students in poverty, and English language learners? (c) What are the components of elementary STEM opportunities to learn that foster interest, participation, and academic success in STEM content areas, especially for marginalized students of color? The modified cultural historical activity theory (CHAT) framework, which undergirds this study allows us as researchers to triangulate our data points and understand what STEM-oriented activities (object) and goal-directed actions lead to STEM foundational thinking. Activity systems analysis is a method that helps to capture multi-mediational processes in human activity (Engeström, 1987, 1999). For example, in researching school structures at the elementary school level that give STEM access and opportunity to students of color, we need to capture information that will inform us about teachers’ mediational processes. 


A triangulation design is the best choice of methodology in order to ascertain what practices are in place in effective learning organizations for recruiting and engaging students of color in STEM curricula. The scope of prior research focuses on separate lines of inquiry (e.g.: STEM perspectives, STEM frameworks, Critical Race Theory, Culturally Responsive Education). By using a triangulation, mixed-methods comparative case study design, we were able to modify an established framework (CHAT), collect several quantitative and qualitative data points, and analyze elementary schools for STEM foundational thinking. For example, something in the rules/policies corner conflicts with something in the students’ ability to take on the role of a collaborative peer with others in their classroom. Researchers chose participating schools in an urban public school district based on a range of academic performance. Researchers used the Colorado Department of Education (CDE) School Performance Framework (SPF) to identify elementary schools based on student achievement and student growth in the 2015-2016 academic school year (ASY).

Chapter III. Mixed Method Comparative Case Study Design

Presently, there is no established agreement on the proper methodology of integrating STEM in elementary schools. However, I felt that interdisciplinary STEM education will be most successful at the elementary level, due to the fact that students spend most of their academic day with the same teacher in all content areas. Research indicates that traditionally underserved, minoirtized population of students engage in problem-based learning activities (e.g.: STEM-foundational thinking, instructional activities and assessments) will exhibit an increased performance in their overall academic achievement, critical thinking skills, and cooperative learning strategies (Cole, 1995; Tharp, R.G., Doherty, R. W., Echevarria, J., Estrada, p., Goldenberg, C., Hillberg, R.S., & Saunders, W.M., 2004). Therefore, we invited seven urban elementary schools from a large urban district to

participate in research designed to understand the school systems that may support STEM-foundational thinking and activities. We focused on the elementary school level because it is in these early academic years where students find their interests in STEM either helped or hindered. Our research analyzed school performance and process data (both quantitative and qualitative) and each school received a report designed to help them incorporate the findings into school improvement efforts. Seven elementary schools agreed to participate. Participation with this project intended to simultaneously help individual schools learn with and from other sites engaged in similar work. Led by the Center for Practice Engaged Education Research (C-PEER) at the CU Denver School of Education and Human Development, this technical assistance brought research expertise to support school improvement efforts. It helped fill gaps in local staff time, bridge challenges accessing performance and process data, and provide access to additional resources and learning from other school sites.