Measuring and improving Pedagogical Content Knowledge of student assistants in introductory physics classes

Background

Learning assistants (LAs) are an important agent to transforming traditional lecturing into inquiry oriented. LAs increase the teacher-student ratio that enables timely support tailored for student-paced exploration especially in gateway courses with large enrollments. Empirical evidence has been gathered supporting the effectiveness of LA programs in promoting students' conceptual understanding, entailing positive attitudinal shifts towards physics, and lowering the DFW rate in a class. However, the mechanism remains unclear. It has been hypothesized that LAs in inquiry-oriented courses use inquiry teaching strategies to scaffold students' learning, such as questioning. Unfortunately, there have been few studies directly investigating LA-student interactions and their impact on students' conceptual learning. The barriers are two-fold.

First, there is a lack of attention on physics content knowledge embedded in teacher questioning. Existing studies have primarily focused on the formats or interactive patterns of questioning and their impact in prompting student articulation. Limited attention is paid to how content knowledge embedded in questioning serves as guidance, directing student learning toward a learning objective when it goes off on a tangent. Consequently, LAs may form a hands-off view of inquiry teaching that students can achieve sophisticated understanding merely by articulating their thoughts, or they may resort to lecturing to address students' difficulties.

Secondly, there is a lack of instruments for efficient assessment of LAs' questioning competencies. Existing studies primarily use discourse analysis for questioning analysis. This method yields detailed insights about questioning but is impractical for multiple videos due to coding workload. Existing studies are typically limited to around 10 videos for analysis, introducing potential bias in assessment because questioning is context contingent. Consequently, LA educators do not have instruments to assess and develop LAs' questioning competencies even though they advocate for this method.

objectives

The objectives of this project are three-fold

1) Design and validate instruments for quantitative assessment of LAs' pedagogical content knowledge in questioning, including:

a. A coding scheme to assess LAs' questioning practice from class videos.

b. Free response questions to assess LAs' PCK-Q that could predict their practice of questioning.

c. Likert-scale or multiple-choice questions that can be automatically graded for large-scale analyses.

2) Build a model about how LAs' PCK-Q contributes to students' conceptual learning, including physics content knowledge and critical thinking skills.

3) Apply the method of instrument development from videos to written questions in other STEM domains.

theoretical framework

GUIDING QUESTION

Following Vygotsky's theory, learning involves students expanding their knowledge scheme into the zone of proximal development, where they cannot perform a task independently but with the support of more knowledgeable others (MKO). Teachers, as critical MKOs in class, play a pivotal role in facilitating this transition. Compared to direct instruction, teachers' support through questioning fosters student-paced exploration and maintains their agency in knowledge construction.

In this project, we focused on one specific type of question, i.e., guiding questions, representing teachers' intervention to student learning when it derails. Different from open-ended probing questions that gauge students’ understanding, guiding questions restrict students’ answer along a designated direction by referring to specific sources of information with guidance embedded in the answers. Guiding questions are typically in a chain that gradually directs students toward the learning objective.

PEDAGOGIAL CONTENT KNOWLEDGE IN THE CONTEXT OF QUESTIONING (PCK-Q)

To ask effective guiding questions, LAs need Orientation toward questioning (O), content knowledge represented in a Curriculum (C), knowledge of Students’ strengths and difficulties in their understanding (S), and knowledge of Instructional strategies (I) to determine guiding questions that could bridge the gap in students’ understanding. Employing the theoretical framework of pedagogical content knowledge (PCK), we referred to those four competencies as O-C-S-I.

project overview

To assess LAs’ PCK in questioning (PCK-Q), we started by analyzing LAs’ questioning practices in their interaction with students captured by class videos. We segregated each video into vignettes regarding different learning objectives and developed codes for LAs’ utterances, such as probing question (pq) and guiding question (gq). The patterns of codes in a vignette determine the vignette level representing the method and impact of LAs’ intervention on students’ learning regarding the learning objective. For example, the pattern of “pq(s)-gq(s)” represents a vignette of effective intervention from LAs through guiding questions, labeled as “Qa”. The frequencies of vignette levels from multiple videos of an LA could indicate their PCK-Q. For instance, LAs with a strong questioning orientation may occasionally have direct-instruction vignettes, but their overall questioning vignettes (including Qa) should be more frequent across videos.

Our video coding scheme enables objective assessment of questioning practices from multiple contexts. However, this method is time-consuming and unwieldy for large-scale analysis. We selected representative vignettes and converted them into written questions where LAs analyze students’ understanding from provided LA-student dialogues and determine appropriate responses. We developed rubrics to derive LAs’ O-C-S-I from their answers to these questions. We validated the free-response questions and established the connection between LAs’ PCK-Q measured by free-response questions and questioning practices captured by class videos. To further simplify written assessment, we converted free-response questions into Likert-scale ones that could be automatically graded. We applied common categories of answers to free-response questions as options for Likert-scale questions that represent different levels of PCK-Q. LAs’ preferences over these options, indicated by their ratings, suggest their PCK-Q. We preliminarily validated Likert-scale questions and suggested equations to quantify O-C-S-I from ratings.

With these instruments we developed, we built models about how LAs’ PCK-Q contributed to students’ conceptual understanding and critical thinking. Meanwhile, we applied the procedure of instrument development in elementary science for preservice teachers (PSTs). We validated the elementary-science version of the instruments, compensated the limit of a small sample size of LAs with over 100 PSTs, and yielded similar findings between college LAs and elementary PSTs for cross-validation. Most importantly, we justified the feasibility of applying our procedure of instrument development in other STEM domains, providing valuable support for STEM education researchers and educators.