A Systematic Review of Problem-Solving Skill Development for Students in STEM Education

TS. Nguyễn Thị Quỳnh Anh (và các cộng sự)

 

Abstract. In the current educational landscape, incorporating active learning activities centred around STEM (Science, Technology, Engineering, and Mathematics), themes have emerged as a potential solution for educators aiming to cultivate students' problem-solving skills. Nevertheless, a dearth of comprehensive research exists concerning the development of problem-solving skills specifically within the realm of STEM education. To bridge this gap, our study undertook a systematic review by utilizing relevant articles and implementing the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) methodology. Adhering to the prescribed steps within the PRISMA model, an extensive evaluation of 29 articles was conducted, spanning the period from 2018 to 2023, focusing on problem-solving skill development for students in STEM education. Through meticulous content analysis and synthesis, this research elucidated key insights pertaining to publication years and countries of origin, employed research methodologies, extracted keywords, identified limitations, and suggested avenues for future investigation. The findings revealed several influential factors that shape the cultivation of integrated teaching competence among students engaged in STEM education. This article bears significance for aspiring educational researchers, school administrators, and mainstream teachers vested in the exploration of this important subject matter.

Keywords: education; STEM; capacity; problem solving; students 

1. Introduction

In the contemporary era of technology, nations engage in competitive endeavours by leveraging their intellectual acumen and adaptability to the rapid advancements in information and technology (Kurt & Benzer, 2020). Technology is widely recognized as a reservoir of knowledge that nurtures innovative ideas  and fosters adaptability within the realm of science (Delahunty et al., 2021; UNESCO, 2011). Education, serving as the optimal conduit, paves the way to accomplish these feats (Astuti et al., 2021; Kurt & Benzer, 2020). However, to enable individuals to generate imaginative solutions and tackle problems from diverse perspectives, it is imperative for the education system to embrace interdisciplinary practices (Kurt & Benzer, 2020; Nguyen et al., 2023). From a psychological standpoint, problem solving is perceived as a situation that encompasses difficulties and challenges, thereby stimulating individuals to seek resolutions actively. The quest for solutions transcends a singular step and necessitates a multifaceted approach (Rasyid et al., 2023; Rochman et al., 2019). Concurrently, STEM education centres on addressing intricate, multidimensional real-world predicaments through the application of established research methodologies (Netwong, 2018; Puchongprawet & Chantraukrit, 2022). Within the framework of STEM, students actively engage in problem-solving activities, generating a myriad of solutions and approaches to address them. Consequently, the implementation of active learning activities grounded in STEM themes emerges as a viable solution for educators to foster and cultivate students' problem-solving skills (Nuangchalerm et al., 2020; Rasyid et al., 2023; Tran & Nguyen, 2020).

Recent studies have shown a significant rise in the volume of research focused on STEM education which has been positively associated with the enhancement of students' problem-solving skills. However, these investigations have not provided a comprehensive exploration of the interrelated issues, challenges, dominant keywords, or emerging research directions pertaining to the development of problem-solving capabilities within the context of STEM education. Consequently, our research endeavours to address this gap by conducting an extensive analysis of bibliographic data extracted from reputable databases, covering the period from 2018 to 2023. Through this approach, the study aims to shed light on the aforementioned aspects and enrich the understanding of problem-solving skill development in the realm of STEM education.

The results presented in Table 1 provide evidence of extensive research conducted in diverse disciplines pertaining to the enhancement of students' problem-solving abilities within the context of STEM education. However, it lacks a holistic understanding of the current state of knowledge in this domain. Thus, it is necessary to conduct a comprehensive study that will rapidly enable novice researchers to acquire the pertinent knowledge in this field. Additionally, it also underscores the necessity for further research that explores STEM education from a theoretical framework, examines the organization of instructional activities, investigates curriculum design, explores the effectiveness of learning tools, and analyzes dominant keywords. These emerging trends warrant thorough investigation to advance our understanding and facilitate the development of students' problem-solving skills. Accordingly, the primary objective of this article is to address the following research questions:

(i) What is the current trajectory of developing problem-solving capabilities for students in STEM education?

(ii) What are the dominant keywords that influence and shape the development of problem-solving capabilities for students in STEM education?

(iii) Which research methodologies are utilized to investigate the development of problem-solving capabilities among students in STEM education?

(iv) What are the inherent difficulties and challenges associated with fostering problem-solving capabilities in students within the context of STEM education?

(v) What are the emerging research directions for advancing the development of problem-solving capabilities among students in STEM education?

The findings of this study offer valuable insights for future research endeavours that concentrate on enhancing students' problem-solving capabilities within the realm of STEM education. Additionally, these findings hold significant relevance for individuals who are interested in devising plans and projects aimed at nurturing problem-solving skills through STEM education in both primary and secondary educational settings.

2. Materials and Method

This study focuses on investigating the development of problem-solving capabilities in students through STEM education. In order to collect data, the study followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines for literature search (Moher et al., 2009). PRISMA is a comprehensive set of factors derived from evidence-based systematic review reports designed to assist reviewers in providing transparent justifications for the purpose of the evaluation and outlining the authors' achievements. Previously, this framework has been successfully employed to address similar research objectives.

2.1. Data Collection

There are four main database directories that are commonly employed for establishing a comprehensive document repository: Scopus, Google Scholar, ScienceDirect, and ERIC. These databases encompass a wide range of resources, including titles, abstracts, and keywords, providing researchers with extensive literature coverage. In the field of educational science, these four databases are widely recognized as indispensable and trustworthy sources for conducting research and accessing scholarly information. Their inclusion ensures a robust and comprehensive exploration of the relevant literature in the field.

2.2. Search Criteria

A systematic collection of articles was conducted to populate our data repository, employing three specific search criteria. Firstly, a subject search term was utilized, requiring the presence of at least one term related to "STEM." Secondly, the search term included the phrases ("problem solving skill" OR "problem solving capacity" OR "problem solving competence"), which had to be found in the title, abstract, or author keywords of the articles. Thirdly, a scope search term was implemented which necessitated the usage of at least one of the phrases ("student" OR "pupil") in the title, abstract, or keywords of the articles.

Based on these rigorous criteria, the search yielded a total of 2,206 articles. Specifically, 324 articles were identified from Scopus, 1,460 articles from Google Scholar, 68 articles from ScienceDirect, and 354 articles from the ERIC online library. These articles were collected from 01 December 2023 to 30 December 2023. The collected document repository comprises articles that meet the specified search criteria, ensuring their relevance to the research topic. These selected articles provide valuable insights into the intersection of STEM education and problem-solving capabilities. They offer a comprehensive understanding of the subject matter and contribute to the existing body of knowledge in this field.

2.3 Data Screening

During the screening phase, the eligibility of the retrieved articles was determined by carefully evaluating the input criteria using the titles and abstracts of the extracted records. A specific set of parameters was established to streamline the selection process and identify pertinent documents:

i) Peer-reviewed evaluation: The articles underwent a rigorous peer-review process in recognized indexed databases. This criterion relied on the credibility of the journals which were subject to thorough peer-review procedures. Consequently, only articles available within these databases were considered for this review. 

ii) Relevance to the topic: The subject matter of each article was examined to ascertain its alignment with the development of problem-solving capabilities in STEM education for students.

iii) Language requirement: Publications were limited to those reported in the English language, ensuring accessibility and comprehension for a wider range of researchers.

iv) Timeframe restriction: Articles published between 2018 and 2023 were included in the selection process, allowing for a comprehensive examination of recent developments in the field. Throughout this phase, articles failing to meet any of the following criteria were excluded from the data repository:

i) Books and book chapters that consisted solely of summaries or introductions, as they typically lack the depth and rigour expected from scholarly articles;

ii) Articles not written in English, as they would pose a barrier to the wider research community and limit the dissemination of knowledge;

iii) Content unrelated to problem-solving capabilities in STEM education, ensuring that only articles directly relevant to the research topic were considered; and

iv) Articles published before 2018, as the focus was on recent research to reflect the current state of knowledge in the field.

By implementing these inclusion and exclusion criteria, a collection of articles was meticulously curated for the data repository. This process ensured that the selected articles were of high quality, directly aligned with the research topic, accessible to a broader audience through the English language, and representative of the most recent advancements in the field.

Figure 1 provides an overview of the application of the PRISMA method, which facilitates the systematic review process by establishing an information flow across multiple stages. Initially, a comprehensive search yielded a total of 2,206 articles from the selected data sources. To ensure data integrity, duplicate articles were eliminated based on their titles, resulting in a refined pool of unique articles. Subsequently, each article underwent individual screening to assess its relevance and alignment with the research scope, leading to the exclusion of 1,854 articles that did not meet the predetermined criteria. As a result, a subset of 352 articles remained eligible for full-text retrieval and further analysis. Among this subset, 11 articles could not be accessed owing to restrictions on availability. The authors diligently examined the eligibility reports and carefully excluded an additional 312 articles that did not meet the specified criteria. Ultimately, a final selection of 29 articles was included in this study. These articles were sourced from reputable databases, ensuring a robust and diverse dataset. To extract relevant information from the selected articles, various analytical techniques were employed. By employing these analytical approaches, a comprehensive understanding of the articles and their relevant data was obtained.

2.4 Data Encoding and Analysis

The MAXQDA software was employed for data extraction purposes, allowing for efficient handling of the research materials. By importing the articles into the software, coding schemes were created which served as a systematic framework for analysis. MAXQDA, a software tool recognized for its utility in qualitative research (Rädiker & Kuckartz, 2019), facilitated the organization, analysis, and exploration of qualitative or unstructured data sources. These encompassed diverse sources such as interviews, evaluations, articles, social media content, and web materials.

The coding schemes employed in the analysis comprised various elements of interest. These included the publication year, the country of the author, the title, the abstract, keywords, the methodology employed in the studies, challenges encountered, as well as research directions pertaining to the factors influencing the development of problem-solving capabilities among students through STEM education in secondary schools. 

3. Results and Discussion

3.1. Current Trajectory of Developing Problem-Solving Capabilities for Students in STEM Education

Figure 2 displays the trend in the number of articles published on the development of problem-solving capabilities for students in STEM education from 2018 to 2023. Notably, there has been a consistent increase in the number of articles in recent years. The years 2018 and 2019 witnessed a limited publication rate, with only one to two articles per year, indicating a relatively low level of attention devoted to this particular topic during that period. However, starting from 2020, there was a significant surge in the number of articles, reaching its peak in 2020 and 2021 with seven articles published each year. Subsequently, a slight decline in the publication rate occurred in 2022 and 2023, with six articles published annually.

These findings indicate that articles addressing the development of problem- solving capabilities for students in STEM education were scarce in 2018 and 2019 but have experienced a steady increase since then. This prompts the question: What factors contributed to the substantial increase in the number of articles in 2020? A comparative analysis of trends observed in other relevant comprehensive studies within the field of education, such as practical mathematics education (Phan et al., 2022), e-learning in higher education (Brika et al., 2021), and the application of information technology and communication in mathematics education (Trinh Thi Phuong et al., 2022), reveals similarities with this current research in 2020. However, when examining the field of integrated teaching in elementary education (Nguyen & Thai, 2023), a decrease in the number of articles in 2020 is evident, followed by a gradual increase in subsequent years.

Among the 29 articles included in the analysis based on country affiliation spanning from 2018 to 2023, Indonesia stands out with the highest number of publications (10 articles), followed by Turkey (7 articles) and Thailand (4 articles). Among a total of 11 countries investigated, these three countries exhibit the greatest interest in research pertaining to the development of problem-solving capabilities for students in STEM education. A visual representation of the data in Figure 3 confirms that Indonesia's contribution surpasses that of other nations in this field. Turkey and Thailand also demonstrate consistent engagement with this  topic, as evidenced by their annual publications, albeit with modest increases in their publication rates. While other countries have conducted research on the development of problem-solving capabilities for students in STEM education, their contributions have been sporadic. The remaining eight countries have each published only one article in this area.

When comparing the findings to various comprehensive studies conducted worldwide, such as practical mathematics education (Phan et al., 2022), the application of information technology and communication in mathematics education (Trinh Thi Phuong et al., 2022), and e-learning in higher education (Brika et al., 2021), it is evident that the geographical distribution of research in these fields prominently revolves around countries such as Indonesia, Malaysia, Thailand, the United States, the United Kingdom, the Netherlands, and China. Notably, Turkey does not feature among the countries explored in these studies.

However, in the context of research on integrated teaching in elementary education (Nguyen & Thai, 2023), it is noteworthy that only one article originates from Indonesia, and no articles are attributed to Thailand. Surprisingly, Turkey emerges as the country with the highest number of articles in this specific domain.

3.2. Dominant Keywords that Influence and Shape the Development of Problem-Solving Capabilities for Students in STEM Education

In order to address the research question at hand, a Wordcloud visualization technique was employed to illustrate the frequency of keywords, with larger words indicating higher frequencies of occurrence. The keywords were extracted from the relevant publications, with each publication typically containing 4-6 keywords. Figure 4 displays a compilation of 96 keywords identified within this study. Noteworthy among the most frequently encountered keywords in the publications are STEM (26 occurrences), education (16 occurrences), problem (15 occurrences), solving (14 occurrences), learning (12 occurrences), skills (11  occurrences), problem-solving (11 occurrences), and thinking (7 occurrences), among others. These prevalent keywords provide clear insights into the primary areas of interest explored by the authors in this research.

The subsequent group of frequently mentioned keywords comprises science (6 occurrences), achievement (4 occurrences), critical (4 occurrences), ability (3 occurrences), students (3 occurrences), engineering (3 occurrences), academic (3 occurrences), and design (3 occurrences), among others. These keywords shed light on the primary areas of focus within the studies, which predominantly revolve around subjects such as theoretical frameworks for STEM problem-solving, organization of STEM lessons, educational programmes, STEM learning tools, and student interests. Furthermore, the authors employed various additional terms, such as approach, century, creative, collaborative, inquiry, knowledge, mathematical, process, and others, to index their articles. These supplementary terms offer avenues for further exploration and serve as prompts for researchers to delve deeper into this subject matter. Upon examining other comprehensive literature reviews in the field of education (Ha et al., 2020; Nguyen & Thai, 2023; Phan et al., 2022; Trinh Thi Phuong et al., 2022), it becomes apparent that the themes of education, mathematics, learning, and students are commonly addressed and extensively investigated in these publications.

3.3. Research Methodologies to Investigate the Development of Problem-Solving Capabilities among Students in STEM Education

The authors of the publications have utilized a range of research methods and data collection approaches, including qualitative, quantitative, mixed methods, experimental, and synthesis analysis. These diverse methodologies are visually presented in Figure 5:

The analysis of Figure 5 demonstrates that studies focused on the development of problem-solving skills in STEM education primarily rely on qualitative and experimental research methods. Notably, these two methods are equally prevalent, with nine articles (31%) dedicated to each approach. Quantitative methods, employed by 17% of researchers, are utilized in five articles, while mixed methods are employed by 14% of researchers. Moreover, a synthesis analysis strategy, encompassing 7% of the articles, is employed in the context of national research.

Compared to previous investigations concerning the development of problem- solving skills in STEM education, this study goes a step further by providing a comprehensive description of the research methods employed. This detailed account serves as a valuable resource for aspiring researchers, facilitating their understanding of the widely adopted qualitative research method. Additionally, the integration of experimental research methods offers fresh perspectives and avenues for researchers seeking to delve deeper into this field.

3.4. Difficulties and Challenges associated with Fostering Problem-Solving Capabilities in Students within the context of STEM Education

After excluding articles that did not report their limitations, a total of 26 articles (accounting for 89.6% of the research groups' publications) identified and addressed various constraints. Table 3 provides an overview of these limitations. One prominent issue is the restriction of studies to specific teaching programmes or narrow topics within particular subjects or grade levels, which has been acknowledged in 11 articles (42.3%). The authors argue that this lack of breadth limits the generalizability of findings across different levels of education (Zengin et al., 2022).

Another limitation frequently acknowledged is the use of small sample sizes in experimental studies. Nine articles (34.6%) acknowledged this concern, recognizing the potential impact on the reliability and generalizability of their results. Additionally, six articles (23%) highlighted limitations due to geographic constraints, which hindered the broader applicability and international generalizability of their findings (Asigigan & Samur, 2021). Furthermore, seven articles (26.9%) emphasized that time constraints for exploration and creativity influenced STEM lessons, leading participants to prioritize exploration over active learning (Puchongprawet & Chantraukrit, 2022) .

The limited availability of experimental data was also addressed in six articles (23%). These articles noted that assessments were exclusively conducted using  Google Forms owing to students' unfamiliarity with other platforms. Additionally, the assessments revealed uneven coverage of technical, technological, mathematical, and scientific content areas (Amalina & Vidákovich, 2022). Moreover, students faced challenges due to their limited scientific and mathematical experience, making it difficult for them to apply concepts comprehensively (Puchongprawet & Chantraukrit, 2022). Hence, the authors recommended that students develop a plan and interdisciplinary understanding before engaging in activities related to the subject matter (Amalina & Vidákovich, 2022). These concerns were reflected in the survey responses of the teachers. In summary, the identified challenges in previous studies include student perceptions, data collection tools, the scarcity of research on STEM problem-solving, the lack of consensus among interdisciplinary techniques, the generalizability of observations across different contexts, as well as other related issues. Researchers have attempted to address these challenges through various strategies in their work.

Unlike similar studies that merely enumerate the challenges without providing detailed quantification, this research has taken a more comprehensive approach. These difficulties have been quantified using specific figures and organized in a descending order of significance. This approach enables interested researchers to obtain a more comprehensive understanding of the landscape. By summarizing these factors, researchers can gain a holistic view of the key elements influencing the development of problem-solving abilities in STEM education. These factors have been the focus of previous and ongoing studies, which have striven to address and overcome these challenges.

3.5. Emerging Research Directions for Advancing the Development of Problem-Solving Capabilities among Students in STEM Education

Table 3 provides clear evidence that research in STEM education can encompass diverse topics and grade levels to cultivate students' problem-solving abilities (Kadkhoda et al., 2023). To promote awareness of STEM fields and offer career guidance, gamifying STEM materials has emerged as a recommended approach, particularly in secondary education (Asigigan & Samur, 2021). The imperative of effective problem-solving-focused STEM instruction necessitates its early integration into the educational journey, enabling seamless skill transfer across disciplines and facilitating future career prospects (English, 2023). Consequently, the initial research direction centres on the integration of STEM education activities across all educational levels, spanning from early childhood to higher education, with the aim of nurturing students' 21st-century competencies, specifically their problem-solving skills (Akcay Malcok & Ceylan, 2020; Kurt & Benzer, 2020).

Concerning the learning strategies and methodologies employed in STEM education, several scholars anticipate substantial transformations that will forge new pathways for learners of all ages. It is imperative to augment the application of information technology and communication within authentic learning experiences to foster students' creativity and innovation (Iwuanyanwu, 2020). Future research endeavours ought to concentrate on effective instructional approaches that empower students to attain proficiency in STEM learning and  seamlessly integrate it into personalized learning strategies (Chen et al., 2019). Additionally, there is a need to investigate the impact of STEM-based education on other cognitive skills in K-12 settings (Tubagus et al., 2023). This has triggered the emergence of various research trends, including the enhancement of teachers' pedagogical expertise (Tubagus et al., 2023).

In the contemporary era of globalization, STEM-based learning has emerged as an appropriate educational model owing to its capacity to tackle interdisciplinary scientific challenges and provide teachers with convenient means to assess students' problem-solving abilities (Amalina & Vidákovich, 2022; Astuti et al., 2021). Moreover, researchers contend that for STEM education to progress further, teachers should concentrate on developing diverse teaching models and modules that are rooted in interdisciplinary subjects, enabling students to engage in higher- order thinking through practical activities (Sarican & Akgunduz, 2018; Usman et al., 2023).

Additionally, the authors propose novel research directions that involve adjusting experimental frameworks, augmenting sample sizes to yield more generalized conclusions, and presenting specific models for STEM learning (Amalina & Vidákovich, 2022; Purwaningsih et al., 2020). Furthermore, investigation is warranted to explore the impact of STEM education on various variables, including different skills, creative thinking, collaborative capacity, and learning motivation (Asigigan & Samur, 2021; Hebebci & Usta, 2022). Similar research trajectories have been identified in other articles pertaining to educational topics (Nguyen & Thai, 2023; Phan et al., 2022), underscoring the need for further exploration within the current landscape of technological advancements.

Moreover, to implement STEM education effectively, it is imperative to enhance collaboration among universities in the realm of teacher training (Asigigan & Samur, 2021; Conradty & Bogner, 2020). Given the current paramount importance of STEM education at all educational levels, it is crucial to conduct additional research that focuses on integrating STEM-related lessons into teacher training programmes offered at universities (Akcay Malcok & Ceylan, 2020; Kaleci & Korkmaz, 2018; Tubagus et al., 2023).

In summary, this article examines research directions aimed at fostering problem-solving skills in students within the context of STEM education. The findings align with the ongoing digital transformation in education (Reffiane et al., 2021; Trinh Thi Phuong et al., 2022). A multitude of studies have demonstrated that effective digital transformation can foster robust learning models, thereby enhancing the  technological capabilities of teachers in their instructional practices (Alatas & Yakin, 2021; Amalina & Vidákovich, 2022).

Moreover, an imbalance has been observed in the research methodologies employed in the published works, with a notable inclination towards qualitative studies and experiments rather than quantitative research. This observation suggests that future researchers could enhance the existing body of knowledge by validating previous studies through the utilization of quantitative methods or a combination of both quantitative and qualitative approaches. Elucidating this asymmetry serves to justify the limitations highlighted in Table 2. Time constraints and the availability of appropriate data collection tools have been acknowledged as hindrances in earlier investigations. This recognition indicates that forthcoming researchers may persist in refining their experimental design and manipulating other variables to augment the generalizability of research on the contemporary impact of STEM education.

Furthermore, Table 2 presents information that is easily verifiable and amenable to further investigation, offering readers an opportunity to delve deeper into the subject matter. For instance, the recurrently emphasized criterion of the generalizability of research inquiries to diverse educational programmes and geographical contexts is illustrative of this aspect.

This study has effectively classified future research directions into five distinct categories: Topic and grade level, encompassing the integration of STEM education across all grade levels to facilitate the development of student competencies; learning strategies and pedagogical methods, such as project-based approaches and constructivist methodologies, which are conducive to the cultivation of students' problem-solving skills; interdisciplinary teaching models, with a particular emphasis on problem-based learning frameworks; adjustment of experiments and other variables, thereby enhancing generalizability and providing supplementary validity testing; and teacher pedagogical competence, identified as a pivotal determinant of instructional success. This systematic classification empowers researchers to examine the advancement of students' problem-solving abilities within the realm of STEM education at a more abstract and generalizable level. Moreover, it enables the identification of teaching strategies and instructional models within a broader scope than what is currently explored in existing studies.

4. Conclusion

This article presents a comprehensive systematic review of the development of problem-solving skills among students in the field of STEM education. To conduct this research, the widely recognized PRISMA model was employed. A total of four data sources were utilized to retrieve relevant studies, and through a rigorous exclusionary process, 29 articles were identified and included in the study. The findings of the review reveal a consistent and notable increase in research efforts focusing on the enhancement of problem-solving skills among students in STEM education in recent years. Among the 11 countries covered by the extensive database, Indonesia emerged as the country with the highest number of publications on this research topic, followed closely by Turkey and  Thailand. Analyzing the keywords prevalent in these studies, as depicted in the word cloud, terms such as STEM, education, problem, solving, learning, skills, and problem-solving were the most frequently occurring. This reaffirms the fundamental significance of these concepts within the context of this article.

Regarding the research methodologies employed, qualitative and experimental approaches emerged as the predominant methods utilized in the reviewed studies. However, this observation presents an opportunity for future research endeavours to enrich the existing knowledge base further by incorporating quantitative methods or adopting mixed-method approaches. Furthermore, the assessment of the limitations and challenges associated with the development of problem-solving skills in STEM education led to the identification of several influential factors. These include the design and effectiveness of teaching programmes, the depth of students' subject knowledge, the implementation of interdisciplinary techniques, and the consideration of experimental variables.

Recognizing and addressing these factors will be crucial in ensuring the effectiveness of initiatives aimed at fostering problem-solving skills among students in STEM education. Overall, this systematic review provides valuable insights into the current state of research on the development of problem-solving skills in STEM education, highlighting the need for further exploration and the consideration of various influential factors.

This systematic review suggests several recommendations to improve problem- solving skills in STEM education. It recommends designing effective teaching programmes, incorporating inquiry-based learning, and exploring the impact of students' subject knowledge's on problem-solving abilities. Interdisciplinary approaches, quantitative methods, and mixed-method approaches should be emphasized. Challenges such as experimental variables should be addressed through robust study designs and rigorous evaluation frameworks.

5. References

Alatas, F., & Yakin, N. A. (2021). The effect of science, technology, engineering, and mathematics (STEM) learning on students’ problem-solving skill. JIPF (Jurnal Ilmu Pendidikan Fisika), 6(1). https://doi.org/10.26737/jipf.v6i1.1829

Akcay Malcok, B., & Ceylan, R. (2020). Does STEM education have an impact on problem- solving skill? Kesit Akademi, 25(25). https://doi.org/10.29228/kesit.46371

Amalina, I. K., & Vidákovich, T. (2022). An integrated STEM-based mathematical problem- solving test: Developing and reporting psychometric evidence. Journal on Mathematics Education, 13(4). https://doi.org/10.22342/jme.v13i4.pp587-604

Asigigan, S. I., & Samur, Y. (2021). The effect of gamified stem practices on students’ intrinsic motivation, critical thinking disposition levels, and perception of problem-solving skills. International Journal of Education in Mathematics, Science and Technology, 9(2). https://doi.org/10.46328/IJEMST.1157

Astuti, N. H., Rusilowati, A., & Subali, B. (2021). STEM-based learning analysis to improve students’ problem-solving abilities in science subject: A literature review. Journal of Innovative Science Education, 9(3). https://doi.org/10.15294/jise.v9i2.38505

Brika, S. K. M., Algamdi, A., Chergui, K., Musa, A. A., & Zouaghi, R. (2021). Quality of higher education: A bibliometric review study. Frontiers in Education, 6. https://doi.org/10.3389/feduc.2021.666087 

Chen, L., Yoshimatsu, N., Goda, Y., Okubo, F., Taniguchi, Y., Oi, M., Konomi, S., Shimada,

A., Ogata, H., & Yamada, M. (2019). Direction of collaborative problem solving- based STEM learning by learning analytics approach. Research and Practice in Technology Enhanced Learning, 14(1). https://doi.org/10.1186/s41039-019-0119-y

Conradty, C., & Bogner, F. X. (2020). STEAM teaching professional development works: Effects on students’ creativity and motivation. Smart Learning Environments, 7(1). https://doi.org/10.1186/s40561-020-00132-9

Delahunty, T., Prendergast, M., & Ní Ríordáin, M. (2021). Teachers’ perspectives on achieving an integrated curricular model of primary STEM education in Ireland:

Authentic or utopian ideology? Frontiers in Education, 6. https://doi.org/10.3389/feduc.2021.666608

English, L. D. (2023). Ways of thinking in STEM-based problem solving. ZDM - Mathematics Education, 55(7). https://doi.org/10.1007/s11858-023-01474-7

Ha, C. T., Thao, T. T. P., Trung, N. T., Huong, L. T. T., Dinh, N. Van, & Trung, T. (2020). A bibliometric review of research on STEM education in ASEAN: Science mapping the literature in Scopus database, 2000 to 2019. Eurasia Journal of Mathematics, Science and Technology Education, 16(10), em1889. https://doi.org/10.29333/ejmste/8500

Hebebci, M. T., & Usta, E. (2022). The effects of integrated STEM education practices on problem-solving skills, scientific creativity, and critical thinking dispositions. Participatory Educational Research, 9(6). https://doi.org/10.17275/per.22.143.9.6

Isatunada, A., & Haryani, S. (2021). Development of science learning tools using the STEM approach to train problem-solving ability and students' activeness in global warming material. Jurnal Pendidikan Sains Indonesia, 9(3). https://doi.org/10.24815/jpsi.v9i3.19599

Iwuanyanwu, P. N. (2020). Nature of problem-solving skills for 21st century STEM learners: What teachers need to know. Journal of STEM Teacher Education, 54(1) https://doi.org/10.30707/jste55.1/mmdz8325

Kadkhoda, A.M., Ahmadi, F., Hamidi, F., & Larki, F.N. (2023). Effects of the STEM approach on students’ problem-solving skill in science education. Quarterly of Iranian Distance Education Journal, 5(1), 144–154Please remove these spaces and align the margin

Kaleci, D., & Korkmaz, Ö. (2018). STEM education research: Content analysis. Universal Journal of Educational Research, 6(11). https://doi.org/10.13189/ujer.2018.061102

Khamngoen, S., & Srikoon, S. (2021). Research synthesis of STEM education approach effected on students’ problem-solving skills in Thailand. Journal of Physics: Conference Series, 1835(1). https://doi.org/10.1088/1742-6596/1835/1/012086

Kurt, M., & Benzer, S. (2020). An investigation on the effect of STEM practices on sixth grade students’ academic achievement, problem-solving skills, and attitudes towards STEM. Journal of Science Learning, 3(2). https://doi.org/10.17509/jsl.v3i2.21419

Lertcharoenrit, T. (2020). Enhancing collaborative problem-solving competencies by using STEM-based learning through the dietary plan lessons. Journal of Education and Learning, 9(4). https://doi.org/10.5539/jel.v9n4p102

Martaningsih, S. T., Maryani, I., Prasetya, D. S., Prwanti, S., Sayekti, I. C., Aziz, N. A. A., &

Siwayanan, P. (2022). STEM problem-based learning module: A solution to overcome elementary students’ poor problem-solving skills. Pegem Egitim ve Ogretim Dergisi, 12(4). https://doi.org/10.47750/pegegog.12.04.35

Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., Altman, D., Antes, G., Atkins, D.,

Barbour, V., Barrowman, N., Berlin, J. A., Clark, J., Clarke, M., Cook, D., D’Amico,

R., Deeks, J. J., Devereaux, P. J., Dickersin, K., Egger, M., Ernst, E., … Tugwell, P.  (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Medicine, 6(7). https://doi.org/10.1371/journal.pmed.1000097

Netwong, T. (2018). Development of problem-solving skills by integration learning following STEM education for higher education. International Journal of Information and Education Technology, 8(9), 639–643. https://doi.org/10.18178/ijiet.2018.8.9.1114

Nguyen, G. T. C., & Thai, D. T. (2023). Integrated teaching in primary schools: A systematicreview of current practices, barriers, and future developments. International Journalof Evaluation and Research in Education, 12(4).https://doi.org/10.11591/ijere.v12i4.26087

Nguyen, G. T. C., Thai, D. T., Phan, T. A., & Nguyen, H. T. (2023). The perceptions of elementary school children toward problem-solving abilities. FWU Journal of Social Sciences, 17(2). https://doi.org/10.51709/19951272/Summer2023/9

Nuangchalerm, P., Prachagool, V., Prommaboon, T., Juhji, J., Imroatun, I., & Khaeroni, K. (2020). Views of primary Thai teachers toward STREAM education. International Journal of Evaluation and Research in Education, 9(4). https://doi.org/10.11591/ijere.v9i4.20595

Nurmaliah, C., Azmi, T. N., Safrida, Khairil, & Artika, W. (2021). The impact of implementation of STEM integrating project-based learning on students’ problem- solving abilities. Journal of Physics: Conference Series, 1882(1). https://doi.org/10.1088/1742-6596/1882/1/012162

Phan, T. T., Duong, H. T., Do, T. T., Trinh, T. P. T., Trinh, T. H., Do, B. C., Tran, T., &

Nguyen, T. T. (2022). A bibliometric review on realistic mathematics education in Scopus database between 1972-2019. European Journal of Educational Research, 11(2). https://doi.org/10.12973/eu-jer.11.2.1133

Puchongprawet, J., & Chantraukrit, P. (2022). Creative problem-solving and creativity product in STEM education. International Journal of Instruction, 7(2), 135–142. https://doi.org/10.29333/aje.2022.7211a

Purwaningsih, E., Sari, S. P., Sari, A. M., & Suryadi, A. (2020). The effect of stem-pjbl and discovery learning on improving students’ problem-solving skills of the impulse and momentum topic. Jurnal Pendidikan IPA Indonesia, 9(4). https://doi.org/10.15294/jpii.v9i4.26432

Rädiker, S., & Kuckartz, U. (2019). Analyse qualitativer daten mit MAXQDA. In Analyse qualitativer daten mit MAXQDA. https://doi.org/10.1007/978-3-658-22095-2

Rasyid, A., Rinto, R., & Susanti, M. (2023). Project-based learning through the STEM approach in elementary schools: How to improve problem-solving ability. Journal of Education For Sustainable Innovation, 1(1). https://doi.org/10.56916/jesi.v1i1.477

Reffiane, F., Sudarmin, S.,Wiyanto, W., & Saptono, S. (2021). Developing an instrument to assess students’ problem-solving ability on hybrid learning model using ethno-

STEM approach through Quest program. Pegem Egitim ve Ogretim Dergisi, 11(4). https://doi.org/10.47750/pegegog.11.04.01

Rochman, C., Nasudin, D., & Rokayah, R. (2019). Science literacy on science technology engineering and math (STEM) learning in elementary schools. Journal of Physics: Conference Series, 1318(1). https://doi.org/10.1088/1742-6596/1318/1/012050

Sarican, G., & Akgunduz, D. (2018). The impact of integrated STEM education on academic achievement, reflective thinking skills towards problem solving and permanence in learning in science education. Cypriot Journal of Educational Sciences, 13(1). https://doi.org/10.18844/cjes.v13i1.3352

Suratno, Wahono, B., Chang, C.-Y., Retnowati, A., & Yushardi. (2020). Exploring a direct relationship between students’ problem-solving abilities and academic  achievement: A STEM education at a coffee plantation area. Journal of Turkish Science Education, 17(2). https://doi.org/10.36681/tused.2020.22

Tan, A. L., Ong, Y. S., Ng, Y. S., & Tan, J. H. J. (2023). STEM problem-solving: Inquiry, concepts, and reasoning. Science and Education, 32(2). https://doi.org/10.1007/s11191-021-00310-2

Topsakal, I., Yalçin, S. A., & Çakir, Z. (2022). The effect of problem-based STEM education on the students’ critical thinking tendencies and their perceptions for problem- solving skills. Science Education International, 33(2). https://doi.org/10.33828/sei.v33.i2.1

Tran, T. G., & Nguyen, T. N. (2020). Develop the collaboration problem-solving competence of students through STEM education in teaching of natural sciences in Vietnam secondary schools. European Journal of Education and Applied Psychology, 2, 8–17. https://doi.org/10.29013/EJEAP-20-2-8-17

Trinh Thi Phuong, T., Nguyen Danh, N., Tuyet Thi Le, T., Nguyen Phuong, T., Nguyen Thi Thanh, T., & Le Minh, C. (2022). Research on the application of ICT in Mathematics education: Bibliometric analysis of scientific bibliography from the Scopus database. Cogent Education, 9(1). https://doi.org/10.1080/2331186X.2022.2084956

Tubagus, M., Suhaimi, Ichsan, Rahman, A., & Santosa, T. A. (2023). Effect size of STEM- based problem-based learning model on problem-solving ability in students. Jurnal Edumaspul, 7(1).

United Nations Educational, Scientific and Cultural Organization (UNESCO). (2011). ICT competency framework for teachers. Journal of Chemical Information and Modeling. https://doi.org/10.1017/CBO9781107415324.004

Usman, G. B. T., Ali, M. N., & Ahmad, M. Z. (2023). Effectiveness of STEM problem-based learning on the achievement of biology among secondary school students in

Nigeria. Journal of Turkish Science Education, 20(3). https://doi.org/10.36681/tused.2023.026

Zengin, R., Kavak, T., Keçeci, G., & Zengin, F. K. (2022). The impact of STEM applications on problem-solving skills of 4th-grade students. Journal of Science Learning, 5(3). https://doi.org/10.17509/jsl.v5i3.48182