The Element Needed In Constructing Engineering Formative Assessment Based On Well-Defined Problems (Dp) For Accreditation: A Systematic Review

The Element Needed In Constructing Engineering Formative Assessment Based On Well-Defined Problems (Dp) For Accreditation: A Systematic Review

S. M. Aripen ^1,2, N.S. Abdullah¹*

Faculty Of Technical And Vocational Education, Universiti Tun Hussein Onn Malaysia, 86400 Parit Raja, Batu Pahat, Johor, Malaysia

Centre For Diploma Studies, Universiti Tun Hussein Onn Malaysia, Pagoh Higher Education Hub, 84600 Pagoh, Johor, Malaysia

ABSTRACT

Accreditation plays a critical role in engineering technology programme because it ensures that all graduates can pursue professional engineering qualifications and that the career choice for graduates is vast. Well-defined problems (DP) are one of the attributes in the Dublin Accord involved in the implementation of the engineering formative assessment to ensure a smooth process of accreditation. At the diploma level, Dublin Accord attributes are crucial for engineering technology programs. In Malaysia, programme outcome and Dublin Accord attributes must align with the course’s learning outcome per ETAC standards requirement. Embedding these attributes into the curriculum, especially for constructing the engineering formative assessment, can be complex and would depend on the educator’s involvement. However, constructing the formative assessment based on Dublin Accord attributes can be challenging as no specific guidelines are provided to the educator. Hence, studying the elements needed in constructing engineering formative assessments based on well-defined problems (DP) attributes is crucial, especially for accreditation purposes.

Keywords :- well-defined problems (DP), engineering, formative assessment, Dublin Accord, element

INTRODUCTION

• The Engineering Technology Accreditation Council (ETAC) is a delegated body by the Board of Engineers Malaysia (BEM). In 2015, ETAC was recognized as the only accrediting body for engineering technology bachelor’s degree, engineering diploma, and engineering technology diploma programmes offered in Malaysia [22]. The primary purpose of this council is to ensure that all engineering technology programs at the Institution Higher of Learning (IHL) in Malaysia follow the standard and quality of BEM requirements [1].
• In Malaysia, an accredited engineering technology bachelor’s degree is substantially equivalent to the engineering degrees of the signatories of the Sydney Accord (SA), meanwhile engineering diploma and engineering technology diploma programmes are substantially equal to the engineering degrees of the signatories of the Dublin Accord (DA) [22]. Thus, it is a requirement for the educator in IHL, especially for the engineering/engineering technology programme at the diploma level, to gain enough knowledge about attributes in Dublin Accord to enhance their skills in designing engineering formative assessments that align with programme outcomes for each course because the implementation of DA attribute in the course is also being assessed for accreditation purposes by ETAC [22]. In the Dublin Accord, there are three different attributes of well-defined problems (DP), well-defined engineering activities (NA), and knowledge profile (DK) [23].
• One of the critical challenges in engineering education is the design of well-defined problems (DP) that serve as effective assessment tools. Well-defined problems (DP) are characterized by their clear structure and expected outcomes, making them ideal for evaluating specific competencies [26]. However, creating such problems that are universally applicable and align with international standards like those of the Dublin Accord is a complex task. The variability in educational contexts and the diverse nature of engineering disciplines necessitate a flexible yet robust approach to problem design [23].
• Formative assessment is closely related to constructive alignment, as it can provide feedback to students on their progress toward achieving the intended learning outcomes and help them identify areas where they need to improve [2]. The goal of constructive alignment is to ensure that the assessment tasks are aligned with the intended learning outcomes and the teaching and learning activities so that students can demonstrate their achievement of the learning outcomes through the assessment tasks [3].
• Therefore, for the educator to design the intended engineering formative assessment that covers the well-defined problems (DP) in the Programme Outcome based on the constructive alignment setup, they need to fully understand the knowledge of well-defined problems (DP) in ETAC. However, educators face challenges in constructing the engineering formative assessment based on the well-defined problems (DP) due to a lack of guidance on embedding these sub-attributes based on the standards requirement by ETAC. Furthermore, only the definition of each sub-attribute and the mapping of programme outcome and well-defined problems (DP) are provided in the ETAC manual [25]. Thus, this paper provides a systematic review to identify elements needed for the well-defined problem (DP) to align with the Dublin Accord in constructing engineering formative for the engineering technology program at the diploma level in Malaysia.

Accreditation

Accreditation refers to evaluating and approving a higher education institution’s quality [4]. Five typical elements of the accreditation process are the creation and use of standards for accreditation, self-evaluations by colleges and universities, assessments of institutions by peers, evaluations by an accreditation body using the developed standards, and awarding recognized status [4][5]. According to Eaton [4], accreditation is frequently a quality standard. It guarantees that the program or organization satisfies the requirements of all parties involved, such as employers, students, and regulatory agencies. An institution, organization, or program’s external recognition is crucial in ensuring that it fulfills quality, competency, and credibility standards [6].

Based on Harvey [6], accreditation is necessary for both program and institutional accreditation. Instead of helping to build a high-income country with a sustainable and equitable economy, graduates from educational institutions that offer subpar programs would be a burden [7]. Accreditation in engineering education allows graduates of a specific engineering program to pursue professional engineering qualifications.  Moreover, accreditation has become a small requirement for universities that teach engineering since, in the absence of it, graduates’ career options will be severely limited [8].

Hence, the International Engineering Alliance (IEA) is a global not-for-profit organization formed to ensure countries cooperate in terms of educational quality and enhanced global mobility within the engineering profession [9][23]. Nine countries signed a signatory agreement under IEA, including Malaysia [23]. The three main accords forming this international alliance are the Washington Accord, the Sydney Accord, and the Dublin Accord.  The Washington Accord is an agreement between accreditation bodies of engineering degree programmes.  The Sydney Accord is an agreement between the accreditation bodies of engineering technology programmes, and lastly, the Dublin Accord is an agreement between the accreditation bodies of engineering technicians [23].

The International Engineering Alliance has classified engineering activities as complex, broadly-defined, and well-defined to distinguish between engineer, technologist, and technician categories. The International Engineering Alliance has created a list of qualities that all three groups should possess after graduation [23]. A brief example of a typical range statement outlining the attributes of problem-solving is given in Table 1. It shows that while the three groups’ graduation qualities are comparable, they are at different levels.  These graduate-level skills are universal and transferable to all engineering specialties.

Table 1 Snapshot of part of the range statement of knowledge and attitude profile as defined by IEA [23]

Attribute	Complex Engineering Problem (WP)	Broadly Defined Engineering Problem (SP)		Well-defined Engineering Problem (DP)
Range of conflicting requirements	Involve wide-ranging and/or conflicting technical, non-technical issues and consideration of future requirements	Involve a variety of conflicting technical and non-technical issues and consideration of future requirement		Involve several technical and non-technical issues and consideration of future requirements
Depth of analysis required	Have no obvious solution and require abstract thinking, creativity and originality in analysis to formulate suitable models	Can be solved by application of well- proven analysis techniques and models		Can be solved in standardized way
Familiarity of Issues	Involve infrequently encountered issues or novel problems		Belong to families of familiar problems which are solved in well-accepted ways	Are frequently encountered and thus familiar to most practitioners in the practice area

Dublin Accord in Malaysia

Malaysia became the 9th country to join the signatory of the Dublin Accord in 2018 [23]. Dublin Accord is the recognition of educational qualifications for engineering technicians, and it was first established in 2002 [10]. The accreditation board for engineering programmes in Malaysia is BEM, whereas for a diploma programme in engineering/engineering technology, it has been accredited by Engineering Technology Accreditation Council, ETAC [25].

The Engineering Technology Accreditation Council (ETAC) is a delegated body by the Board of Engineers Malaysia. ETAC started as a Protem Council in 2011 to provide a smooth transition in the accreditation of Engineering Technology and Engineering Technician education programme [22]. ETAC is the only body that is accredited for engineering technology bachelor’s degree, engineering diploma, and engineering technology diploma programmes that have been offered in Malaysia. 21-person ETAC is comprised of seven groupings that have been established by BEM, which are BEM, learned bodies, industry/employer, Public Services Department (PSD), Malaysian Qualification Agency (MQA), Ministry, and public representatives [22].

From IEA [23], the Dublin Accord is the required educational base for engineering technicians established in an international agreement. To ensure that Malaysian engineer, technologist, and technician graduates meet an international standard by implementing Dublin Accord in the engineering technology programme. It is also for mutual recognition of engineering technology degrees and diplomas and their graduates across the member countries [22] by following the guidelines provided by the International Engineering Alliance custodian of the Dublin Accord [23]. The graduate attributes that have been measured in the Dublin Accord consist of three graduate attributes: well-defined problem solving (DP), well-defined engineering activities (NA), and knowledge profile (DK) [25].

Tables 2, 3, and 4 show the details of each graduate attribute involved in the Dublin Accord. Based on Table 2 for well-defined problems, there are seven attributes with the definition provided by IEA: depth of knowledge required, range of conflicting requirements, depth of analysis required, familiarity of issues, extent of applicable codes, extent of stakeholder involvement and level of conflicting requirements, and interdependence. Meanwhile, well-defined engineering activities in Table 3 have five attributes with the definition: range of resources, level of interaction, innovation, consequences to the society and environment, and familiarity. Meanwhile, there are no attributes for knowledge profiles, as they only have definitions for each profile [25].

Table 2 Well-Defined Problems (DP) [25]

	Attribute	Well-Defined Problems
1	Depth of Knowledge Required	DP1 Cannot be resolved without extensive practical knowledge as reflected in DK5 and DK6 supported by theoretical knowledge defined in DK3 and DK4
2	Range of conflicting requirements	DP2: Involve several issues, but with few of these exerting conflicting constraints
3	Depth of analysis required	DP: Can be solved in standardised ways
4	Familiarity of issues	DP4: Are frequently encountered and thus familiar to most practitioners in the practice area
5	Extent of applicable codes	DP5: Are encompasses by standards and/or documented codes of practice
6	Extent of stakeholder involvement and level of conflicting requirements	DP6: Involve a limited range of stakeholders with differing needs
7	Interdependence	DP7: Are discrete components of engineering systems

Table 3 Well-Defined Engineering Activities (NA) [25]

	Attribute	Well-Defined Engineering Activities (NA)
1	Range of resources	NA1: Involve a limited range of resources (and for this purpose resources includes people, money, equipment, materials, information and technologies)
2	Level of interaction	NA2: Require resolution of interactions between limited technical and engineering issues with little or no impact of wider issues
3	Innovation	NA3: Involve the use of existing materials techniques, or processes in modified or new ways
4	Consequences to the Society and environment	NA4: Have consequences that are locally important and not far-reaching
5	Familiarity	NA5: Require a knowledge of practical procedures and practices for widely applied operations and processes

Table 5 Knowledge Profile (DK) [25]

	Knowledge Profile (DK)
1	DK 1 :A descriptive, formula-based understanding of the natural sciences applicable in a sub-discipline
2	DK 2:Procedural mathematics, numerical analysis, statistics applicable in a sub-discipline
3	DK3: A coherent procedural formulation of engineering fundamentals required in an accepted sub-discipline
4	DK4: Engineering specialist knowledge that provides the body of knowledge for an accepted sub-discipline
5	DK5: Knowledge that supports engineering design based on the techniques and procedures of a practice area
6	DK6: Codified practical engineering knowledge in recognised practice.
7	DK7: Knowledge of issues and approaches in engineering technician practice; ethics, financial, cultural, environmental and sustainability impacts.

Formative Assessment

Constructive alignment is an outcomes-based approach to teaching and learning that focuses on defining learning outcomes prior to teaching and developing teaching and assessment methods to achieve those outcomes [11]. The aim is to improve the quality of teaching, learning, and assessment by aligning them with intended learning outcomes. Constructive alignment is an outcomes-based approach to teaching that has significantly impacted student success in higher-order learning. It provides a framework for adjusting teaching and assessment methods to achieve desired learning outcomes and assess the standards at which they have been achieved. Constructive alignment in curriculum design has been found to enhance the quality of teaching, learning, and assessment [12].

This approach ensures students understand their expectations and provides active learning and engagement opportunities. In other words, constructive alignment is a way to ensure that what students are expected to learn is aligned with what they are taught and how they are assessed [13]. The constructive alignment allows for the design and development of educational programs that meet the needs of learners in different contexts. It also encourages using cognitive skills required for lifelong learning [14].

Applying constructive alignment involves the use of learning activities and assessment methods that are aligned with the desired learning outcomes. Constructive alignment is a framework for outcome-based education that involves aligning curriculum components such as outcomes, performance indicators, assessment methods, and standards. It aims to ensure coherence and effectiveness in teaching and learning. It also emphasizes the alignment between three significant elements of every course: the learning outcomes, activities, and assessments. As mentioned previously, the main goal of constructive alignment is to ensure that students achieve the intended learning outcomes by aligning the learning activities and assessments with those outcomes [15].

To achieve constructive alignment, educators should clearly articulate the learning outcomes for their course and align their course activities to develop those outcomes. They should also assess students at a level consistent with the learning outcomes provided and the learning activities used. This means that assessments should measure student development and competency in the learning outcomes identified and to the same level indicated in those learning outcomes [15]. Assessment is required for every course at a university to assess student performance, particularly at the tertiary level. Educators must design appropriate assessments for student assessment so that they can achieve their required outcome-based education in the lesson plan. Formative and summative assessments are essential elements in higher education, providing insight into student learning outcomes and offering opportunities for feedback [16].

Standard course designs include various assessment types, such as written exams, obligatory tests, and more significant, longer-running assignments. These assignments are typically assessed after the final version has been handed in, with occasional feedback or intermediate assessments based on the student’s current work status [16]. Formative assessment is an essential aspect of effective learning and teaching practices. The aim is to improve cognitive activity and provide students timely feedback throughout the learning process. In pedagogy, formative assessment is considered an indispensable feature of effective learning and a crucial skill of pedagogical foresight, as it provides timely feedback from the educator at all stages of mastering the educational program [24].

Unlike formative assessment, summative assessment occurs at the end of a learning period, such as a trimester or semester. It determines what a student has learned and how well they have learned it. Summative assessments are often used to award grades or other evidence of achievement [24]. It is now generally agreed, as outlined by Wylie & Lyon [17], that stated formative assessment is not a test, assessment, or quiz administered at the end of a period of learning but rather an ongoing set of practices for gathering evidence of student learning to inform the next steps in the process to support teaching and learning. Although this process can draw on a wide range of informal and formal methods, instruments, and tools to gather evidence of student learning, it is only worth engaging in the process if there is still the possibility of it learning to influence [17].

It is generally too late by the end of a chapter or unit. They also emphasize that the formative assessment process involves students and teachers collecting and considering evidence of learning. In contrast, teachers do not do formative assessments with students but with students [17]. Bennett [18] also mentioned that a formative assessment is best viewed not as a test or process but as a thoughtful integration of processes and specifically developed methods or instruments. In engineering education, especially at the diploma level, educators need to design the assessment, especially formative assessment, based on Dublin Accord criteria [25]. This is because it is one of the requirements from the Engineering Technology Accreditation Council to fulfill these criteria in each course so that BEM can accredit the programme. This accreditation is crucial as each diploma for an engineering programme must have this accreditation as a requirement by Board of Engineers, BEM in Malaysia [25].

For the engineering programme to be accredited smoothly, implementation of Dublin Accord into the formative assessment must be followed by all educators for their course. The construction of formative assessments should also be based on the Dublin Accord, which aligns with the learning outcomes of the courses. This learning outcome also aligned with the programme outcome, PO of the engineering technology programme. Based on the manual by ETAC (2020), there are 12 PO that must be fulfilled by all engineering technology programme: knowledge, problem analysis, design and development of solutions, investigation, modern tools usage, the engineer and society, environment/sustainability, ethics, individual and teamwork, communication skills, finance and project management, and lastly, lifelong learning (ETAC, 2020).

Implementing this Dublin Accord into the formative assessment proves to be difficult because the understanding of the Dublin Accord attributes is complex, and it depends highly on the educator of the course, either directly or indirectly [9]. Rita (2020) further explained that even though most educators are experts in their fields, teaching and student evaluation have not typically been the focus of their training. Hence, constructing a formative assessment of the Dublin Accord can be reduced to educators’ beliefs about learning and teaching. Students generally focus on what will be assessed, while educators focus on content. The mode of the formative assessment, therefore, sends a strong signal to the student about what is essential to learn [9]. In addition, evaluating the DA attribute in the formative assessment made by educators cannot be assessed thoroughly as no guidelines direct the evaluation of DA attributes [9].

Furthermore, no official guideline is provided in the manual ETAC itself, as only a relationship between the PO and DA attribute has been mapped, and only a definition for each DA attribute [25]. There is also a lack of studies on constructing formative assessments based on DA attributes, especially in Malaysia. A study about students’ performance for the course of civil engineering design projects before and after the pandemic Covid 19 about PO 8 and 10 that related to the affective domain was conducted but missing the details of the implementation of the Dublin Accord (Ahmad, 2023). Similarly, a comparison study for students’ performance with two different subjects was conducted to evaluate student’s performance in PO2, P03, PO4, PO5, PO8, and PO10. However, this study also missed the discussion about Dublin Accord implementation to the course involved, although the programme was accredited by ETAC [20].

According to Radloff et al., a student-centered, learning-oriented approach that emphasizes the development of knowledge and abilities is the most successful way to incorporate attributes. However, he also pointed out that educators’ belief plays a significant role in teaching and assessing the attributes (Radloff). Hence, it is believed that constructing a formative assessment based on the DA attribute is the best way to implement it as it is strongly associated with the constructive alignment of the course in the engineering technology programme because it can give students feedback on how they are doing in reaching the desired learning outcomes and assist them in pinpointing areas in which they still need to improve [2].

Method

In this study, the researcher used a systematic literature review to identify elements needed for the well-defined problem, DP1 until DP7 of the Dublin Accord. These elements are needed due to the lack of guidelines for constructing the formative assessment in the ETAC manual. After giving an overview of the eleven papers that were systematically identified, searched, and selected for this study, a more detailed reflection on those that specifically focused on the elements needed in constructing formative assessment is provided. To ensure comprehensive searching was done, systematic searching strategies were run in this chapter, namely. The three sub-processes of systematic searching strategies were performed, namely identification, screening and eligibility.

Identification

Identification is a process of searching for synonyms, related terms, and variations of the main keywords for the study based on Table 6. It aims to provide more options for selected databases to search for related articles for the review. Suitable keywords are first determined, and the identification process relies on online thesaurus, keywords used by past studies, keywords suggested by Scopus, and keywords suggested by experts as shown in Table 6.

The searching process was run on selected leading and supporting databases based on the main and enriched keywords either by using advanced searching techniques – which is by using Boolean operator, phrase searching, truncation, wild card, and field code functions separately or by combining these searching techniques into the entire searching string based on Table 7. Manual searching techniques such as handpicking, snowballing, reference tracking, and requesting relevant articles from colleagues were also practiced. In the identification process, 108 potential articles/documents were found. All articles/documents proceeded to the second process, namely screening.

Table 6 Identification process

Section	Main keywords	Enriched keywords	Searching techniques used	Databases used	Number of potential articles found in the identification process
Topic 1   Objective 1: To identify elements needed for the well-defined problem (DP) of the Dublin Accord in constructing formative assessment.	Elements Well-defined problem Dublin Accord Constructing Formative assessment	Elements = criteria Well-defined problem = broadly defined problem, complex engineering problem Dublin Accord = Sydney Accord, Washington Accord, graduate’s attributes Constructing = construction, formation, development Formative assessment = project, assignment, project assignment	Manual searching (handpicking, snowballing, reference tracking) Search string development (refer to Table 2)	Main databases used – Scopus, ERIC, IEEE Xplore	108

Table 7 Full search string used in selected databases (Scopus)

Topic

Scopus

Topic 1   Objective 1: To identify elements needed for the well-defined problem (DP) of the Dublin Accord in constructing formative assessment.

1.        TITLE-ABS-KEY ((“element” OR “criteria”) AND (“well-defined problem” OR “broadly defined problem” OR “complex engineering problem”) AND (“graduate attributes” OR “Dublin Accord” OR “Sydney Accord” OR “Washington Accord”) AND (“construction” OR “formation” OR “development”) AND (“formative assessment” OR “assignment” OR “project assignment”)) 2.        TITLE-ABS-KEY ((“graduate attributes” OR “Dublin Accord” OR “Sydney Accord” OR “Washington Accord”) AND (“construction” OR “formation” OR “development”) AND (“formative assessment” OR “assignment” OR “project assignment”)

Screening

In this process, all 108 articles were screened based on the criteria for article/document selection, which is done automatically based on the sorting function available in the selected databases. The same criteria were used across the selected databases, and whenever the sorting functions were unavailable, the articles were excluded manually.

As it is almost impossible for the researchers to review all the existing published articles, Okoli (2015) suggested that researchers determine the range of periods they can review [27]. Based on the search process on the selected database, it was realized that several studies related to complex engineering problems have emerged since 2014 until the present. However, some elements related to formative assessment had been found before 2014.

Therefore, based on this, the timeline between 2004 and 2024 was selected as one of the inclusion criteria. Furthermore, to enhance the data required for the study, all articles with empirical data and literature reviews published in a journal, conference proceedings, or chapter in a book are included. Moreover, only articles published in English are incorporated in the review to avoid confusion in understanding (Table 8). This process excluded 42 articles as they did not fit the inclusion criteria, and the remaining 66 articles were used for the eligibility process.

Table 8 Inclusion and exclusion criteria

Criteria	Inclusion	Exclusion
Timeline	2004 to 2024	Before 2004
Publication type	Article journal, conference proceeding, chapters in book	None
Language	English	Non-English

Eligibility

Eligibility is the third process, where the authors manually monitor the retrieved articles to ensure all the remaining articles (after the screening process) align with the criteria. This process was done by reading the title and abstract of the articles. If there is still no clear understanding of the relevance of the selected articles to the study, the article’s content was examined. This process excluded 55 articles that has duplicated records between databases, and the objective of the selected articles was not to focus on students.

There were 11 selected articles/documents for chapter two of this dissertation. This number of articles was selected after identification, screening and eligibility. Based on Krauss (2020), the study maturity depends on number of articles been published is higher to indicate the topics was investigated more [28]. Hence, the number of articles is sufficient to extract out the information needed about the element to construct on engineering formative assessment. The comprehensive searching process based on the systematic searching strategy can be viewed in Figure 1.

Data extraction and synthesis

After completing the identification, screening, and eligibility steps, the data extracted from the 11 primary studies listed in Table 9 will be synthesized and analysed to help answer the research questions/objective, which are to identify elements needed for the well-defined problem (DP) of the Dublin Accord in constructing formative assessment. In this process, the researcher focused the reading on the section on findings and discussion, and if needed, the abstract and conclusion have also been referred to.  Figure 1 presents a flowchart that outlines the process of identifying, screening, and evaluating articles for a research study. The process starts with articles retrieved from three different sources: Scopus (19 articles), IEEE (45 articles), and ERIC (44 articles). The screening stage involves 108 articles that are ready for the screening process. From this pool, 66 articles are excluded due to being duplicated, or in the form of non-articles. The remaining 66 articles then go through an eligibility process, where 11 articles are excluded because they are not in line with the study’s objectives. The final step is the qualitative synthesis, where 11 articles are deemed ready for this stage. The image provides a clear visual representation of the various stages of the article selection process, making it easy to understand the flow and the number of articles at each step. The summarization for all the steps involved in finding the literature review been shows in Figure 1.

Fig 1 Flow chart of the systematic searching strategies

RESULT

The aim of this study is to identify elements needed for the well-defined problem (DP) of the Dublin Accord in constructing a formative assessment. A systematic literature review (SLR) identified seven key elements necessary for constructing well-defined problems in formative assessments: fundamental knowledge, application of knowledge, real situation problems/case studies, problem-solving skills, critical thinking skills, hands-on learning skills, and teamwork skills. This essay analyses these elements, examining how the findings from various studies interrelate and contribute to a comprehensive understanding of engineering education. Table 9 shows the results of a systematic literature review for elements needed for a well-defined problem (DP) in the formative assessment of the Dublin Accord.

Table 9 Elements Needed for Well-Defined Problems in Formative Assessment

Element	Author(s) and Year	Findings
Fundamental Knowledge	Mat Isa et al., 2023	Emphasizes the need for fundamental theories in engineering education to enhance cognitive skills.
	Kamaruzaman et al., 2017	Highlights the importance of understanding fundamental knowledge for problem identification and solving.
	Ggibani, 2021	Discusses fundamental knowledge as a means to assess and encourage student learning.
	Ammar & Rais, 2023	Identifies the ability to recognize problems through fundamental knowledge as a key evaluation criterion.
Application of Knowledge	Mat Isa et al., 2021	Focuses on applying knowledge in research-based projects to demonstrate students’ abilities.
	Mat Isa et al, 2019	Emphasizes design projects that apply engineering knowledge practically, enhancing cognitive skills.
	Kamaruzaman et al., 2017	Discusses challenges students face in applying theoretical knowledge practically in real work environments.
Real Situation Problems/Case Studies	Mat Isa et al., 2021	Stresses integrating real-life scenarios to align educational projects with industry practices.
	Diogo, 2021	Highlights the motivational impact of real industry problems on students.
	Gqibani, 2021	Advocates for assessments that link real-life learning to classroom knowledge.
Problem-Solving Skills	Mourtos et al., 2004	Differentiates between problem-solving and exercise-solving, emphasizing the need for complex problem-solving skills.
	Mat Isa et al., 2023	Highlights problem-solving skills as essential for engaging students in practical projects.
	Massoud & Zubair, 2023	Supports project-based learning as a driver for developing problem-solving skills.
	Diogo, 2021	Discusses how projects incorporating problem-solving skills provide new insights into industry practices.
Critical Thinking Skills	Bronwyn, 2020	Uses feedback and structured processes to promote critical thinking in students.
	Mat Isa et al, 2021	Highlights the role of feedback in enhancing critical and creative thinking skills through problem-based learning.
	Gqibani, 2020	Encourages formative assessments that engage students and promote long-term retention of concepts.
Hands-On Learning Skills	Mat Isa et al., 2023	Engages students in practical exercises to enhance communication and understanding of engineering problems.
	Mat Isa et al, 2019	Identifies hands-on learning as crucial for understanding user needs and improving student performance.
	Massoud & Zubair, 2023	Supports hands-on learning for improving student expertise in design and synthesis.
Teamwork Skills	Mat Isa et al., 2021	Promotes communication and teamwork in collaborative educational projects.
	Bronwyn, 2020	Highlights teamwork in projects to stimulate higher-order thinking among students.
	Gqibani, 2021	Stresses the importance of training students to work effectively in teams for future employment.

Discussion

In this study, there are seven elements have been identified for the well-defined problem (DP) aligned with the Dublin Accord. They are fundamental knowledge, application of knowledge, real situation problems or case studies, problem-solving skills, critical thinking skills, hands-on learning skills, and teamwork skills. The details discussed have been discussed in the next section.

Fundamental Knowledge

Fundamental knowledge forms the basis of any educational endeavour, particularly in engineering, where a strong grasp of foundational concepts is crucial for problem-solving and innovation. Mat Isa (2023) emphasize the integration of fundamental theories into engineering curricula, arguing that this foundational knowledge is indispensable for developing cognitive skills. In contrast, Kamaruzaman (2017) focus on the practical implications of fundamental knowledge, highlighting its role in enabling students to identify and address real-world problems effectively. This dual emphasis on theoretical understanding and practical application underscores the multifaceted nature of fundamental knowledge.

Qgibani (2021) further expands on this by suggesting that fundamental knowledge serves as a catalyst for continuous learning and intellectual growth. This perspective aligns with Ammar and Rais’s (2023) assertion that evaluating students’ ability to recognize and articulate problems based on their fundamental knowledge is crucial for outcome-based assessments. Together, these studies highlight the essential role of fundamental knowledge in both academic and professional contexts, serving as the cornerstone of effective engineering education [29],[35],[34],[36].

Application of Knowledge

The application of knowledge is a critical element that bridges the gap between theoretical understanding and practical problem-solving. Mat Isa (2021) and Mat Isa (2019) advocate for projects requiring students to apply their engineering knowledge in collaborative environments. Mat Isa (2021) emphasize the cognitive benefits of comprehensive projects, while Mat Isa (2019) highlights the importance of addressing sustainability issues. This focus on real-world application is crucial for developing students’ higher-order thinking skills and preparing them for industry challenges.

Kamaruzaman (2017) provide a contrasting perspective by identifying the difficulties students face in applying theoretical knowledge practically. This gap underscores the need for educational strategies that emphasize practical application, ensuring that students are equipped to meet industry demands. Collectively, these studies underscore the importance of integrating practical application into formative assessments, preparing students for the complexities of professional engineering practice [30],[32],[35].

Real Situation Problems / Case Studies

 Real situation problems or case studies are integral to aligning academic learning with industry practices. Mat Isa (2021) and Diogo (2021) emphasize the motivational impact of engaging with real industry problems, highlighting the importance of experiential learning. Mat Isa (2021) focus on enhancing students’ understanding of work environments, while Ricardo emphasizes the motivational aspects of real-world engagement. This alignment with industry practices is crucial for preparing students for professional challenges [31],[37].

Gqibani (2021) adds another dimension by advocating for assessments that link real-life learning to classroom knowledge. This integration ensures that students’ educational experiences are relevant and aligned with industry standards [34]. Together, these studies highlight the importance of incorporating real situation problems into formative assessments, providing students with opportunities to engage with complex, real-world challenges.

Problem-Solving Skills

Problem-solving skills are a cornerstone of engineering education, enabling students to tackle complex challenges and develop effective solutions. Mourtos (2004) and Mat Isa (2023) both emphasize the importance of engaging students in complex problem-solving activities. Mourtos (2004) differentiate between problem-solving and exercise-solving, highlighting the depth and analytical thinking required for real-world problem-solving. Mat Isa (2023) focus on the importance of well-designed projects in enhancing students’ problem-solving abilities, emphasizing the role of practical engagement in skill development [29],[38].

Massoud and Zubair (2023) support project-based learning to drive problem-solving skills development, noting that it leads to satisfactory student performance. Diogo (2021) highlights the insights gained from projects incorporating problem-solving skills, underscoring the value of experiential learning. Collectively, these studies emphasize the importance of developing robust problem-solving skills through formative assessments, preparing students for the challenges of the professional world [37],[39].

Critical Thinking Skills

Critical thinking skills are essential for fostering analytical and evaluative capabilities among students. Bronwyn (2020) and Mat Isa (2021) both emphasize the role of feedback and structured processes in promoting critical thinking. Bronwyn highlights the importance of feedback in enhancing students’ critical evaluation of their performance, while Mat Isa (2021) focus on the role of feedback in refining proposed solutions. This feedback-driven approach encourages students to engage in meaningful dialogue, explore diverse solutions, and deepen their understanding of complex concepts [32],[33].

Gqibani (2020) advocates for formative assessments that engage students and promote long-term retention of concepts, leading to critical thinking and creative problem-solving. These studies collectively underscore the importance of fostering critical thinking skills through formative assessments, preparing students for the complexities of professional practice [34].

Hands-On Learning Skills

Hands-on learning skills engage students in practical activities, enhancing their understanding of engineering concepts and promoting active learning. Mat Isa (2019) and Massoud and Zubair (2023) both emphasize the importance of engaging students in hands-on activities to promote active learning. Mat Isa (2023) focus on the role of hands-on activities in enhancing students’ communication skills and understanding of engineering problems, while Massoud and Zubair (2021) highlight the importance of hands-on learning for improving expertise in design and synthesis.

This emphasis on experiential learning is crucial for preparing students for real-world challenges, ensuring that they are equipped with the practical skills necessary for professional success. Collectively, these studies highlight the significance of hands-on learning in engineering education, underscoring its role in developing practical skills and enhancing student engagement [29],[32],[39].

Teamwork Skills

Teamwork skills are essential for fostering communication and collaboration among students, preparing them for the collaborative nature of professional work. Mat Isa (2021) and Bronwyn (2020) both emphasize the importance of integrating teamwork skills into collaborative educational projects. Mat Isa (2021) focus on promoting communication and teamwork, highlighting the role of collaborative projects in enhancing student engagement. Bronwyn highlights the role of teamwork in stimulating higher-order thinking, emphasizing the cognitive benefits of collaborative learning [32],[33].

Gqibani (2021) stresses the importance of training students to work effectively in teams, ensuring that graduates are prepared for the collaborative nature of professional work. These studies collectively underscore the importance of teamwork skills in preparing students for diverse professional environments, highlighting the role of collaboration in engineering education [34].

The aim of this study is to find the element needed in constructing formative assessments that integrate well-defined problems, DP that promote constructive learning and students’ educational pursuits in engineering. This part of the study is concerned with matching the key components that emerged in the systematic literature review with the construction of a formative assessment specially designed to address the attributes of the Dublin Accord.

When integrate these elements into the DPs, the assessment focuses on how the formative assessments can incorporate fundamental knowledge, application of knowledge, real situation problems/case studies, developed problem-solving skills, skills on critical thinking, skills in acquiring hands-on experiences and skills in teamwork groups.

In doing so, there is a possibility for assessments to be constructed not only because of the purpose of checking students’ knowledge and abilities but also because of developing skills that are crucial to becoming an engineering professional.

This integration of the element based on the well-defined problems (DP) is to help make sure every aspect of the formative assessment fits logically into an overall evaluation plan. The purpose is to bring the educational processes to the level when the student can learn the subject in depth and apply the acquired knowledge and competencies in practice in the selected occupation successfully. By aligning with the well-defined problems (DP), engineering programmes can strengthen their accreditation procedures so that students graduating with engineering skills meet the demands of today’s engineering world.

In the well-defined problem (DP), seven sub-attributes need to be mapped with the programme outcome for every course in the engineering programme at the diploma level. According to the ETAC manual, each programme outcome must be mapped with at least three DPs, including the DP1.

The designation or the selection of the DP can be made by the educators depending on the suitability of the courses.

Hence, after the verification has been made, educators must construct the projects and assignments involved in the courses according to the chosen well-defined problems. Therefore, the elements provided in the study can be helpful to the educator for the development of the question in the formative assessments.

CONCLUSION

In conclusion, the systematic literature review highlights seven key elements essential for constructing well-defined problems (DP) in formative assessments aligned with the Dublin Accord. An analytical comparison of the findings reveals the interconnectedness of these elements and their collective significance in engineering education. By integrating these elements into formative assessments, educators can enhance the quality of engineering education and ensure that graduates are well-equipped to meet industry standards and expectations. Implementing the Dublin Accord is crucial for the accreditation process of engineering programs, ensuring that educational institutions produce graduates who are knowledgeable and capable of applying their knowledge in practical settings.

Failure to do so may affect the result of accreditation of the engineering technology programme, especially for the diploma level. Although essential, there is a lack of studies on this matter, especially in Malaysia. It was also observed that there are no guidelines for educators to follow when constructing the formative assessment based on the Dublin Accord attribute in the manual standard provided by the Engineering Technology Accreditation Council. Currently, most studies focus on evaluating students’ performance regarding program outcomes, but there is no discussion on Dublin Accord attributes. Programme outcome and Dublin Accord attributes must also be aligned with the course’s learning outcome. Hence, studying the students’ achievements regarding Dublin Accord attributes is vital.

Hence, it is suggested that more research be conducted on applying these Dublin Accord attributes, especially in constructing the formative assessment in the future. It is recommended that a generic framework for the guideline of using the Dublin Accord attributes as there is no official guideline or template provided that educators can follow based on standards from the Engineering Technology Accreditation Council

ACKNOWLEDGEMENT

This research was supported by Universiti Tun Hussein Onn Malaysia (UTHM) through Tier 1 (vot Q514).

REFERENCES

1. Abdul-Talib, S., Zaharim, A., Ariffin, J., Yang, Y. C., & Abdullah, S. (2007) Route To Becoming A Member Of Evaluation Panel For The Engineering Accreditation Council (EAC) Of Malaysia. International Journal of Education And Information Technology, 2(1), 49-51.
2. Njuguna, James (2020) Constructive alignment of intended learning outcomes, learning activities and assessments for an engineering master’s degree course module. https://doi.org/13140/RG.2.2.36031.12965
3. Akib, Erwin & Martuty, Arie & Ghaffar, Mohd & Ahmad, Jamilah (2018) The Correlation Between Assessment for Learning and Constructive Alignment Based on Age. International Research-Based Education Journal. 1(169). https://doi.org /17977/um043v1i2p169-174
4. Eaton, J. S (2009) Accreditation in the United States. New Dir. High. Educ. 145, 79–86, https://doi.org /1002/HE.337
5. Cueto, J., Burch, V. C., Adnan, N. A. M., Afolabi, B. B., Ismail, Z., Jafri, W., Olapade-Olaopa, E. O., Otieno-Nyunya, B., Supe, A., Togoo, A., Vargas, A. L., Wasserman, E., Morahan, P. S., Burdick, W., & Gary, N. (2006) Accreditation of undergraduate medical training programs: Practices in nine developing countries as compared with the United States. Education for Health: Change in Learning and Practice, 19(2), 207–222. https://doi.org/10.1080/13576280600783570
6. Harvey, L. (2007) The Power of Accreditation: views of academics. Journal of Higher Education Policy and Management, 26(2), 207-223. https://doi.org/10.1080/1360080042000218267
7. Endut, A. S., Majid, F. A., Ibrahim, A. B., & Ashari, H. (2013) Responsive Outcome Evaluation as an Internal Quality Assurance Mechanism Alternative at IHLs in Malaysia. Procedia – Social and Behavioral Sciences, 90, 13–21. https://doi.org/10.1016/j.sbspro.2013.07.060
8. Mohd Said, S., Chow, C. O., Mokhtar, N., Ramli, R., Tuan Ya, T. M. Y. S., & Mohd Sabri, M. F. (2013) Accreditation of engineering programs: An evaluation of current practices in Malaysia. International Journal of Technology and Design Education, 23(2), 313–328. https://doi.org/10.1007/s10798-011-9180-6
9. Rita Steenkamp & Lagouge Tartibu (2020) Practical approaches to implement graduate attributes in engineering faculties. 2020 IFEES World Engineering Education Forum – Global Engineering Deans Council (WEEF-GEDC). https://doi.org /10.1109/weef-gedc49885.2020.9293651
10. Koh, Y., Chua, Y., Sin, M. (2019) Engineering Programme Accreditation: A Comparison Study Between the Accreditation Criteria of Academic Curriculum in Malaysia and Singapore. Conference: 19th Annual SEAAIR Conference, 177-185.
11. H. (2023) Doctoral education process and product using constructive alignment in software engineering and computer science, Journal of Teaching and Learning in Higher Education, 4(2). https://doi.org /10.24834/jotl.4.2.853.
12. Lawrence, J., E. (2023) The Application of Constructive Alignment Theory in Designing a Curriculum Unit in Information Systems. The education review, 7(4), 471-482.
13. Hamdoun, W, M. (2023) Constructive Alignment Approach: Enhancing Learning and Teaching. British journal of multidisciplinary and advanced studies, 4(2), 162-170.
14. Romanowski, M., H., Hadeel Alkhateeb, Youmen Chaaban. (2023) Problematizing constructive alignment in higher education in gulf cooperation council countries. Teaching in Higher Education, 1-19.
15. Abejuela, H., J., M., Hannah T. Castillon, Marlie Joy G. Sigod. (2022) Constructive Alignment of Higher Education Curricula. Asia Pacific Journal of Social and Behavioral Sciences, 20.
16. Köppe, C., Middelkoop, R.. (2019) On Using Hybrid Pedagogy as Guideline for Improving Assessment Design. CEUR Workshop Proceedings, 2712. 29-40.
17. Wylie, C., E., & Lyon, C., J. (2020) Developing a Formative Assessment Protocol to Support Professional Growth. Educational Assessment, 25(4), 314-330. https://doi.org/10.1080/10627197.2020.1766956
18. Bennet, R., E. (2011) Formative assessment: A critical review. Assessment in Education: Principles, Policy and Practice, 18(1), 5-25. https://doi.org/10.1080/0969594X.2010.513678
19. Ahmad Idzwan Bin Yusuf, Nurul Izziyantie Binti Mat Noor, Nurul Farraheeda Binti Abdul Rahman, Mohd Firdaus Bin Mohd Akhbar (2023) Evaluation of Affective Domain for Civil Engineering Design Project Before and During COVID-19: A Case Study in UiTM Johor, Asian Journal of University Education (AJUE), 19(1), 147-155, https://doi.org/10.24191/ajue.v19i1.21233
20. Narita Binti Noh, Ahmad Idzwan Bin Yusof, Lee Siong Wee, Muhammad Bazli Faliq Bin Mohd Puaad, Nurul Izziyantie Binti Mat Noor (2023) Correlation of Student Attainment for Civil Engineering Design Project and Structural Concrete and Steel Design for Undergraduate Students in Concurrent Semester, International Journal of Academic Research in Progressive Education and Development, 12(1), 178-188, https://doi.org /10.6007/IJARPED/v12-i1/16124
21. Radloff, B. de la Harpe, H. Dalton, J. Thomas and A. Lawson (2008) Assessing graduate attributes: Engaging academic staff and their students, ATN Assessment.
22. Boards of Engineers’ Malaysia. (2024, March 21). BEM. Retrieve (2024, March 21), from http://bem.org.my/web/guest/engineering-technology-accreditation-council
23. International Engineering Alliance. (2024, March 21). IEA. Retrieve (2024, March 21), from https://www.ieagreements.org/accords/dublin/
24. National Foundation for Educational Research. (2024, March 21). NFER. Retrieve (2024, March 21), from https://www.nfer.ac.uk/assessments/an-introduction-to-formative-and-summative-assessment/
25. (2020). Engineering Technology Accreditation Standard 2020. Engineering Technology Accreditation Council, BEM.
26. Reynolds, S. (2016). Examining the influence of ill- and well-defined problems in a first-year engineering design course. Journal of Engineering Education, 105(2), 207-219.
27. Okoli, C., (2015). A guide to conducting a standalone systematic literature review. Communications of the Association for Information Systems, 37(1), 879-910.
28. Kraus, S., Breier, M., Dasí-Rodríguez, S., 2020. The art of crafting a systematic literature.
29. Mat Isa, C. M., Azmin, N. F. M., Ibrahim, S. N., Oh, C. L., (2023). Implementation of Complex Engineering Problem Solving (CEP) and Complex Engineering Activities (CEA) in Malaysian Engineering Curriculum: A Pilot Study. International Conference on University Teaching and Learning (InCULT), 1-6, doi: 10.1109/InCULT59088.2023.10482478
30. Mat Isa, C. M. Mustaffa, N.K. Preece, C. N. & Lee, W. K. (2019). Enhancing Conceive-Design-Implement-Operate and Design Thinking (CDIO-DT) Skills Through Problem-Based Learning Innovation Projects. IEEE 11th International Conference on Engineering Education (ICEED), 41-46. doi: 10.1109/ICEED47294.2019.8994935
31. Mat Isa, C. M., Mohammad, N. I. A., Saad, N. H., & Christopher N. P. (2021). Programme Outcome Attributes related to Complex Engineering Problem Capability: Perceptions of Engineering Students in Malaysia. Asian Journal of University Education (AJUE), 17(4), 95-105. https://doi.org/10.24191/ajue.v17i4.16220
32. Mat Isa, Che Maznah & Chai Lian, Oh & Liew, Chia & Saman, Hamidah & Che Ibrahim, Che Khairil Izam & Yusof, Zulkiflee. (2021). Effective Implementation of Complex Engineering Problems and Complex Engineering Activities in Malaysian Engineering Curricular. Asian Journal of University Education (AJUE), 17(4), 170-178. https://doi.org/17. 170. 10.24191/ajue.v17i4.16219
33. Bronwyn Swartz. (2020). ‘Assessment as Learning’ as a tool to prepare engineering students to manage ill-defined problems in industry. IFEES World Engineering Education Forum – Global Engineering Deans Council (WEEF-GEDC), 1-5, doi: 10.1109/WEEFEDC49885.2020.9293638
34. Gqibani, S. (2021). Re-think assessment methods for Engineering Technology Programmes: A case study in Mechanical Engineering Technology. World Engineering Education Forum/Global Engineering Deans Council (WEEF/GEDC), 260-269, doi: 10.1109/WEEF/GEDC53299.2021.9657340.
35. Kamaruzaman, F. M., Hamid, R., Mutalib, A. A., (2017). A review on issues and challenges in incorporating complex engineering problems in engineering curriculum and proposed solutions. 7th World Engineering Education Forum (WEEF), 697-701
36. Ammar, K., & Rais, R. N., (2023). Outcome Based Performance Evaluation Approach Using Rubric Based Capstone Projects Assessment. 20th International Multi-Conference on Systems, Signals & Devices (SSD), 777-783, doi: 10.1109/SSD58187.2023.10411275
37. Diogo, R. A., Venâncio, A. L. A. C., Santos, M. A. M. R. Loures, E. F. R. dos Santos, N. (2021). Real Engineering Problems in an Undergraduate Course: The learning methodologies and assessment tools. World Engineering Education Forum/Global Engineering Deans Council (WEEF/GEDC), 59-168, doi: 10.1109/WEEF/GEDC53299.2021.9657331.
38. Mourtos, N. J DeJong Okamoto, N. & Rhee, J. (2004). Defining, teaching, and assessing problem solving skills. 7th UICEE Annual Conference on Engineering Education, 1-5
39. Massoud, Y. & Zubair, M. (2023). An Interdisciplinary Project-Based Learning Approach for Engineering and CS+[X] Students through AI-Enabled Biomedical Imaging System. IEEE International Symposium on Circuits and Systems (ISCAS), 1-4, doi: 10.1109/ISCAS46773.2023.10181976.