E-portfolio system development for undergraduate clinical dentistry: An action research study

Introduction: Higher education institutions are responsible for accommodating students to synthesise the knowledge and skills they obtained in the classroom or workplace through reflections. In the field of health professional education, the use of e-portfolio has provided an empirically validated conceptual model of integrated knowledge and learning. Therefore, it is essential to analyse the need to establish a new platform to engage stakeholders and thus ensure daily use of e-portfolio by students in undergraduate dentistry program in Indonesia. The present study was aimed to develop an e-portfolio platform based on a needs analysis and explore stakeholders’ perceptions following its implementation. Methods: The present study was a participatory action research study using a cross-sectional design with a qualitative approach that involved trainees and supervisors in an undergraduate clinical dentistry program. The study was divided into three stages: the exploration stage, the pilot study stage, and the evaluation stage of the e-portfolio platform. Data were collected using in-depth interviews and focus group discussions with stakeholders, which included faculty members and students. Data were analysed thematically. Results: The e-portfolio platform was developed according to the results of the needs analysis stage while considering stakeholders’ expectations and expected features. The results of the evaluation stage indicated a positive response from respondents. Respondents considered the e-portfolio to be very useful in students’ professional development, as it helped students to understand the content, assisted in learning, and improved their understanding of self-reflection. Conclusion: Performing a needs analysis prior to the development of an e-portfolio system provided information on technical issues regarding its development as well as the adequate supporting system required prior to its implementation. Although the implementation of the e-portfolio developed in this early stage was not conducted ideally, students reported a satisfactory impact on their learning.


INTRODUCTION
Higher education institutions are responsible for producing flexible, adaptive, and creative graduates who can challenge the future. The curriculum in higher education must be designed to accommodate students with opportunities to conduct reflections as well as to synthesise and determine the knowledge and skills they obtained in the classroom or workplace. Many educators believe that integrated e-portfolio-based learning may facilitate these needs. 1 The use of e-portfolio provided an empirically validated conceptual model of integrated knowledge and learning, which was described in six models of student abilities, including the ability to identify and adapt knowledge in different contexts, adapt to differences to create solutions, regulate self-learning, be a reflective learner and possess excellent feedback-seeking behaviour, view problems from various perspectives, and develop a digital professional identity. 2 In the field of health professional education, portfolios have been used as an assessment method as well as a monitoring and learning method. 3 The use of e-portfolios in an undergraduate dentistry program was reported to have the predictive validity of multiple competency scores in a specific module. 4 The use of e-portfolio in postgraduate dentistry has also shown several advantages regarding students' learning, motivation, and communication skills. The motivation of postgraduate dentistry students increased during conducting reflections, as it helped students to identify their weaknesses and helped them to come up with possible solutions.⁵ However, the use of e-portfolio has not been reported in any undergraduate dentistry program in Indonesia to date. Currently, there are multiple service providers of web-based e-portfolio available for any educational program. These providers allow students to collect and organise evidence of their achievements in various types of media (e.g. audio, video, photos) and connect them with learning outcomes and other competency standards. However, the use of a one-fits-all platform had low face validity since it was unable to ensure the level of interactivity and reporting data management appropriate for each institution. 6 It is also known that several methods and models-such as the student model, feedback model, and visual model-can be adapted by an educational institution to develop e-portfolio. 7 The critical thing to consider is the involvement of students and faculty members as users in their development to ensure that the e-portfolio system will be used in an integrated manner with the curriculum to ensure the successful use of e-portfolio on a daily basis. 6, 8 Therefore, it remains essential to analyse the need to establish a new platform to understand the purpose of developing an e-portfolio system as an assessment tool, learning method, monitoring device, or for a combination of different purposes. 2 A needs analysis for this new initiative within the curriculum (i.e. portfolios) was also essential to identify possible steps toward communicating it to the involved stakeholders to ensure their involvement and sense of ownership in the development process. 9 In order to develop an e-portfolio that may accommodate stakeholders' needs, a needs analysis of the required features and structure of an e-portfolio should be conducted to ensure its user-friendliness. Furthermore, the mentoring and assessment system of the e-portfolio would also need to be formulated by considering various aspects related to learning in the undergraduate clinical dentistry program. The mentoring process was necessary for portfolio development because it helps students to conduct self-reflection more accurately and increases their engagement through formative or summative assessments. [10][11][12] The present study was aimed to develop an e-portfolio platform based on a needs analysis and to explore stakeholders' perceptions following its implementation.

Context
The present study was conducted in a public faculty of dentistry in Indonesia. The undergraduate dentistry program at the school implemented a competency-based curriculum and was divided into pre-clinical and clinical stages. During the clinical stage, the school implemented a competencybased curriculum with a requirements-based approach, requiring trainees to be exposed to a minimum number of cases and provide treatments relevant to the National Competency Standard (NCS) under supervision. Students provided integrated individual oral health care in integrated clinics as well as 6-week external clinical rotations for oral surgery and dental public health.

Design
The present study focused on the development of an e-portfolio system in clinical dentistry training based on a needs analysis study. This study involved a participatory action research study using a cross-sectional design with a qualitative approach that involved trainees, program administrators, and supervisors in both integrated clinics and external clinical rotations as prospective users of the e-portfolio. 13 The study was divided into three stages: (1) the needs analysis stage, (2) the development and pilot stage, and (3) the evaluation stage. The needs analysis stage was aimed at obtaining perspectives from users and program administrators regarding the needs, features, learning outcomes, and teaching learning methods expected from the e-portfolio. Following the exploration stage, the research team collaborated with a professional web developer to develop a web-based e-portfolio. The developed e-portfolio was then used in the setting for six weeks at the pilot study stage. At the end of the six-week pilot program, focus group discussions (FGDs) were conducted to explore trainees' perspectives regarding the roles and benefits of the e-portfolio (evaluation stage).

Respondents
Respondents of this study were clinical stage students that were selected using maximum variation sampling strategy representing variation of gender, cohorts, academic achievements, and clinical rotation as shown in Table 1. This sampling strategy was used to allow different perspectives. As many as 24 student respondents were selected, signed informed consent, and committed to participate in the study, started with the initial FGDs in needs analysis stage, tried the developed e-portfolio in the development and piloting stage, as well as participated in the final FGDs in the evaluation stage.
This study also involved faculty members. Two faculty members were interviewed in the needs analysis stage. Five faculty members who were clinical supervisors participated in the evaluation stage to provide further suggestions and recommendations for further development of the e-portfolio.

Data Collection
There were three stages of the data collection which described as follows: Needs analysis stage. Two in-depth interviews were conducted with faculty members of the clinical dentistry program in the dental school. An in-depth interview was aimed at exploring the current curriculum mapping, assessment methods used, and the expected outcomes to be taught or assessed using e-portfolio.
A total of twenty-four clinical dentistry students participated in four FGDs each aimed at exploring the trainees' perspectives regarding e-portfolio features that were considered useful and user-friendly for daily use by trainees. The data obtained from this stage was analysed thematically to obtain the essential features of the e-portfolio, as well as to define a possible mechanism for applying e-portfolio as a new method in an undergraduate clinical dentistry program.
Development and pilot stage. The results of the needs analysis stage were used to inform the development of the e-portfolio system. The developed e-portfolio system was then piloted for six weeks. Training sessions were conducted to introduce the platform to students as well as to introduce the basic concept of self-reflection prior to the pilot period.

Evaluation stage.
Another four FGDs-with the same respondents as those participated in the needs analysis stage-were conducted following the six-week pilot stage to obtain students' perceptions of using an e-portfolio (i.e. what are the roles and benefits of conducting reflection in an e-portfolio). An FGD with five faculty members was also conducted for data triangulation.

Data analysis
Data analysis was conducted following data collection, with the principal researcher (NG) acting as the moderator and interviewer in all indepth interviews and FGDs. All in-depth interviews and FGDs were audio-recorded and transcribed verbatim. All data obtained were then analysed using a thematic analysis approach in order to identify emerging themes and subthemes, followed by data reduction and member checking. 14 All respondents have signed the written consent forms before participating in the study. This study has obtained an ethical clearance from The Health Research Ethics Committee Faculty of Medicine Universitas Indonesia Cipto Mangunkusumo Hospital with the number of 0004/UN2.F1/ ETIK/2019.

Needs analysis stage
Two main themes emerged from the thematic analysis of FGD and in-depth interview transcripts conducted in the needs analysis stage: (1) Stakeholders' expectations of e-portfolio and (2) The features of e-portfolio.

Stakeholders' expectation of e-portfolio.
Most students had never heard of e-portfolio being used as a learning or assessment instrument in the field of dental education. "I've never heard of the use of portfolios in education before. All I know is that portfolios in design and architecture show one's work. So perhaps a portfolio in dental education works similarly to collect all the cases and treatments provided, thus showing our track records as junior practitioners." (needs analysis FGD 2-MI 4-p. 2) Most students and faculty members expected that an e-portfolio would mainly be used as an assessment and monitoring instrument that provides information on the quality and quantity of students' work in the clinics.
"e-portfolio for undergraduate clinical dentistry were expected to show not only the number of clinical requirements but also to evaluate someone's performance, how often he/she works with patients, and how good their work is to ensure that supervisors conduct proper supervision and assess students' professionalism over time." (needs analysis in-depth 2-p. 4) An assessment system for e-portfolio was also mentioned by students and directors.
There was a discrepancy between students' and directors' expectations of how e-portfolio should be assessed. Faculty members elicited the needs of summative assessment methods that showed students' abilities and professional development over time, as well as the ability to monitor students' progress in achieving the NCS.
"To date, we have relied upon the lists of clinical skills, some workplace-based assessment results, and length of study to determine students' final score. We had a hard time determining the quality of students' work over time; for example, in working with patients in a clinical requirement, sometimes students took a long time to finish the whole therapy. We could not determine the cause-was it a case that it was too difficult? Or did students lack knowledge, skills, or availability to patients? Or was it something else? We need an instrument that gives us the 'big picture' about students' performance in order to assess them as competent or incompetent." (ID 1-p. 4) However, students largely hoped for a formative tool that enhances feedback provision. Although students were mostly hesitant to have their self-reflection in the e-portfolio graded, they hoped that faculty members were assigned to continually provide feedback for their reflections. Students revealed that e-portfolio might be more suitable for formative assessments, thus allowing them to record and receive more feedback from assigned faculty members as preceptors to discuss their personal and professional development recorded in their e-portfolio over time.
"Sometimes we feel that we need to discuss how we are doing in the clinic. What has

been done and what to do next. During meetings with academic counselors, we can discuss thatbut we do not have anything to show them, so it is difficult to discuss our achievements and difficulties." (needs analysis FGD 4-PH2-p. 6)
Students noted that there were many existing rubrics for grading in the undergraduate clinical dentistry program that are not used effectively; therefore, additional grading systems, such as e-portfolio grades, might not work unless they replace all of the existing grading systems.
"Until now, there are plenty of papers we need to fill in. For example, when we treat a patient, we have to fill in their status as our documentation, assessment rubric, dental hospital medical record, registrar book. Later, during the administration session, we have to write this information in each department book.

If there is an additional document for assessment such as a portfolio, be it digital or manual, I don't think it will be used for a long time unless it substitutes other documents. And the thing is, when substituting something, there will be a time when we still have to do everything before it is fully substituted or implemented. It would be too much administrative work." (needs analysis FGD 1-EI 2-p.8)
Students also noted that they needed an assessment instrument that allowed them to evaluate their performance against the NCS, which would better prepare them for the exit exam via curriculum mapping and self-monitoring tools.
" Features of e-portfolio. According to the thematic analysis, the research team identified key features of e-portfolio that suited the basic needs of students and faculty members. The identified features are presented in Table 3.

Development and pilot stage
The research team discussed the emergent themes and, together with the professional IT developer, translated it into essential features of the e-portfolio. The exploration stage brought multiple technical features that were feasible to accommodate in this early development of the e-portfolio system, including non-real-time input, non-graded reflections, and opportunities for sorting and grouping according to learning activities, clinical cases, and NCS. The early developed e-portfolio was developed in web-based platform to provide the needs of accessibility from various type of devices. Moreover, self-reflection prompts were also added to the platform, as self-reflection distinguished portfolio from other assessment instruments.
Since the result of the needs analysis stage demonstrated that students feel reluctant to have their e-portfolio assessed, the research team decided that all learning activities and reflections submitted by students were neither graded nor submitted to supervisors' accounts to help the students engage with the process of writing selfreflections in the introductory phase of learning e-portfolio. Students were also given freedom in choosing which learning activities to submit into their e-portfolio. To further simplify the process, it was also decided that students were asked to submit narratives of self-reflection in regards to their achievement of one competency domain only, which was professionalism.

Evaluation stage
At the end of the six-week pilot program, students were reassembled to attend FGDs. Notably, three themes emerged from the FGD at the evaluation stage regarding students' and faculty members' perspective on the roles and benefits of e-portfolio: (1) students understood more about the content reflected (professionalism), students understood more about the steps and advantages of conducting self-reflection, and (3) students were able to develop action plans in learning.
Students understood more about the content reflected. Moreover, it was revealed that respondents obtained a deeper and wider understanding of the content reflected, especially regarding professionalism as the first domain in the NCS. 13 Students' previous understandings of professionalism were limited to medical ethics. However, after reflecting on it within the e-portfolio, students viewed professionalism as a wide domain with various components.
"I previously thought that professionalism was limited to treating patients according to the standard procedures. After I reflect on my competence in this platform, I realized that professionalism was beyond that-and that it includes communication skills, reflective skills, etc. It made me realize that I needed to learn and practice more." (ID 2-p. [3][4][5] Students understood more about the steps and advantages of conducting self-reflection. Students valued the prompts of self-reflection included in the platform. They noted that the platform not only helped them to evaluate their performance in terms of the list of activities submitted, but also in conducting self-reflection. "

DISCUSSION
Driessen 3 elicited three purposes of portfolio development: as an assessment instrument and as a learning, reflective, and monitoring instrument. The results of the needs analysis stage demonstrated that stakeholders' expectations for e-portfolio were mainly to accommodate the needs of an assessment and monitoring tool. Despite the lacks of knowledge and experience of stakeholders in using e-portfolio, the most frequently mentioned feature of an e-portfolio relates to its function as an assessment and monitoring instrument. The results of the present study are in line with a previous study conducted by Vernazza 15 , which noted that the use of e-portfolio in clinical dentistry was mainly for monitoring students' achievement of acquired clinical skills and assessment instruments. It also showed similarities regarding the popular features of e-portfolio, which include the section in which students recorded their clinical activity and feedback. 15 The results of the needs analysis stage also showed that the e-portfolio was expected to provide features for documenting the clinical cases and patient treatments provided by students. The need for an e-portfolio to support learning and self-assessment was also mentioned several times by students in this study, thus highlighting that some of the students have a good level of feedback-seeking behavior. 16 Therefore, the main purpose of e-portfolio development in this study was to support learning.
The use of e-portfolio as an instrument to support learning was essential to promote lifelong learning and help students to develop their personal and professional identity as a reflective dental practitioner. 17 To better support the use of e-portfolio as a reflective and learning instrument, students needed to have a sufficient understanding of the steps and benefits of conducting self-reflection. The provided training sessions conducted before the pilot stage introduced the basic concepts of self-reflection, while the reflection prompt feature used in this study represent the researchers' effort to support students' reflective learning. 18 To support its further utilization as a learning and reflective instrument, it is necessary to use e-portfolio longitudinally and integrate them into the curriculum so that they can support the habituation of students to continuously conduct self-evaluations and seek feedback regarding their performance. The developed e-portfolio has enabled students to independently enter and select their learning activities. Through the inseparable process of repeatedly selecting learning activities in an e-portfolio and conducting self-reflection, students can improve their reflective abilities. This process may also increase the participation and utilization of e-portfolio as learning and self-reflection instruments. 17,19 In addition, the formative and summative assessments of selfreflection made in the e-portfolio can also be conducted in accordance with the concept that assessment can support further learning. 20 However, the students' FGD results also demonstrate that most students were hesitant to have their self-reflection assessed, and instead preferred a hidden self-reflection feature. According to Hofstede's cultural model 21 , this may have resulted from the high power distance culture of Indonesian people. This resulted from high power distance culture, thus, causing students to feel reluctant to seek direct feedback on their self-reflection due to a fear of receiving negative feedback. This notion is also supported by the results of Oktaria and Soemantri 22 , which show that the unpreparedness of Indonesian students to receive negative feedback from a faculty member is a barrier for them to seek feedback in general. Therefore, since it is essential to provide a safe environment to prevent resistance during the pilot stage, there was no faculty member assigned to provide feedback or grade students' reflections.
The results also highlighted the lack of evidence submitted by students during the pilot stage. This may have been caused by the nonintegrated platform with other documents that were in hard copy form. The use of a non-realtime platform with participatory and voluntarily submission also played a role in the limited submission observed during the pilot stage. Despite the limited submissions, students valued the use of e-portfolio. Notably, three themes emerged from the evaluation stage which demonstrated that students obtained several benefits from utilizing e-portfolio as part of their learning experience in the clinical stage of their studies. Students obtained deeper knowledge regarding an NCS domain (professionalism) as a focus of reflection and understood more about its facets and subdomains. 12,23 Students also elicited the benefit of gaining a better understanding of self-reflection and its influence on their learning. The e-portfolio impacting students' learning was also mentioned in a previous study conducted by Tonni 5 , which noted that e-portfolio use increased students' self-awareness, motivation, and self-confidence in postgraduate training in orthodontics. The roles of e-portfolio that emerged from this study are also in line with results of a systematic review on the educational effects of portfolios in an undergraduate study conducted by Buckley. 24

Study Limitations
This research was an action research study conducted at one dental school in Indonesia. As such, the results are context-specific for the field of dentistry and cannot provide a description of the longitudinal use of the piloted e-portfolio system. However, this study provides evidence of best practices in learning e-portfolio development through an action research approach involving a needs analysis stage, development stage, and evaluation stage.

CONCLUSION
Performing a needs analysis prior to the development of an e-portfolio system provided both technical context regarding its development as well as an adequate supporting system, which was required prior to its implementation to increase its feasibility. The role of the e-portfolio developed based on the needs analysis stage showed a satisfactory impact on students' learning and motivation. The result of this study was presented to the corresponded stakeholder of dental school involved in this study.